Open or Blind?
(December 11th, 2015) Do you think the peer review process is fair? How should the reviewers be picked? A recent study examined whether the scores and quality of reviews were affected by the author suggesting the reviewer, or by the reviewers name being known to the authors.
Most scientific journals use one of two peer review methods. The first is known as “single-blind”, where the authors’ names are known to the reviewer but the identity of the reviewer is anonymous to the author, and the second is “open”, whereby both parties know each other's identities. Furthermore, some journals will ask authors to suggest reviewers, whilst others do not.
A paper published in September in BMJ Open offers the first comprehensive look at the effect of author-suggested and non-author-suggested reviewers in journals operating on open or single-blind peer review models. The study was led by Maria Kowalczuk, Biology Editor at BioMed Central (BMC). Maria told me, “Open peer review is becoming more widely used in biomedical journals” and “we operate open peer review on all 45 medical journals within the BMC series”, whilst “the majority of our 290+ journals have adopted the ‘traditional’ single-blind model.” As yet, however, the best method is disputed, especially as to whether asking authors to suggest reviewers is a good idea. Recently, several papers, also at BMC, had to be withdrawn when it turned out, the suggested reviewers were fake.
Kowalczuk and colleagues directly compared 200 reviewer reports from BMC Infectious Diseases (open) and 200 reports from BMC Microbiology (single-blind). They also compared 200 of each type from the Journal of Inflammation. Some of the criteria, the authors were looking for concerned differences in acceptance rates and the “quality of reviews”, which were rated from 1-5 for various components relating to politeness, constructiveness and the extent to which different points were discussed. Of course, there is also a third review method known as “double-blind”, whereby the authors’ names and reviewers’ names are unknown to each other but this method wasn’t touched in this paper.
So, what did they find? The authors showed that “open peer review reports for BMC Infectious Diseases were of higher quality than single-blind reports for BMC Microbiology”. However, one has to keep in mind that these are journals covering different subjects, and for the Journal of Inflammation, which employed both methods, there was no difference in review quality. Maria said, “Although we have found that open peer review is slightly better than single blind (...) the difference is not so large as to undermine the single-blind model.”
So what about acceptance rates? Perhaps not surprisingly, the results showed that for each journal, author-suggested reviewers “were significantly more likely to recommend acceptance, irrespective of the peer review type”. According to Maria, it should, however, be noted that “editors tend to agree more often with the recommendation of the reviewer, who was not suggested by the authors”. Additionally, for the Journal of Inflammation, the reviews obtained in open review were more positive (recommended acceptance rate: 67%) than with single-blind review (50%). The authors don’t think it has to do with differences in the peer review method, since the journal is relatively small and hence, reports spanned a large time period (2007-2011).
As for BMC, it doesn’t look like the publisher has plans to change all their journals to open peer review just yet. “It is ultimately the editor’s decision, which peer review model they prefer to use in their journal (...) Different scientific communities have different preferences for the peer review models they use (...) In some disciplines, the double-blind model is seen as the most objective, while in other (perhaps smaller) fields it may be practically impossible to blind reviewers to the identity of the authors.”
Maria also believes “that peer review is here to stay as the innovations that have been proposed and tested within the last few years, such as post publication peer review or decoupled peer review, seem to complement the traditional process rather than replace it”. She believes that “peer review is understudied” and hopes that the BMC’s “new journal, Research Integrity and Peer Review, will also stimulate other groups to do more research into peer review, publication ethics and standards of reporting”.
So which method do you prefer? Maybe your opinion as an author and a reviewer are different? Would you be worried that if you were to give someone a bad review, they may get annoyed and retaliate by reviewing your paper unfavourably? If you were perhaps a younger, less experienced reviewer, would you feel pressurised to give more senior people in the field good reviews?