The Ethics of Peer Review: The End? (5/5)

(May 19th, 2016) Peer review is at the heart of research communication. However, the process is far from perfect and many problems have been identified. Among them, ethical questions about the responsible conduct of reviewers who can wield considerable power behind a cloak of anonymity.

The former BMJ editor, Richard Smith, explains how BMJ tried various experiments to improve the quality of peer review including open peer review. The first was to ‘double blind’ the process. The reviewers were still anonymous, but now the authors were too - the reviewers did not (in theory) know who had authored the manuscripts they were reviewing. However, Smith said this provided no evidence of improved reviews.

Therefore, in 1999 they started open peer review. Both the authors and the reviewers knew each others’ identities. The main argument in favour of open review is the assumption that reviewers will behave more responsibly if they publicly sign their reviews (see Part 4). But critics of open review say that being aware of the position in the scientific hierarchy of reviewers and authors will influence reviews - junior scientists may be discouraged from critically reviewing the work of more senior researchers for fear of reprisals. Conversely, a review by a highly respected researcher will carry more weight than the opinion of somebody whose work is not so highly rated.

BMJ extended open peer review from revealing reviewers’ identities to publishing their reviews and correspondence online. In 2011, it launched the journal, BMJ Open, which publishes articles with the names of peer reviewers, the review reports and previous versions of manuscripts as pre-publication histories. Smith said they hoped, as a result, that peer review would be transformed “from a black box into an open scientific discourse”.

Since 2012, F1000 Research also makes all peer review reports and reviewer names public – even for articles that are still under review or revision. Furthermore, it says it wants to make it easier for reviewers to take recognisable credit for the quality of their reviews, for example, by providing unique identifiers (DOIs) for their reports. F1000 Research argues that placing everything in the open can be helpful for everyone since reviewers are usually in a position to put the work in a broader context of the field, and often mention this context in their reports. “They can also point out where the work could be expanded into new areas, and may still have some lingering questions. All of this is useful for everyone to read – not just the authors. It’s also important to remember that not all journals use the same criteria for publication. Some journals may turn a great paper down just because it doesn’t fit the scope of the journal. Other journals publish all sound science, including some papers that get extremely high praise in the referee reports.” 

Post-publication review - The end of peer review as we know it?

In a sense, when one realises how relatively recent the peer review process is, it becomes easier to think of alternatives. Some scientists are already advocating that we abandon pre-publication peer review completely and instead allow publications to appear directly online without any prior peer review. Subsequently they can be judged online and reviewed by the community as a whole. Smith agrees that given the problems of pre-publication reviews, one solution is to publish first and then allow all those who are interested to read and judge the contents – “a very quick and light form of peer review, then let the broader world critique the paper”. The outstanding example here is ‘arXiv’, the online repository for maths and physics, which has been disseminating studies without the hassle of peer review since 1991; by 2014, it was receiving more than 8000 submissions a month.

F1000 distinguishes three models of post-publication peer review:
1. Review by formally invited reviewers, after publication of the un-reviewed article (as used by F1000 Research);
2. Review by volunteer reviewers, after publication of the un-reviewed article - in this case the reviewers are not invited by the journal;
3. Comments on blogs or third party sites, independent of any formal peer review that may have already occurred on the article; for example PubPeer allows anonymous researchers to comment on any article with a DOI, or those published as preprints in arXiv 

Certainly such models of scientific publication represent a huge change from traditional peer review. It remains to be seen whether biology will embrace post-publication peer review sites similar to physics’ arXiv. Two pre-print archives for biology were launched in 2013, Peer J Preprint and BioRxiv. But in 2014, PLoS’s Martin Fenner told LT that he didn’t think this model had a “high chance of success in the life sciences”.

New ethical problems may arise from opening up the reviewing process. When anyone can go online and be part of the publication reviewing process, there is no guarantee that they will behave any more ethically than the traditional peer reviewers. Indeed, given the formality of the reviewing process, traditional reviewers are likely to be more acutely aware of their ethical responsibilities than researchers who make quick comments on journal websites. Some worry that an open reviewing process over the internet with online comments might fall victim of the kind of anonymous attacks found on other internet discussion sites and blogs, for example, “trolling”.

Nevertheless, open peer review also exposes the reasoning behind the publication of research articles. It allows readers to compare the critique side-by-side with the manuscript and provides the possibility of more informed decisions about the article. “Open peer review allows the reader to take more responsibility for what they read”. It remains to be seen if the transparency of open peer review will make reviewers act more “ethically”, but knowing that careless and unfair reviewing may be publicly exposed might have a more stimulating effect on reviewers who disregard the ideals set out in the ethical oaths.

It has even been suggested that the recent development of post-publication reviews could mark a permanent change in how science is communicated. Perhaps in a near future, the system of peer review that has risen over the last half-century will be remembered as a defunct model belonging to “a peculiar period in the history of science, an aberration produced by an explosion of researcher productivity and the constraints of print publication, eventually superseded by a fuller, nonstop scientific conversation.”

Jeremy Garwood

Photo: Arnold

Last Changes: 07.05.2016

Information 4

Information 5

Information 6