The Ethics of Peer Review: Principles (3/5)
(May 13th, 2016) Peer review is at the heart of research communication. However, the process is far from perfect and many problems have been identified. Among them, ethical questions about the responsible conduct of reviewers who can wield considerable power behind a cloak of anonymity.
Many of the problems with peer review result from the personal behaviour of the reviewers themselves. Are they fair and impartial in their critiques? It has been suggested that reviewers would do their job much better if they were made more aware of their ethical responsibilities, detailing what is expected of them by the scientific community as a whole. A good example of this approach is found in “Ethics of Peer Review: A Guide for Manuscript Reviewers”. Sara Rockwell says that reviewers must behave ethically because peer review is a “cornerstone” of modern science and medicine. If they don’t, the whole of science suffers. “This body of unpaid, and often unidentified, volunteers offers a collective opinion on the expected standards of scientific rigour for the discipline. Their opinions on such matters as which techniques are current, valid and appropriate; how data should be analysed and presented; and how rigorous authors must be or how speculative they can be in the interpreting their data become de facto standards of the field.”
To put reviewers in the right ethical state, she presents a list of questions they should ask themselves during the reviewing process. For example:
When deciding whether to review a paper: Do you have the expertise the editor is looking for? Is the work too close to your own? Do you have any real or apparent conflicts of interest? Do you have the time to review the article within the time frame requested by the editor?
Once you have received the paper: Does seeing the full manuscript change your thoughts about your ability to review it? How do you handle the paper? Can you pass the paper on to someone else to review?
As you read and review the paper: Can you contact the author about the work or the paper? Can you seek help with your review?
“The Peer Reviewer's Oath”
Along similar lines, several oaths have been proposed to be taken by peer reviewers. The first we’ll consider is appropriately titled “The Peer Reviewer's Oath”. This is presented in 3 sections with 14 points that encourage the reviewer to be well-organised so that they can be clear and direct and efficient and honest when they review manuscripts. The Peer Reviewer's Oath proposes a scheme of systematic rigour to be applied each and every time in order to ensure a “fair and equal treatment” and to prevent negligence and careless errors.
For example, when first entrusted as the peer reviewer of a paper:
“I will immediately design a schedule (how many pages per day, etc.) to allow an initial reading, with ample time left for re-readings and analysis.
I will judge the paper for its merits. Whether the author is a Fields medalist/Nobel laureate, or an obscure graduate student, I will judge the paper the same.”
Later, when writing the review:
“I will write clearly. I will communicate my thoughts clearly and concisely, as if writing my own paper.
I will treat the author with professional respect and dignity.
I will not use subjective/unfalsifiable judgments about ‘interestingness’ as the sole reason for rejection.”
And, having written it:
“I will carefully proof-read the review before submitting it.”
“The Reviewers’ Charter”
One of the earliest calls for a reviewers’ oath appeared in 2012. Although it is referred to as “the Reviewers’ Charter”, authors Mariann Bienz and Kathy Weston explain that it is meant to be “a sort of scientific Hippocratic Oath” where scientists could publicly declare their intention to review papers as fairly and rapidly as possible.
The Charter proposal was linked to the newly-launched open access journal, e-Life. This was announced as a way to “reform” the current journals system with the prominent backing of the Wellcome Trust, the Howard Hughes Medical Institute and the Max Planck Society. Bienz and Weston suggested that they should also “reform the reviewing system” since “it would be marvelous if we lived in an open access world in which papers were properly and fairly reviewed and published in the shortest possible time.”
Two main problems they highlight in the current “fairly lousy process” are: the lengthy gap between submission and publication of a paper due to taking too long in the reviewing stage. “Delaying publication delays scientific progress, hinders careers and is poor value for money.” Secondly, they point to further delays caused by reviewers who ask for a long list of additional experiments. “Reviewers often come to a paper with the attitude ‘what else could be done?’ rather than ‘what is essential to support the claims in the paper?’”
Their Reviewers’ Charter calls for reviewers to commit themselves to six principles:
1. To determine whether the data support the conclusions of the paper, and if they do, to facilitate publication of the paper as rapidly as possible, i.e. decide whether a yes-no decision is appropriate, or whether a yes-if decision is necessary.
2. To request further experiments only as a last resort, and only if they are essential to validate the conclusions of the paper. No experiments extending the study beyond its conclusions, or with unreasonable cost or time implications, should be proposed. An estimate of time required for the additional work should be provided.
3. To review papers within two weeks, and to decline to review at once if this is not possible.
4. If asked to re-review a paper, to assess the revision solely on the criteria specified in the initial reviews.
5. To request the editor’s permission before asking a lab member to review a manuscript, to state the involvement of the lab member in the review, and to ensure adherence to the charter guidelines – “the enthusiastic dismembering of a paper is fine for journal club but not for the peer review process”.
6. To undertake to actively campaign for journals to introduce a policy of cross-reviewing – asking reviewers to comment on each other’s assessments of the paper in order to counteract any personal bias, misinterpretation of the data, or non-adherence to the reviewers’ charter.
By signing up to this Charter, Bienz and Weston say that scientists would be agreeing to adhere “to a set of easy-to-follow and easy-to-absorb guidelines, and to act as ambassadors to spread these principles amongst their colleagues, especially their junior ones who might still be new to the game.” As ever, there is the difficulty of getting formal recognition for their Charter, but the authors suggest that if it were accepted, it could become “a stipulation of funding bodies that grant holders should sign up for it. Journals could use the Charter list to select reviewers, and there could also be monitoring of recusants. Being a Charter signatory could become an essential part of being a practising scientist.”
However, their Charter does not consider the possibility that reviewers should openly reveal their identities to authors. Part 4 looks at advocates of ‘Open Review’ who insist that this is a vital reform for the peer review process.