Peer-review mentoring: a way to engage a more diverse pool of peer reviewers?

This week we are celebrating the peer reviewers who support their communities by evaluating the work of their fellow researchers to guide editorial decision making. But we also acknowledge increasing ‘reviewer fatigue’ – the fact that the same experts are called upon time and again to act as referees. In one study that modelled peer review supply and demand in biomedical journals, the authors concluded that “20% of the researchers performed 69% to 94% of the reviews”. Not surprisingly perhaps, people who publish more are asked to review more. This is possibly fair: they create more work to be peer-reviewed, so they should contribute more reviewing. However, this reduces the opportunity for early-career researchers to gain experience and confidence in peer review – potentially a lost opportunity, given that “first-time reviewers are more likely to accept and complete the task”. Moreover, even with a short publication record, these young researchers are likely to be thoroughly conversant with their own research areas and well qualified to provide an expert review. There is also a recognized geographical imbalance in peer review participation: in particular, the USA is overrepresented and China underrepresented in peer review compared with their research output. Similarly, a gender imbalance was noted by the American Geophysical Union.

What can be done not only to involve early career researchers but also to increase the diversity of the reviewer pool? Surveys have noted the need for peer review training and the lack of provision as barriers. In recent years, learned societies, journal editorial boards, publishers and institutions have developed peer-review resources and training programmes, and publications such as the Journal of Functional Programming have begun to provide specific guidance to young reviewers. Even with these resources, doing one’s first “live” review can be discouragingly daunting.

We all know that it’s not uncommon for a senior researcher to co-review with a junior colleague behind the scenes. This alleviates the burden on the senior colleague and also is a good induction into peer review for the less experienced researcher. However, many journals explicitly forbid this practice because of the confidentiality of the editorial decision-making process. Moreover, the person invited to do the review was selected for his or her specific expertise and experience. Guidelines from the Committee on Publication Ethics emphatically state: “Do not involve anyone else in the review of a manuscript (including early career researchers you are mentoring), without first obtaining permission from the journal.”

Some journals now acknowledge and explicitly encourage this informal mentoring, with the proviso that it must be transparent. It is also only fair to recognize the contribution of the junior researcher. The COPE guidelines go on to state: “The names of any individuals who have helped with the review should be included so that they are associated with the manuscript in the journal’s records and can also receive due recognition for their efforts.” Journals actively encouraging co-reviewing in this way include the EMBO Press journals, Methods in Ecology and Evolution, and the journals of the American Psychological Association. They insist that the senior researcher must retain overall responsibility for the review and must use the co-review process to coach the junior colleague. If the motivation for passing on the review is simply that they have no time, they should decline to do the review.

Some organizations have gone a step further and developed formal mentoring programmes. EULAR (the European League Against Rheumatism) has reported on a peer-review mentoring programme, giving recommendations for implementing similar programmes on other journals. Their report notes that “a significant proportion of previous participants have become independent reviewers” and that “these were young researchers (<40 years), with limited review experience, who became reviewers in the top journal of the field of rheumatology”. Other initiatives have taken place or are under way.

One of these is the programme to train young peer reviewers run by the Journal of the Developmental Origins of Health and Disease. At an editorial meeting in Cape Town back in November 2015, Dr Amita Bansal, herself a trainee representative on the Council of the DOHaD Society, proposed that the journal should provide opportunities to its trainee membership to become involved in manuscript reviewing. She then worked with the editorial team to develop the programme, from which she benefited herself. The programme has two benefits: providing a future source of competent referees as well as providing training and feedback to post-docs in the early stages of their career. The trainee reviewers learn by reading the reviews of the other referees, and the Associate Editor handling the manuscript may choose to pass on text or concepts from the trainee reviewer to the authors. The programme has proved highly popular with the post-docs who have participated, and senior PhDs are now approaching Dr Bamisal asking to be involved also.

Some have expressed doubts about the effectiveness of peer-review training programmes, including mentoring. It could well be that participants in these programmes were already motivated and had the confidence to do well. Other initiatives certainly appear to have been successful, at least according to some criteria. Even if the effectiveness of such programmes on the quality of peer review is not yet proven, at the very least mentoring is an opportunity for journal editors to engage with early career researchers and has the potential to encourage a diverse range of new reviewers to join the reviewer pool and gain confidence through learning on the job from experts.

Leave a reply

Your email address will not be published. Required fields are marked *