A proactive approach to making peer review more transparent

It seems that every second blog post written, talk given, or tweet tweeted on the topics of science and research nowadays is in one way or other concerned with the shortcomings of the established peer review process and the ways in which one could evaluate papers more openly and transparently. For now, the discussions usually get stuck at the crossroads between needing to publish in high impact publications and the fact that these are more often than not rather “old-fashioned” when it comes to their peer review practices. But isn’t it in our hands, as scientists, to change this?

Admittedly, the reviews we get are anonymous and often not full of praise to put it mildly – yet that shouldn’t stop a researcher from wanting to share the review with others, provided there is an easy technical solution for doing that. If you think about it, if the review is very (un-)helpful, it’s in everyone’s interest to share it for the sake of exhibiting how (bad) the reviewers of a particular publication are. If the review reflects really well on the submitted paper, then there seems to be a natural impetus for the scientist to share it. And even if the review is very critical of the paper (in which case scientists tend to switch into full-on denial mode), it’s still in your interest to share the review and collect more opinions on the paper in order to understand if there is really a flaw in it, or if the reviewer was just being unprofessionally picky for some reason.

Going somewhat in this direction RePEc have recently issued a call to editors to submit the reviews that were written by the referees of their journals for open display. Even if the idea is very much biased towards whatever the editors deem as appropriate for showcasing, I applaud the effort. However, the scientists themselves should be also more proactive in this area and just publish the reviews that they obtain!

PS. There might be a slight legal caveat to this idea, so I asked an organizer of an event if he would see a problem in me publishing the reviews I got on submitting for presentation. The answer I got was that while it’s something they probably can’t do anything about legally, the reviewers might get “annoyed if what they wrote appeared on the Internet unexpectedly”. Well (especially considering that this would still be done anonymously) I don’t see a reason why they should be, unless they’re ashamed about what they wrote.

Advertisements

7 Comments

Filed under PaperCritic

7 responses to “A proactive approach to making peer review more transparent

  1. Having had some interesting discussions on Twitter, I took it upon myself to ask some journal editors what they think about the possibility of publishing reviewer reports. Here is what the editor of one SAGE-published journal tells me:

    “The peer review process is confidential in nature and reviewers tend to write their comments under the assumption that they will be read by the editors and the authors only. So the question of copyright doesn’t directly apply since like all confidential reports the text are not supposed to be disseminated publicly (anonymously or otherwise).

    If there was some good reason for you to want to make public some of the text of any reviews you get (this is not usual), I believe that the right procedure would be for the editors or SAGE to first ask the reviewers to see whether they agree to this.”

    I think the main issue here is how you define things – I guess an alternative word to “confidential” in the above could be “closed” and then the nice argument kind of falls apart in terms of being defensible in open science.

  2. Some links to different people’s opinions on this:

  3. Amy

    Has anyone ever considered being able to post grant proposls for review? What would be the implications of this?

    • I would imagine that these are really much more private, as ideas are so much easier to steal than results (go prove that you had the idea first..).

      • Amy

        That’s true, but if you post a proposal pehaps that’s proof it’s your idea? All I can say is that I have already been involved with a grant competition that required proposals to be submitted by posting them so that they were instantly publically available–before the review panel selected awardees. And anyone could comment on them! I know the peer review of journal articles is a bit different, but really the peer review of grant proposals has been questioned for a long time too…perhaps OA will be cradle to grave…?

  4. If someone repeatedly submits bad reviews, it’s likely that that man wont be asked to examine a manuscript again. Being invited to review a manuscript is an honor, not just because you are now being identified for your eminence in a particular location of investigation but also because of the obligation and support you offer to the journal and scientific community. The purpose of the article is to determine how best to peer evaluation an article.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s