Post-publication peer review (PPPR) is something that we all would like to flourish, yet at the same time seem to struggle to find a proper format for. Arguably the most accessible and widely accepted form of PPPR is writing comments in the form of letters, or directly on a publisher’s website. However, Kent Anderson has recently discussed the problems associated with comments being used for PPPR, the main issue being a lack of substance and quality in such ´reviews´.
In search of new forms of PPPR, a new movement called ´altmetrics´ has established itself recently as a potential front-runner, basically switching the focus from traditional full-text reviews to widespread usage-based metrics, ranging from the more traditional citation counts, to more recent inventions, such as tweets, Facebook likes, etc. Unfortunately though, these metrics suffer from several drawbacks themselves, as discussed by David Crotty in one of his recent posts. Not surprisingly, the main issue identified was that these metrics are extremely hard to interpret, especially since one needn’t be an expert in the field (or even know what an article is about) to retweet its link to others for example.
The big question that remains then is: Does a form of PPPR exist that can provide a sufficient amount of substance, not be too demanding for the reviewer and in fact provide an incentive for the reviewer in itself? To answer this question, I suggest that we stop looking at public metrics for just a second, and focus on the reviewer. This person will in most cases will be a (semi-) professional scientist, meaning that the person will be spending a lot of time reading and evaluating the work of others, looking for gaps that could be addressed in future research.
In effect, every researcher spends half the time writing personal comments and reviews on published books and articles, which may partially end up in the “Introduction”, “State of the art” and “Literature review” sections of the researcher’s articles, books, dissertations, etc. There is already an abundance of quality PPPR out there, our task is to encourage researchers to make (some of) this material public, by providing the right tools and formats.
How can we do this? The short answer is: by providing even more value to the researcher. I’m still mulling over the details somewhat, so I’ll put off the full proposal for a follow up post which should appear later this week. I’d be happy to hear what your thoughts are so far, though I guess the solution is the most interesting aspect to all this.
WebScio 
Thanks for the post! On a personal level it is very timely, because two weeks ago my new paper appeared on PLoS ONE. Page views are a cinch, especially when there is press coverage associated with your paper, but substantive comments are (unsurprisingly) hard to come by. Yet PPPR happens all the time right under our noses in the countless, two-person email exchanges between academics. I naively thought it would be easy to bottle up the essence of professional email correspondence and relocate it to the comments section of my paper but it’s been like pulling teeth. Most common excuse by far is time (“limited bandwidth”) followed by technophobia. Even the idealistic trainees are strapped for time. I guess it just shows how much interpersonal relations have atrophied over the last decade of hyper-competition for publications, grants and jobs.
The only way to recover is to rebuild peer relationships one at a time. If every scientist seriously cultivated 5-10 diverse professional relationships — be they lab meetings, journal clubs, or collaborations — we would have the online scientific community we all want. (Except maybe Kent Anderson. That guy is hard to please).
-Ethan
“In effect, every researcher spends half the time writing personal comments and reviews on published books and articles, which end up in the “Introduction”, “State of the art” and “Literature review” sections of the researcher’s articles, books, dissertations, etc. There is already an abundance of quality PPPR out there.”
What’s the difference between this and keeping track of citations?
I do like the idea of cultivating relationships with peers. I use Twitter that way. In fact, that’s how I found your post (thanks @peerevaluation!).
Hi Britt,
Thanks for the comment. I guess I should have added some modifier like “partially end up in the .. sections of the researcher’s articles”. Mainly because privately, a researcher will likely write a more detailed review of the papers he reads for the sake of both remembering the details in the future and keeping track of his thoughts. Only a small portion of these will end up in an article though, due to space limitations and relevance to the usually very narrow focus of any one article.
Let’s see if we can get anywhere from here to make reviews more open :)
Cheers,
Martin
Pingback: Want to write a review? Pick your format of choice! | WebScio