Last week, I summarized some of the main issues with the recent approaches to post-publication peer review (PPPR), such as comments and different types of altmetrics. The conclusion was that, while these approaches have a certain amount of potential, they usually fail to reach a high quality level, predominantly due to their very short and unstructured nature.
As mentioned in the previous post, most researchers will be writing summaries and reviews of research articles on a regular basis, yet these will usually be kept private. And even if one were to find an incentive that would drive researchers to publish these reviews, they would in most cases be of little value to others. The reason for this is that every researcher has his own particular focus of interest and will write his review accordingly, meaning that there will be a lot of conflict between researchers, particularly ones coming from different disciplines. The solution to the problem, in my opinion, lies in making reviews more systematic even at the personal level.
The need for systematic reviews has so far been relatively well addressed only in the health care section, with the establishment of the Cochrane Collaboration and Cochrane Reviews in the 1990′s. Unfortunately, other disciplines have so far failed in developing a similarly structured format for reviewing its advances. However, I would not necessarily see this as a failure of the disciplines, but as an indication that it can be simply infeasible to agree on any one particular review format for a specific research area.
In fact, there is no reason why smaller fields of study should not be given the opportunity to develop their own review formats that would have a higher chance of being adopted by its scholars because they would provide more value to both review writers and readers, than any generalized review format could ever do (from this perspective, a free-text review is basically as generalized a format as it gets).
In order to facilitate the above, PaperCritic will be rolling out a review template system this Summer that will enable its users to collaboratively define review formats that are most meaningful for their particular field of study. Furthermore, we will also be launching both public and private interest groups, aimed at bringing together scholars from one disciplines and sharing their views on the latest research in the field.
Ultimately, the hope is that having more structured reviews will make it significantly easier for scientists to both produce and consume these, thus making it a much more enjoyable process. Additionally, having a set of established review templates available in one’s field might just provide the needed encouragement for researchers to publish their reviews by alleviating the fear of not having composed the review well enough.
As ever, comments and suggestions are more than welcome as this is really a proposal and work in progress. Other than that, we’re looking forward to bringing the review templating system to you as soon as possible, so keep following us here and on Twitter.