PaperCritic basics in place – Where to next?

Since the public launch of PaperCritic was announced one and a half months ago, we have been working hard on making the basics of the app as solid as possible. In particular, with the help of some very valuable comments from our first users, we were able to identify and solve the majority of the low-level usability issues of our site, such as being able to access one’s Mendeley library, being able to easily access a list of own reviews, as well as edit and delete these.

As ever, we are not going to sit and wait for something to happen on its own and so will continue improving our app as best we can. Given the number of very encouraging and positive comments about PaperCritic that we received over the better part of the last two months, we would like to take this opportunity to ask our potential users:
Where to next? What features should we add to our site next in order to make it more attractive and usable for everyone?

The following is a list of the features that were already requested (in no particular order). Which of these do you think should be addressed as soon as possible (and which maybe put on hold for now)? Can you think of something essential that is seemingly missing from the list? Do let us know!

  • Email subscriptions: Users should be able to subscribe to reviews on papers that interest them (this is already possible via RSS on a paper basis, though not cumulatively). In addition, let users auto-subscribe to the documents from their Mendeley library.
  • Pre-publication review: Note that this is actually possible as Mendeley has an “Unpublished work” document type, but it might be useful if unpublished works and their reviews were more explicitly separated from post-publication reviews on the site.
  • Trackback system: While we hope that users will take the time to come to PaperCritic to submit their reviews, we also want to be the hub for all mentions about a paper – allowing for bloggers and tweeters out there to send trackbacks to PaperCritic will allow us to aggregate all related content about a paper and become a central repository of public opinion in scientific publishing.
  • API: This is probably the most common request for every app nowadays – there should be an API that would allow publication houses, bloggers and others to fetch every review on a given paper via its Mendeley UUID, DOI or similar identificator.

The list can be clearly extended quite a bit, but these seem to be the most relevant points for now. As mentioned above, if you agree/disagree or have alternative suggestions – do let us know! However, I would like to stress out that this post is intentionally only touching on technical features of the site – attracting users and increasing the impact of post-publication reviews will be discussed shortly.



Filed under PaperCritic

8 responses to “PaperCritic basics in place – Where to next?

  1. i would love to:
    – be able to upload my annotated copy of the paper (such as i create in iAnnotatePDF when reading it, and from which i normally write my reviews), OR i suppose papercritic could provide a way for me to annotate papers with your own tool.
    – see reviews from colleagues i know/trust their opinions
    – form reading groups, eg. for a class or a reading group at my university.
    – see a tag cloud of the top words used in the review
    – let people

    with regards to the first suggestion, and from the standpoint of a developer and a grad student, IMHO it is important to think about peoples’ current workflows, and alleviate pain points or make sure to integrate with the part of their workflow that IS working for them. how do people currently review papers? how can it be easy for them to get their content into your site, and what will bring them back?

    for example, something else that would be AMAZING and would definitely bring me back regularly is a reading list for PDFs. in theory mendeley offers this right? i mean, i can save papers their from a bookmarket. but it is slow and clunky, and requires immediate attention which is totally distracting from my workflow. but compare the experience of saving a paper in mendeley to saving an article with instapaper… worlds apart. instapaper’s bookmarklet is incredibly lightweight. it doesn’t distract me from what i’m currently doing, it Just Works (TM), in the background, and i can go to it later. it syncs offline so i can then read the articles on my ipad on the metro. etc.

    anyway, i’m sure everyone has slightly different workflows and reading habits. my point being to keep making sure you tap into those as you build…

    • Hi Jessy,

      Thanks a lot for your comments, these are some very interesting points!

      The point that you make about plugging into people’s workflows is certainly something I was thinking about as at the moment it’s slightly cumbersome to go into PaperCritic and post one’s reviews there in addition to whatever other paper management tool you’re using. Of course ideally, if PaperCritic were integrated directly into Mendeley, this would all not be a problem, but for now I guess we need to think of something else.

      Having the ability to connect with colleagues and form reading groups is also something we could quite surely look into – especially since it’s already possible to make contacts and form groups at Mendeley, all we need is to plug into their API some more and represent the connections at PaperCritic.

      As for a reading list of PDFs – I quite understand the feature, but it feels as if it falls much more into Mendeley’s domain – as you’re aware it’s already possible to do that over there and iirc the bookmarklet doesn’t actually require immediate attention, apart from clicking on the X to close it – the document will be saved as ‘unread’ in your library anyway, which is pretty much what you want.

      As always, we’ll be trying to bring you these features as soon as possible, so stay tuned for updates. If you have any more suggestions, please do not hesitate to contact us again ;)


  2. Dom


    I’ve just had a look at your app and it looks like it has great potential.

    As a public health professional I use a range of critical appraisal tools to review papers such as the CASP frameworks for epidemiological studies.

    Is there any way to customise PaperCritic to include the headings used by these types of review tools rather than just a large comments box?



    • Hi Dom,

      Thanks a lot for checking out PaperCritic and the kind word!

      Could you give me a link to the CASP framework or an example of such a review maybe please just so that I would get an idea of what you’re asking for.

      I need to see that first of course, but in general I can imagine that there are a lot of different ways of reviewing papers, judging by the reviews I got for my papers and how I’ve seen other people do it – there is sadly no standard there and it’s very hard to accommodate every field that people use.. I’m just saying that in general, that we might need to introduce more standards or find some fluid solutions for different review formats that people use.

      Thanks again,

    • Hi again,

      I’ve been asked a similar question recently regarding the possibility of integrating different frameworks into PaperCritic and I’m just wondering if you could tell me if you are reviewing papers using CASP for example for yourself or for some repository/company, and if the latter is the case – if the reviews are then usually public or private afterwards.

      Basically I’m just trying to figure out what people do, and if we can help such groups by addressing some specific aspects of our tool.


      • Dom

        Hi Marty,

        Our critical appraisals are most frequently used in day to day practice to guide public health decision making and healthcare commissioning. Different people may have slightly different perspectives of course. I’m not aware of a single repository of critical appraisals for papers at present, but it might be a good idea in certain areas. It would be helpful to collaborate on and share reviews as often the critical appraisal process is done in a group. The other use might be in the development systematic reviews, e.g. Cochrane reviews?

        Thanks for showing interest in this idea!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s