Skip to content

Introducing Open, Post-Publication, Expert Peer Review on ALiEM


ExpertPeerReviewStamp2x200Today, we are busting open the concept of peer review for publications on blogs!

The peer review process has been criticized for its flaws, but is universally accepted as a necessary part of the scientific process. Peer reviewing allows experts in a field to determine the validity of a study or an article so that those of us who are less expert can reap the benefits of their knowledge. Until recently this process was almost universally pre-publication and anonymous.  Authors would go through months of review and revision based on feedback of experts whose name they didn’t even know. In the last decade journals such as BMJ Open moved to an open peer review process by divulging the reviewer’s identities to the author and publishing the reviews of the experts online for open access to all readers. This open peer review model prevents redundancy and encourages transparency in the scientific process.

Peer Review on a Post-Publication Basis

Even with open peer review, most reviews are done pre-publication. The majority of articles are eventually published, but authors might have their paper rejected by several journals before a journal accepts. This process of repeated pre-publication peer review can result in considerable delay to disbursement of knowledge.  For these reasons, post-publication peer review has become a hot topic. Post-publication review allows data to be made readily available to readers while simultaneously the publications undergo formal peer review.  Online blog sites are particularly primed for this sort of peer review since their purpose is usually to distribute knowledge in a timely fashion.

Blogs: The Ideal Platform for Post-Publication Peer Review?

Most of us, including those of us at ALiEM, believe that blogs have a built in informal post-publication peer review process since readers are likely in the same field and can comment easily and openly under each blog post. However, the lack of rigorous review by accepted experts in the field has undermined the validity of the online clinical blogs in the mind of others in the scientific community.

My Goal As ALiEM Resident Editor

My goal over the next year as the Resident Editor of ALiEM is to help transition the ALiEM articles to a more formal open post-publication peer review. This means that once the articles are published on the ALiEM website, we will solicit reviews from experts in the field.  The experts will be those with established credentials in the topic of interest or authors of the papers cited in the blog post. Once received, their comments will be posted at the bottom of the blog post with a bold red “Expert Peer Review” title separating the post from the review. Coupled with the fact that all blog posts are peer-reviewed by Dr. Michelle Lin (ALiEM Editor-in-Chief) on a pre-publication basis, we feel that this book-ended peer review approach will yield greater content accuracy and hopefully also reader engagement.

How will you know that a blog article has been “Expert Peer Reviewed”?

  1. We will label the top of the peer reviewed posts with our new “Expert Peer Reviewed” stamp, which may appear one day, one week, or one year after the original publication, depending on when we obtain an expert peer review response.


  2. You will be able to find the Expert Peer Reviewed articles in their original categories as well as in our new “Expert Peer Review” link under the “Clinical” drop down menu.EPR Screenshot 3n
  3. You can find the 5 most recently peer reviewed posts by viewing the right column widget called “Newly Expert Peer Reviewed”. This column will update whenever an expert peer review is added to a post. epr ss 4
  4. We will be adding a monthly blog post highlighting the articles that have been peer reviewed that month so that you can revisit your favorite posts to see what the experts said.  As usual, please take part in the dialogue by adding your comments below each blog post and feel free to direct questions to the experts.

You can find our first 5 peer reviewed articles here:

I look forward to getting to know the readers more over the next year. Please feel free to leave your comments and suggestions about our new project.

Natalie Desouza, MD

Natalie Desouza, MD

ALiEM Resident Editor
Emergency Medicine PGY-4 Resident
University of California San Francisco
San Francisco General Hospital
  • Corey Heitz

    This is a fantastic idea. Am looking forward to it!

    • Michelle

      Me too! Good thing Natalie is ALL OVER THIS!

  • Matthew Silver

    Brilliant!! The million dollar question is … what is the ACGME/RRC’s take on this and other peer-reviewed electronic resources? MedEd Portal is acceptable. Anyone poke this bee hive yet?

    • Michelle

      Ooh interesting. I for one have not poked the bee hive. My beekeeper outfit hasn’t arrived in the mail yet…

      I assume you are asking from an author’s CV standpoint. Say, if we got an “expert peer review” of one of your tricks of the trade (, could this be listed under a Peer-Reviewed Publication on your CV? I would say yes, since is it not a publication (publicly available document) which is peer-reviewed in a thoughtful and directed manner by an expert?

      Will send out a tweet to ask the collective. Great Q. Hope you are well in San Diego.

      • TChanMD

        It might help to have #ALiEM list very transparently a terms of reference for how we will recruit expert/peer reviewers? Maybe our need resident editor can help with leading us in a way to figure out how/who to review (e.g. via author recommendations, cited authors, etc..)?

        Another option is to have a bevvy of volunteers like #MedEdPortal does, and have ppl “recommend” others (essentially a snow-ball nomination process for reviewers)

        e.g. when I can’t review, MedEdPortal’s automatic reply is the following:

        “It is unfortunate that you are unable to review the resource at this time. I will keep you in mind for future submissions that fall under your area of expertise. In addition, we would like to ask your help as we endeavor to expand our pool of peer reviewers. While we are interested in receiving nominations for reviewers across all biomedical disciplines, we are particularly in need of reviewers for the following fields: Anesthesiology, Dermatology, General Surgery, Neurology, Ophthalmology, Pediatrics, and Psychiatry. If you have colleagues that might be willing to serve as a reviewer simply email their names and e-mail addresses to”

        Something like this might be good?

  • fedUpRuralDoc

    Brilliant – this is the way of the future!

    One has got to beware of the cult of personality – so limiting to established experts and authors of cited papers seems to make sense. Similarly there’s the risk of confirmation bias – so having open comments (as always) makes this a robust method AND is consistent with FOAMed.

    Well done

    • Michelle

      I agree that getting both crowd-sourced reviews and expert peer reviews on a post-publication basis are different and still equally valuable. Well said.