Open Access & Peer Review Under the Microscope

xkcd cartoonist Randall Munroe depicts the accelerating pace of scientific publishing and the rise of open access publishing for a special issue of Science.

A recent article in Science, “Who’s Afraid of Peer Review? made available via open access by AAAS, has generated a lot of discussion around the complicated question of peer review in journals, with particular focus on the many new open access journals.   The article reports on a test of the rigor and validity of peer review in OA journals, and its results have been receiving a lot of coverage.  The author wrote a fake paper with a faulty research method and submitted it to open access journals (reminiscent of the 1996 ‘Sokal affair‘ in which researcher Alan Sokal submitted a nonsense paper to Social Text, an academic journal of postmodern cultural studies), then revealed it as a hoax on the date of publication.  

In this most recent study, researcher John Bohannon submitted a paper with obvious experimental design flaws claiming anti-cancer properties for a natural product derivative to 304 open access journals; it was accepted at over half of them.

The author did not send the paper to subscription-based journals (those with a pay wall to readers); his study method has been criticized accordingly, because it does not provide a valid comparison of peer review between OA journals and a comparable range of journals with different (i.e. subscription-based) business models.  Peer review is undertaken by volunteers, so it does not follow that the business model is a good predictor of the quality of peer review.  Open access publication has many benefits to science and society, but in these early years, there is no question that we see an abundance of new journals with questionable practices.

Interestingly, the range of journals that accepted the paper included open access journals published by reputable commercial publishers, including Elsevier, Sage, and Wolters Kluwer, as well as those published by recently emerged opportunistic publishers seeking to profit from the open access movement.

Those who read the whole paper will see that a scant 18% of the journals listed on one of the resources, Beall’s list of likely predatory OA journals, rejected the paper, while a full 82% accepted it, – good news for those who rely on that watchlist as an aid to screen out questionable journals.  The author concludes, “Some say that the open-access model itself is not to blame for the poor quality control revealed by Science‘s investigation. If I had targeted traditional, subscription-based journals, Roos* told me, ‘I strongly suspect you would get the same result.’  But open access has multiplied that underclass of journals, and the number of papers they publish. ‘Everyone agrees that open-access is a good thing,’ Roos says. ‘The question is how to achieve it.'”

*David Roos is a biologist at the University of Pennsylvania.

NOTE: The question of peer review in open access journals is an important one.   The Library provides support for those interested in OA publishing by helping them evaluate open access journals See the  Scholarly Publishing and Communication research guide, particularly the section on OA Journals and Publishers.

Barbara DeFelice, the Library’s Director of Scholarly Communication and Digital Resources programs, contributed to this blog post.

Also of interest:

Couzin-Frankel, J. (2013). Secretive and Subjective, Peer Review Proves Resistant to Study. Science, 341(6152), 1331–1331. doi:10.1126/science.341.6152.1331
A brief news article on peer review in last week’s Science magazine.   Though focusing on biomedical publishing, many of the issues raised are common to other scientific disciplines and have been the subject of discussion and controversy for years.

And from years past …
Enserink, M., 2001. Peer Review and Quality: A Dubious Connection? Science 293, 2187–2188.
I Hate Your Paper | The Scientist Magazine® [WWW Document], 2010. The Scientist. URL http://www.the-scientist.com/?articles.view/articleNo/29962/title/I-Hate-Your-Paper/ (accessed 9.22.13).
Mccook, A., 2006.  Is Peer Review Broken? | The Scientist Magazine® [WWW Document]. The Scientist. URL http://www.the-scientist.com/?articles.view/articleNo/23672/title/Is-Peer-Review-Broken-/ (accessed 9.22.13).

One thought on “Open Access & Peer Review Under the Microscope

  1. PRE-GREEN FOOL’S GOLD AND POST-GREEN FAIR GOLD OA

    I would be surprised if there weren’t subscription journals that would have accepted the Bohannon bogus paper for publication too.

    But I would be even more surprised if as high a proportion of subscription journals — matched for field, age, size and impact-factor — would have accepted Bohannon’s bogus paper as did the pay-to-publish OA journals (“Gold OA”).

    Subscription journals have to maintain enough of an appearance of peer review to sustain their subscriptions. Pay-to-publish Gold OA journals just have to maintain enough of an appearance of peer review to attract authors (and maybe the lure of pay-to-publish is enough to attract many authors in our publish-or-perish world without even the appearance of peer review, especially when the journal choice is justified by the fashionable allure — or excuse — of the journal’s being an OA journal).

    This problem would not be remedied by just lowering Gold OA journal publication fees.

    Nor is it a systemic problem of peer review.

    It is a problem of peer review for pay-to-publish Gold OA journals at a time when there is still far too little OA and most journals are still subscription journals, most authors are still confused about OA, many think that OA is synonymous with Gold OA journals, and, most important, there are not yet enough effective mandates from research funders and institutions that require authors to make all their papers OA by depositing them in their institutional OA repositories (“Green OA”), regardless of where they were published.

    If it were mandatory to make all papers Green OA, authors would simply deposit their peer-reviewed final drafts in their institutional OA repositories, free for all, immediately upon acceptance for publication. They would not have to pay to publish in Gold OA journals unless they especially wished to. Once all journal articles were being made Green OA in this way, institutions would be able to cancel all their journal subscriptions, which would in turn force all journals to cut costs and convert to Gold OA publishing at a much lower fee than is being charged now by OA journals: post-Green Fair Gold instead of today’s pre-Green Fool’s Gold.

    But, most important, the reason the Fair Gold fee would be much lower is that the only remaining service that journals (all of them having become Gold OA) would be performing then, post-Green, would be peer review. All access-provision and archiving would be done by the global network of Green OA institutional repositories (so no more print or PDF editions or their costs). And for just peer review, journals would no longer be charging for publishing (which would then just amount to a tag certifying that the article had been accepted by journal J): they would be charging only for the peer review.

    And each round of peer review (which peers do for free, by the way, so the only real cost is the qualified editor who evaluates the submission, picks the referees, and adjudicates the referee reports — plus the referee tracking and communication software) would be paid for on a “no-fault” basis, per round of peer review, whether the outcome was acceptance, rejection, or revision and resubmission for another (paid) round of peer review.

    Unlike with today’s Fool’s Gold junk journals that were caught by Bohannon’s sting, not only will no-fault post-Green, Fair-Gold peer-review remove any incentive to accept lower quality papers (and thereby reduce the reputation of the journal) — because the journal is paid for the peer review service in any case — but it will help make Fair-Gold OA costs even lower, per round of peer review, because it will not wrap the costs of the rejected or multiply revised and re-refereed papers into the cost of each accepted paper, as they do now.

    So post-Green Fair Gold will not only reduce costs but it will raise peer-review standards.

    None of this is possible, however, unless Green OA is effectively mandated by all institutions and funders first.

    Harnad, S. (2013) The Science Peer-Review “Sting”: Where the Fault Lies. Open Access Archivangelism 1059 http://openaccess.eprints.org/index.php?/archives/1059-.html

    ________ (2010) No-Fault Peer Review Charges: The Price of Selectivity Need Not Be Access Denied or Delayed. D-Lib Magazine 16 (7/8). http://eprints.ecs.soton.ac.uk/21348/

    ______ (2007) The Green Road to Open Access: A Leveraged Transition. In: Anna Gacs. The Culture of Periodicals from the Perspective of the Electronic Age. L’Harmattan. 99-106. http://eprints.ecs.soton.ac.uk/13309/

    ______ (1998) The invisible hand of peer review. Nature [online] (5 Nov. 1998), Exploit Interactive 5 (2000): and in Shatz, B. (2004) (ed.) Peer Review: A Critical Inquiry. Rowland & Littlefield. Pp. 235-242. http://www.nature.com/nature/webmatters/invisible/invisible.html

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s