A recent article in Science, “Who’s Afraid of Peer Review?” made available via open access by AAAS, has generated a lot of discussion around the complicated question of peer review in journals, with particular focus on the many new open access journals. The article reports on a test of the rigor and validity of peer review in OA journals, and its results have been receiving a lot of coverage. The author wrote a fake paper with a faulty research method and submitted it to open access journals (reminiscent of the 1996 ‘Sokal affair‘ in which researcher Alan Sokal submitted a nonsense paper to Social Text, an academic journal of postmodern cultural studies), then revealed it as a hoax on the date of publication.
In this most recent study, researcher John Bohannon submitted a paper with obvious experimental design flaws claiming anti-cancer properties for a natural product derivative to 304 open access journals; it was accepted at over half of them.
The author did not send the paper to subscription-based journals (those with a pay wall to readers); his study method has been criticized accordingly, because it does not provide a valid comparison of peer review between OA journals and a comparable range of journals with different (i.e. subscription-based) business models. Peer review is undertaken by volunteers, so it does not follow that the business model is a good predictor of the quality of peer review. Open access publication has many benefits to science and society, but in these early years, there is no question that we see an abundance of new journals with questionable practices.
Interestingly, the range of journals that accepted the paper included open access journals published by reputable commercial publishers, including Elsevier, Sage, and Wolters Kluwer, as well as those published by recently emerged opportunistic publishers seeking to profit from the open access movement.
Those who read the whole paper will see that a scant 18% of the journals listed on one of the resources, Beall’s list of likely predatory OA journals, rejected the paper, while a full 82% accepted it, – good news for those who rely on that watchlist as an aid to screen out questionable journals. The author concludes, “Some say that the open-access model itself is not to blame for the poor quality control revealed by Science‘s investigation. If I had targeted traditional, subscription-based journals, Roos* told me, ‘I strongly suspect you would get the same result.’ But open access has multiplied that underclass of journals, and the number of papers they publish. ‘Everyone agrees that open-access is a good thing,’ Roos says. ‘The question is how to achieve it.'”
*David Roos is a biologist at the University of Pennsylvania.
NOTE: The question of peer review in open access journals is an important one. The Library provides support for those interested in OA publishing by helping them evaluate open access journals See the Scholarly Publishing and Communication research guide, particularly the section on OA Journals and Publishers.
- See the Science special issue on “Communication in Science: Pressures and Predators”
- Discussion of the Bohannon article in The Scholarly Kitchen blog – “Open Access ‘Sting’ Reveals Deception, Missed Opportunities”
Barbara DeFelice, the Library’s Director of Scholarly Communication and Digital Resources programs, contributed to this blog post.
Also of interest:
Couzin-Frankel, J. (2013). Secretive and Subjective, Peer Review Proves Resistant to Study. Science, 341(6152), 1331–1331. doi:10.1126/science.341.6152.1331
A brief news article on peer review in last week’s Science magazine. Though focusing on biomedical publishing, many of the issues raised are common to other scientific disciplines and have been the subject of discussion and controversy for years.