Behind the Research: What is the Peer Review System?
Published: 15 November 2016
Dr Federica Giordani explains the process by which the quality of scientific research is assessed before publication in academic journals.
You may have heard of the ‘peer review system’ but what exactly is it?
Peer review is the process by which the contribution of a piece of research is assessed by independent experts in the field before being published in an academic journal. It is a practice of quality-control that aims to make sure that only the most rigorous research is published. If you are undertaking a career in academia you will inevitably be involved in peer review, either as an author or, eventually, as a reviewer. Still, for many early career researchers, some aspects of the peer review system can remain unclear. Why does peer review matter? What criteria are evaluated in a manuscript? Who makes the final decision on a paper? What are the benefits for the reviewer?
All of these questions and many others were recently addressed during a workshop organised by the charity Sense about Science through its programme Voice of Young Science at Glasgow Caledonian University. During the event, young scientists had the chance to discuss in small groups the pros and cons of peer review and get useful inside information and advice from experts representing the academic and editorial worlds.
The good and the bad
The peer review system is widely recognised as the main tool in ensuring that scientific publications are of an acceptable standard. External review helps to enforce the principles of sound science – namely that experimental work adheres to correct protocols, statistical analyses are carried out correctly, and that conclusions are realistic based on the evidence presented.
However, peer review is not without problems and is not a ‘fairy tale’ explained Professor Sergio Della Sala, professor of Human Cognitive Neuroscience at the University of Edinburgh and editor-in-chief of CORTEX. Conflicts of interest, plagiarism and outright fraud can, and do, happen. This is demonstrated by the number of retracted papers, which is estimated to be in the order of 500-600 per year (and rising) according to Retraction Watch. The ‘publish-or-perish’ pressure researchers experience can promote the submission of sloppy science to journals, and an upsurge in the number of open access predatory journals (i.e. publications aiming to accept high fees for publication without concern for scientific integrity), can exploit this mindset. As Professor Della Sala added, ‘yes, publishers do care about the science, but they want the money first’.
How to get through peer review
When the system works as it should, however, peer review ensures that only valid, significant and original research gets published. Moreover, reviewers' comments benefit the authors and result in improvements to the final version of the manuscript. To get through peer review, though, as well as the scientific content, the presentation of results in a manuscript is important. Professor Richard Shape from the MRC Centre for Reproductive Health and Deputy Editor of Human Reproduction, gave some tips:
‘first, make sure the manuscript tells a story; second, that it shows explicitly how the study advances our understanding in the field; third, make sure you do not confuse interpretation with evidence’.
He also underlined a fundamental rule:
‘the reviewer is always right (even when he is wrong); so always respond positively, do not dismiss, be respectful and refute only with sound science’.
Just think, in the end, that reviewer might well be you!
Reviewing is good for scientists
When we scientists are on the other side of the fence and act as reviewers, we contribute our time, expertise and objectivity for free. Although some journals have adopted initiatives to acknowledge reviewers' contribution (such as publishing annual lists of reviewers), payment for peer reviewing is not an option. The panellists agreed that it may encourage some to review work they are not qualified for and could well promote misconduct.
So why do researchers contribute to peer review? One important reason is because they like to play their part and contribute to the academic community. Peer reviewing can have various advantages such as allowing researchers to read and comment upon the most up to date advances in their field, as well as improving their critical thinking skills. As Professor Shape explained, ‘peer review is a shortcut to experience: you can learn from the tribulations of others’.
While many young researchers may be unsure about how to engage with the system, the advice from the panellists was that the best way to start your career as a reviewer is to shadow an experienced reviewer.
Peer review and the public: does it matter?
The importance of this quality check is not restricted to the world of academia. Indeed, it has great value for policymakers, reporters and wider society, as it is an essential tool to weigh up all of those science-related claims that we come across on a daily basis. The feeling, however, is that too often the concept and the meaning of peer review remain unknown or not well understood outside of the circle of experts. The charity Sense about Science, represented at this workshop by Emily Jesper and Joanne Thomas, aims to fill this gap in knowledge, by emphasising the importance of critical thinking for all sectors of society.
It is not always clear whether a news story or health claim has been based on science that has been peer-reviewed. If in doubt, Sense About Science suggest asking for evidence, and have published a guide and tool for getting in touch with those making strong claims.
By understanding more about the research process, we are better able to make up our own minds about the scientific claims that we see regularly in the media. Peer review is an important part of the research process, and while it is not a perfect system, it helps to ensure that most of the research that is published is of a certain quality and reliability. This is crucial if that science is then used to inform public policy or wider societal debate.
Glasgow, 10/11/2016
First published: 15 November 2016