Remember me
REVIEW ARTICLE
Year : 2022 | Volume
: 17
| Issue : 6 | Page : 342-349
Scientific peer review in the modern era: A comprehensive guide
Nimrat Kaur Sandhu1, Rhea Wason2, Ashish Goel3
1 Department of Public Health, University of California, Merced, CA, USA
2 Maulana Azad Medical College, New Delhi, India
3 Department of Medicine, Dr. B. R. Ambedkar State Institute of Medical Sciences, Mohali, Punjab, India
Correspondence Address:
Prof. Ashish Goel
Department of Medicine, Dr. B. R. Ambedkar State Institute of Medical Sciences, Mohali, Punjab
India
Source of Support: None, Conflict of Interest: None
CheckDOI: 10.4103/0973-3698.364681
The scientific peer review process involves scrutinizing the work of researchers by a panel of scientific experts who are selected by the journal to which they have submitted a manuscript. It ensures that only the best ideas and the highest quality of research are circulated through the publications in the scientific journals. The concept of peer review preceded the development of the scientific journal, has co-evolved with the scientific method, and has now become irreplaceably integrated with it. The traditional method of peer review involved multiple submissions and revisions and was criticized for being time-consuming and biased with the reports of professional misconduct even among the renowned journals. Several guidelines now ensure a more ethical, consistent and rigorous review process. Novel methods of review such as the use of registered reports and the dynamic review methods are also being explored by publishers to deal with these issues. Although the role of peer review as a gatekeeper for high-quality research has been well established, its value in future research is rapidly evolving. Newer methods and adherence to guidelines will go a long way in improving this imperfect pillar of science.
Keywords: Journal article, peer review, research
In recent times, the process of peer review has been established as the yardstick to assess the quality of submissions for the scientific publication. Journals without a robust system of review by peers are considered sub-standard and often predatory, often not looked upon kindly in the scientific community.
The process of peer review subjects a researcher's work to the scrutiny of a panel of scientific experts, to assess the quality of the manuscript under consideration by a journal for publication. These experts provide inputs to improve the article.[1] A peer review panel consists of scientific experts who are notable figures in the field as well as scholars who are well-informed about the topic but are not experts in the field. Although it is a technical and time-consuming process, the reviewers are generally not paid for their time and effort. This makes the peer review process a noble altruistic act through which these experts can make valuable contributions to the body of science in their respective fields.
The peer review process ensures that only the best ideas and the highest quality of scientific research are circulated through the publication of manuscripts in the scientific journals. The standardized checklists used during this process, aim to maintain, and improve upon the standards of scientific research. Each journal has its own set of peer reviewers who are responsible for maintaining the quality of the articles selected for the publication and citation metrics of the journal.[2] This process is of considerable importance as scientific evidence tends to build on itself. If the results of one study are inappropriate, then the results of all the subsequent studies based on those concepts become questionable leading to unnecessary confusion and delays in the development of meaningful solutions to complex scientific problems. This paper attempts to provide an overview of the peer review process, its benefits, drawbacks, and changes in this process over time, particularly during the COVID-19 pandemic.
Historical Origins of the Peer Review ProcessThe concept of peer reviewing a manuscript dates back to ancient Greece and preceded the development of the scientific journal itself. The earliest description of a method to peer-review a manuscript was described in the book “Ethics of a physician” written by the Syrian physician Ishaq bin Ali Al Rahwi between CE 854 to 931.[3] The earliest record of a formal process of peer review appears in 1655 and is attributed to Henry Oldenburg, then editor of the “Philosophical Transactions by the Royal Society,” the journal which first incorporated the concept of peer review.[4] However, the earliest record of a peer-reviewed publication appears as the manuscript entitled “Medical Essays and observations” published by the Royal society of Edinburgh in the year 1731, several decades after the description of a formal peer review process.[5]
The peer review process based on these 18th century concepts became popular in the 19th and 20th centuries. The British Medical Journal (BMJ) started to conduct peer review of all non-editorial submissions by known experts in the field as early as 1893. The Journal of American Medicine (JAMA) instituted the formal peer review process in 1976 and Nature introduced the peer review process to screen the articles for publication in 2003.[6] The evolution and popularity of this technique in the modern era are related to two independent events, including the publication of an innovative book on the peer review process called “A Difficult balance” by Stephen Lock in 1985 and the decision by JAMA to hold conferences to present research based on the editorial peer review process in 1986. These events led to the widespread adoption of the peer review process by journals such as the Annals of Internal Medicine, Lancet, and other prominent scientific journals leading to the formal institutionalization of this technique.[7],[8]
Selection of Peer ReviewersPeer reviewers are experts chosen from a database maintained by the journal and selected according to their area of expertise. They have a fair knowledge about the subject being discussed.
Many journals send invitations to peer review through E-mail and provide digital platforms and checklists to guide reviewers on how to appropriately complete the peer review process. A study conducted by the Publishing Research Consortium found that an average reviewer returns approximately eight reviews in a calendar year.[9] In a survey conducted by Sense about Science, an independent nonprofit organization in the United Kingdom, over 4000 peer reviewers were asked to examine their level of interest in the current system and found that up to 90 percent of them felt satisfied with the system and nearly one third of them were willing to review up to 10 manuscripts in a year.[10]
Types of Peer ReviewThere are different types of peer review and include different levels of blinding.[11] These could be open, single blind/anonymized, double-blind/anonymized, and triple blind/anonymized peer review. In open peer reviews, both the author and the reviewers are aware of each other's identities. It discourages plagiarism, ensures timely completion and use of polite language in the reviews. However, it can lead to bias as some reviewers may try to avoid criticizing the work done by colleagues or superiors. Despite some criticism, there has been widespread support for open peer review across disciplines. It makes both authors and reviewers stakeholders in the process creating a favorable effect on the quality of the articles. This process has been successfully implemented by the BMJ and BioMed Central.[12] In single blind peer reviews, utilized by the Journal of American Medical Association, the identity of the reviewer is kept confidential. They allow reviewers to maintain their confidentiality and avoid undue pressure from the authors. However, they can still be problematic due to the conscious or sub-conscious bias of the reviewer toward the authors.
In the double-blind peer review process such as that utilized by the American Journal of Public Health (AJPH), both the authors and the reviewers are not aware of each other's identities. Although double-blind peer reviews are considered to reduce bias by maintaining confidentiality of both the authors and the reviewers, the process is not fool proof as reviewers who have worked with the authors or are familiar with their work may recognize their writing while reviewing the manuscript. In the triple blind review process such as that utilized by the British Journal for the Philosophy of Science, the identity of the author and the reviewer are kept confidential from the editors and the article itself is anonymized using complex techniques.
In a transparent peer review, the report of the reviewers is published along with the manuscript to ensure complete transparency. A collaborative peer review refers to the process wherein two or more reviewers submit a single peer review report about the manuscript.[13] The type of review depends on the editors of the journal to which the manuscript has been submitted for the publication.
Newer Techniques for Peer ReviewPostpublication peer review
Postpublication peer review is utilized by platforms such as PubMed and comprises of feedback submitted by other researchers after reading the published manuscript.[13] While postpublication reviews can present diverse perspectives on a topic, they can also lead to a wide variety of unsolicited commentary on controversial topics.[14]
Registered reports
A relatively new form of peer review utilized by some scientific journals is referred to as the use of registered reports. Under this system, proposals for a study are subjected to peer review in the design phase to receive feedback about the underlying concepts and methods used to conduct the study. If the proposal passes this phase, then the study is given a pre acceptance for publication if the research team can successfully complete the study using the mentioned methods and provide an evidence-based interpretation of the results. This technique has been found to improve transparency and reproducibility in scientific research.[15],[16]
Dynamic peer review
Another widely utilized technique is called the dynamic peer review method.[15] In this technique, the manuscript is published in preprint media sites such as Naboj.com or medRXiv. This permits the simultaneous evaluation of the manuscript through online reviews on the website as well as peer reviews conducted by the journal in which the article has been submitted. It reduces plagiarism, permits the free exchange of ideas on important topics, and shortens the publication cycle. Both these techniques have been widely employed during the COVID-19 pandemic to enhance the speed of conducting research and publishing scientific information that could be used to help healthcare providers.
The difference between the older and newer techniques of peer review lies on the renewed emphasis on faster dissemination of scientific knowledge, particularly in the era of the COVID-19 pandemic without compromising on the quality of content. These techniques ensure that new and innovative ideas which may have faced rejection on traditionally closed meritocratic forums due to bias, unethical conduct and inconsistent reviews are subjected to a rigorous, high-quality peer review process in a democratic manner to ensure the rapid and unhindered progress of science.
Process of Being Selected as a Peer ReviewerThere are different ways of becoming a peer reviewer. As a researcher ascends the professional ladder, they can apply to become a reviewer for a journal or they can be invited by journals themselves to peer review an article. The peer review process involves a critical analysis of the manuscript and providing comments to improve the manuscript. The reviewer once selected, receives an electronic message to review a manuscript for publication. He/she can choose to accept or reject such an invitation. Different journals have different publication guidelines including topics of interest, word length, number of figures and tables as well as recommendations related to the content, content of the articles selected for publication, and style of referencing which authors and reviewers are required to adhere to. In addition, depending on the level of expertise of the reviewer, he/she may be required to undergo training on how to conduct a high-quality peer review. Journal editors play an important role in the selection of peer reviewers. They select individuals who have expertise in the content of the manuscript. They are responsible for ensuring that they have adequate training on how to adhere to the publication guidelines. They keep track of deadlines and make sure that the reviews are free from bias.
Process of Peer Reviewing a ManuscriptThe Committee on Publication Ethics (COPE) has created a list of ethical guidelines for peer reviewers which lays down the fundamental principles which peer reviewers are expected to adhere to during this process. [Figure 1] provides a flowchart which is meant to guide reviewers during the each stage of the review process. A summary of these steps will be discussed in this section to guide potential reviewers on how to proceed in an ethical fashion when they receive an invitation peer review a manuscript.
Figure 1: COPE Guidelines for peer reviewers.[17] Source: (https://connect.springerpub.com/sites/default/files/2020-08/what-to-consider-when-asking-for-peer-review.png). COPE: Committee on Publication EthicsAccording to the COPE guidelines, the following steps should be followed to conduct a fair and good quality peer review.[17]
When a potential reviewer receives an invitation to peer review an article, first they need to consider is the legitimacy of the journal and the manuscriptNext, they need to determine if they hold expertise in the subject matter of the article and can review the article within a reasonable timeframeIf they are not experts in the field or they cannot return the review in a timely fashion, they should decline the invitationThey should declare any conflicts of interest upfront and provide updated and accurate information about their professional expertise before participating in the reviewIf they choose to accept the invitation from the journal editor, they are provided with a checklist of items they need to consider when they review the articleThe contents of the manuscript and the review should remain confidential and should not be used for personal or professional benefitThe review should be conducted in an objective manner without taking into consideration any implicit or explicit personal or professional bias in the assessment of the articleEach section of the article is clearly evaluated. The title should be concise and clearly reflect the content of the article. The abstract should be brief and structured according to the journal guidelinesThe introduction should provide a brief and relevant overview of the literature related to the topic of discussion. The methods section should clearly explain the type of study, the recruitment strategy, number of participants, the statistical or other analytical techniques used for analysisThe results sections should comprise of figures and tables which accurately represent the outcomes of the studyThe discussion section should provide an explanation of the results and make comparisons with similar studies conducted in the past. It should explain the strengths of the study and the limitations including potential biases and the impact of confoundersThe references section should have relevant references for the literature cited in the article and should be formatted according to the style recommended by the journalThe reviewers should also provide general comments on the grammatical and formatting errors in the manuscriptThe usual timeline for peer review of an article is 3 to 4 weeks, although some journals may review an article either earlier or later based on the availability of peer reviewers and the time devoted by them on this process.[18][Table 2] provides an example of a checklist commonly used to review the articles submitted to the journals which contribute to the National Library of Medicine. Some journals may have additional guidelines for authors and reviewers. These guidelines can change and are updated periodically. The peer reviewers should be able to access references either through open access or through the institutional login of the journal for which they are acting as a reviewer. It enables a more comprehensive review of the article. In addition, the use of open-access references allows easy reproducibility and cross-referencing of articles in future publications.[19]
Peer Review of Non-research Related ArticlesIn addition to the publication of results of clinical trials and other forms of research studies, journals also publish some nonresearch related articles such as case reports, systematic reviews, editorials, review articles, and methodology articles. Review of such articles necessitates the adoption of different criteria.[18]Case reports are evaluated for correct diagnosis, reasonable treatment regimen, confidentiality, and clear description of outcomes. Systematic reviews are evaluated for the use of clear inclusion and exclusion criteria, appropriate selection of studies and description of results. Review articles are evaluated on the content which has not been previously presented, whether they provide an unbiased view of the subject, its contribution to the field of study and its understandability for readers who are not experts in the field. Methodological articles should clearly describe the methods, the rationale behind the selection of the method, its comparison with other methods, and its utility in the field. Editorials are often invited and pre-approved for publication. They are evaluated on their contribution to the topic under discussion, whether they present a logical and coherent argument according to the existing or emerging evidence on the subject. The Equator Network is a reputed and widely accepted online resource for accessing the latest updates and guidelines on the publication for different types of articles.[20]
Medical journals tend to apply the higher standards of quality to review manuscripts as compared to nonmedical journals due to the higher possibility of patient harm and legal liability. These standards are updated on a regular basis as many such articles form the foundations of evidence-based guidelines through the inclusion in systematic reviews and meta-analysis conducted by international organizations such as the Cochrane collaboration. The quality of the journal itself is calculated based on the number of citations, impact factor, the level of circulation, and the manuscript acceptance rate. Important factors to consider while assessing the quality of a journal include having recognized experts as editors, clearly defined website, relevant areas of interest, affiliation with a university or scholarly association. Journals with Medline indexing, high rates of peer review, high impact factor, high levels of circulation, and low rates of acceptance have higher quality scores compared to other journals.[21]
Common Errors in the Peer Review ProcessThe peer review process is associated with some common mistakes. First, some reviewers lack training on how to conduct reviews appropriately or according to the guidelines of the journal. This lack of knowledge and experience may lead to inadequate reviews.[22] Reviewers in different regions or with different levels of expertise may hold varying opinions about the same topic which when combined with a lack of thorough understanding of the review criteria can lead to inconsistent reviews which make it difficult for authors to appropriately edit manuscripts for the publication.[23] Some reviewers may have biased opinions on the topic, may favor the publication of only positive findings or may favor the acceptance of articles that present a positive view of their own research or institution.[24] Finally, the reviewers may engage in unethical conduct such as failing to maintain confidentiality leading to the theft of ideas, personal attacks, and requiring the addition of references of their own work for allowing the publication of the manuscript.[25],[26]
Strengths and Limitations of the Peer Review ProcessThere are several benefits of the peer review process to the body of science and to the individuals involved in the process. Peer review has become an established part of the scientific process and the Institute for Scientific Information considers only peer-reviewed articles to calculate the impact factor.[27]
The peer review process has been shown to positively impact the performance evaluation reports of early career researchers and helps them to advance in their careers.[28],[29]
The strengths and limitations of the peer review process are included in [Table 1].
Understanding the limitations of the conventional peer-review process several alternative methods have been proposed and are now in use. Some journals provide authors an option of opting out of re-review reducing faster turn-around rates. Similarly, methodologies such as cascading (sharing reviews on rejected manuscripts between multiple journals), portability (submission to a new journal with the peer-reviews from the previous journal), and crowdsourcing (immediate publication while simultaneously inviting online reviews).[30]
Cost of Peer ReviewA recent study using publicly available data to calculate the researcher's time and salary-based contribution reports that globally reviewers spent over 100 million hours or more than 15 thousand years in 2020 to peer review manuscripts at an estimated time-based monetary value of 1.5 billion US Dollars for US-based reviewers, 400 million US Dollars for UK-based reviewers, and over 600 million US dollars for China-based reviewers.[31]
Due to the tedious and technical nature of this exercise, there have been calls to provide financial incentives to reward peer reviewers for their efforts. While monetary remuneration and other forms of incentives including free subscriptions or honorary memberships were a common feature in the past, this is not so in the current times.
Practices of Nominating Reviewers during SubmissionThere has been a steady increase in retraction rates from 0.38/10,000 articles in 1985 to 5.95/10,000 articles in 2014. A recent study examining the number of retracted articles among countries which had published more than 100,000 articles between 2003 and 2016 found that the top five countries in terms of retracted papers were Iran (14%), Tunisia (12%), Bangladesh (10%), Pakistan (10%), and India (10%).[32] It has been reported that authors manipulated the system to list themselves as reviewers, utilized fake online reviewers or paper-mills which provided favorable reviews to their articles.[33] This practice was more common when authors were allowed to nominate reviewers particularly for collaborative research projects. They used generic email addresses and received favorable reviews within few days of manuscript submission.
Many publishers have now removed the online system of nominating reviewers. Editorial associations such as COPE and the International Committee of Medical Journal Editors have issued guidelines on how to detect fraudulent publications. The Cooperation and Liaison between Universities and Editors group has also published recommendations to improve the integrity of research by enhancing the cooperation between research institutions and scientific journals.[34]
Techniques to Improve the Peer Review ProcessJournal editors can play a significant role in the improvement of the peer review process. First, they can select competent reviewers who are experts in their relevant fields. Second, they can provide guidance to the reviewers on the requirements of the journal and offer training on how to conduct high-quality peer reviews. Third, they can evaluate the reviewer's comments to look for bias, inconsistent or inappropriate reviews and provide constructive feedback. Finally, they should respond to commentaries and reviews related to the articles to weed out fabrication, falsification, or plagiarism in the articles after publication.[26]
Other ways of improving the process include the introduction of a more democratic system of peer review through the declaration of editorial and peer reviewer conflicts of interest, the development of more uniform interdisciplinary guidelines on how to conduct high-quality peer reviews, the use of a wide pool of reviewers, and the need to conduct additional research on the accept/reject rate of submitted manuscripts and potential reasons behind them.[35] There has now been an increased interest in postpublication peer reviews as well as open peer reviews of preprint articles. There is an urgent need to develop standardized guidelines, incentivize reader participation in the process, and examine potential biases and discrepancies to ensure fair and diligently performed reviews conducted in a democratic manner. The reviewer's comments should be relevant and timely and should comprehensively address the issue in a polite and professional manner. The authors of the manuscript should be given ample opportunity to explain their point of view as well as the right to accept or decline such revisions.[36, 37]
ConclusionThe peer review process aims to serve as a gatekeeper for high quality and meaningful scientific research. The publication cycle often involves multiple submissions, oscillates between rejection and revision. It is a gentle balance of an author's patience and perseverance. The process of peer review came into existence and has evolved with the scientific method to play a pivotal role in the advancement of science. Applications of technology and innovative techniques will improve this process which shall continue to play a dominant role in scientific writing in the foreseeable future. The role of peer review in science has been well established, its value and direction in the future is evolving.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
References
Comments (0)