Conducting a replication study is a valuable way for undergraduate students to learn about the scientific process and gain research experience. By promoting the evaluation of existing studies to confirm their reliability, replications play a unique, though often underappreciated, role in the scientific enterprise. Involving students early in this process can help make replication mainstream among the new generation of scientists. Beyond their benefit to science, replications also provide an invaluable learning ground for students, from encouraging the development of critical thinking to emphasizing the importance of details and honing research skills. In this piece, we outline 10 simple rules for designing and conducting undergraduate replication projects, from conceptualization to implementation and dissemination. We hope that these guidelines can help educators provide students with a meaningful and constructive pedagogical experience, without compromising the scientific value of the replication project, therefore ensuring robust, valuable contributions to our understanding of the world.

Citation: Moreau D, Wiebels K (2023) Ten simple rules for designing and conducting undergraduate replication projects. PLoS Comput Biol 19(3): e1010957. https://doi.org/10.1371/journal.pcbi.1010957

Editor: Russell Schwartz, Carnegie Mellon University, UNITED STATES

Published: March 16, 2023

However, replicating a study can also be a challenging and time-consuming task. One challenge comes from the fact that replication projects often require a high level of attention to detail and meticulous planning, as students must carefully follow the methodology of the original study in order to ensure that their replication is valid and accurate. This can be especially difficult if the original study used complex or specialized techniques, or if the materials or equipment needed to conduct the study are not readily available. In addition, replication projects may require a significant time commitment and resource investment, as students may need to collect and analyze large amounts of data in order to accurately replicate the original study.

When choosing a study to replicate, it is important to select a project that is appropriate for the student, yourself, and also given the resources available. For example, consideration of the specific skill level of your students is key. This means selecting a study that is not too complex or advanced for students to handle, given their current level of knowledge and experience. Choosing a study that is overly difficult or technical may result in frustration and difficulties for the students and could even lead to poor results or a lack of interest in the project. On the other hand, choosing a study that is too simple may not provide students with a meaningful learning opportunity or challenge them to think critically.

There are several factors to consider when determining whether a study is appropriate for the skill level of your students. These may include the complexity of the study’s methodology, the amount of data that needs to be collected and analyzed, the level of statistical or technical knowledge required to understand the study results, and the amount of time and resources that will be needed to complete the project. It may also be helpful to consider the interests and goals of your students, as well as their previous experience with research or similar projects. Have they conducted or participated in research before? Do they have experience writing up results? Do they already have basic or perhaps more advanced statistical skills? By carefully considering these factors, you can choose a study that is challenging yet achievable for your students and that will provide them with a valuable learning experience.

In addition to skill level, the availability of funding, equipment, and personnel, as well as the amount of time available to complete the study should also be factored in. This will help ensure that the study chosen is appropriate to the resources and time available and that it can be completed in a timely and efficient manner. To do this effectively, make sure you consider the scope of the original study and the methodology used, as these factors can have a significant impact on the resources and time required to complete the replication. For example, if the original study used complex statistical techniques or required specialized equipment, the replication may require additional resources and time that are not available given the constraints of the program or the institution.

Finally, corresponding with the original authors about the replication can help build relationships and foster a sense of community within the scientific field. This can be especially beneficial for undergraduate students, as it can facilitate the development of new connections and help gain valuable mentorship and guidance from experienced researchers. When done well and in a considerate manner, this open process can help increase the visibility and reputation of your students within the scientific community, possibly leading to future collaborations and opportunities for them.

There are several reasons why it is essential to follow the original study’s methodology as closely as possible when conducting a replication study. One reason is that replication allows researchers to identify and correct errors or biases, and this function might be compromised if the replication protocol does not match the original. Another, perhaps more important reason stems from the primary purpose of replications, which is to verify the results of previous studies and confirm that they are reliable and consistent. In this context, if a replication departs substantially from the original, it is difficult to know what a discrepancy in findings would mean. Are the original study and the replication coming to a different conclusion because the original findings were not reliable? Or is it because of the change in methodology or poor implementation of the replication? Without abiding by the original protocol, the conclusions one can draw from a replication can be limited, especially when results are discrepant across studies. By replicating a study using the same methods as the original study, researchers can ensure that the results are robust and trustworthy.

Despite these potential challenges, some replication studies might deliberately opt to vary parameters, for example, by involving a different population (e.g., healthy versus clinical) or different measures (e.g., one measure of cognitive ability versus another) than in the original investigation. Those studies might be extremely valuable, but they need to be very carefully planned to yield insight, and often require a deeper understanding of the research question and of its subfield compared to direct replications. Because such thorough understanding takes time, undergraduate students often do not bring this kind of expertise to a project, so it typically falls upon the supervisor to design an adequate replication study. That said, such replication studies arguably have greater potential to spark new programs of research for students to pursue, which might be a factor to consider for those aiming for a career in science. More complex replication protocols might thus be more suited to more experienced graduate students, whose time commitments are usually greater.

Although this advice is valid irrespective of the type of research project, clearly documenting all steps of the research process is particularly important for replications. This means keeping thorough and detailed records of all aspects of the study, including the research question, hypotheses, design, participants, measures, procedures, data collection and analysis, and results. By providing a detailed and transparent record of your study, students make sure other researchers will understand their methodology and will be able to evaluate the reliability and validity of the results.

Throughout the project, good documentation also enables you to review your student’s work. By keeping a clear and detailed record of the study, you can identify any areas where the student may have made mistakes or where they could have done things differently. This can be a valuable learning experience for undergraduate students, as it allows them to gain insight into the research process and to develop critical thinking skills. Furthermore, documentation also ensures that a replication is reproducible; this means that other researchers should be able to replicate the replication study using the information that was provided.

Using sound and adequate statistical analyses is key to extract the most out of replication data. Here, it is useful to distinguish between 2 different scenarios: one where statistical analyses have been decided for you, and one in which you have to come up with your own statistical plan with your student.

In the second scenario we mentioned, the replication might either be large scale but not as readily structured, or it might be part of a more local effort, for example, at the scale of your own lab or research group. Under these circumstances, building an appropriate statistical plan is your responsibility and may involve a little more hands-on work. Oftentimes, using the analysis plan of the original study is the best course of action for the replication as well, yet these are not always available and may need to be inferred from the methods section of the original paper. In some cases, however, the original analysis plan might not be sound, either due to errors or because the statistical framework the original authors used is suboptimal.

Writing a report of the findings that highlights the key points and implications of the study is another very important aspect of replication work. Crucially, this is key even (especially) if the original study did not replicate, or if the results turned out to not be significant. By presenting the results in a clear and easy-to-understand manner, you can help other researchers understand the methodology you and your student used and facilitate critical evaluation of the reliability and validity of the results.

Finally, writing up a research report also facilitates clear communication about the ramifications of the study—by highlighting the key points and implications of the findings, students can learn to clearly communicate their work and its potential impact on the scientific community and beyond.

One way to foster proper citation practices is to provide clear guidelines on how to cite sources properly and direct students to relevant resources, such as citation style manuals (e.g., APA Publication Manual, MLA Style Manual, Chicago Manual of Style), online citation tools (e.g., Zotero, Mendeley, Endnote, Paperpile), or library databases (e.g., PubMed, ScienceDirect, Scopus, Web of Science). When appropriate, it can also be helpful to incorporate citation training into the curriculum more formally, for example, by making citation practices a central component of students coursework, with opportunities to practice and receive feedback. Finally, supervisors should foster an open and supportive environment where students feel comfortable asking questions and seeking clarification on citation practices—although the rules and conventions that govern citations are an inherent part of supervisors’ professional life, they often remain challenging for less seasoned students.

In addition, publishing a paper with an experienced supervisor can help students gain a deeper understanding of the peer-review process and its significance. They can learn about the criteria used by reviewers to evaluate papers, the types of feedback and comments received, and the process of making revisions based on the feedback. This experience can help students appreciate the role of peer-review in improving the quality and credibility of scientific research and develop a better understanding of how to write and present their findings effectively to meet the standards of the scientific community.

Finally, working toward publication with an experienced supervisor gives students a chance to learn about the ethical and legal aspects of scientific publishing. This includes understanding the principles of plagiarism, data fabrication, and misconduct, and how to avoid these unethical practices. Students can learn about the importance of obtaining informed consent from study participants, properly citing sources, and obtaining appropriate permissions for using images and other copyrighted materials. They can also develop awareness for how to properly report conflicts of interest and ensure the transparency and accuracy of their research findings. Understanding the ethical and legal aspects of scientific publishing is critical to maintaining the integrity and reputation of the scientific community, and students can benefit greatly from learning these principles while working with their supervisor on a publication.

Conducting a replication study with undergraduates can be a valuable and rewarding way for students to learn about the scientific process and gain research experience. Replication studies provide an accessible entry point to the world of scientific research, while at the same time, engaging them in the laudable process of verifying previous results to confirm their robustness and reliability. Overall, this process helps build confidence in the scientific community and advance our understanding of the world around us. By following the guidelines outlined in this article, we hope educators and students can increase the likelihood that their replication work will not only constitute an enjoyable learning experience, but will also make a meaningful, high-quality contribution to the scientific record.