I am excited to announce that this summer, the NIH Office of Strategic Coordination (OSC) launched the new Replication to Enhance Research Impact Initiative (Replication Initiative), a pilot approach to advance rigor and transparency in biomedical research by supporting the direct third-party replication of NIH-funded studies.
The importance of reproducibility and replication in biomedical research is a familiar discussion to many. Research must be reproducible to ensure it is reliable as studies advance further in the clinical pipeline. Over the past decade, NIH has launched many activities to enhance the reproducibility of NIH-supported research, including changing language in application and review processes to promote rigorous and transparent experimental design. A 2021 report from the Advisory Committee to the NIH Director discussed recommendations to improve the replicability of animal studies funded by NIH, and the recent NIH Policy for Data Management and Sharing aims to enable validation of research results through the sharing of scientific data.
With so much attention and careful thought given to this issue, why is it that the research community still faces challenges around the replication of biomedical studies? I am reminded of my own experience in the lab where, as a graduate student, I developed a way to interface an emerging chromatographic method with mass spectrometry to determine complex carbohydrate structures. While successful from a technical perspective, the method never caught on, in part because it was complicated and difficult. I remember presenting progress on this chromatographic project at conferences and getting questions about some routine chemistry I was using because so many labs had trouble executing it reliably. I had been taught those reactions by a skilled organic chemist, and there were little things we routinely did in our lab that were not detailed by the authors of the original publication. I would always take a moment to share the “tricks” we used to make our chemistry successful.
Many researchers can likely empathize with being on both sides of these experiences. Despite thoughtful experimental design and careful execution, any number of factors can contribute to an experiment succeeding in the hands of one researcher and failing to replicate that success for another: differences in material sources, laboratory conditions, experimental steps that are not rigorously recorded, or less experience handling a sensitive piece of equipment. It is important to note that issues with reproducibility in biomedical research are separate from issues of fraud or misconduct. If a scientific study cannot be replicated, that does not imply misconduct from the investigators. It can be hard to predict where challenges with replication will pop up, and there remains much to learn about how to enhance research reproducibility.
Through the new Replication Initiative, we aim to test whether direct replication of selected research studies is a fruitful approach to enhancing research reproducibility, and if yes, for what types of studies, and under what circumstances. As a pilot approach to answer these questions, we are partnering with contract research organizations (CROs) to replicate NIH-supported research. The Replication Initiative has released the Notices of Special Interest NOT-RM-24-009 and NOT-RM-24-013, which provide funding for researchers with a current Common Fund award or current NIH award, respectively, to engage with an independent CRO to replicate their NIH-supported studies. The funding will support engagement with the partnered CROs and necessary activities such as preparing reagents or manufacturing devices, while the CROs receive separate funding to perform the replication experiments. NOT-RM-24-013 is currently accepting applications through November 15th, 2024, and any interested researchers are encouraged to apply.
We hope that this type of independent replication will benefit the original researchers by providing additional data that could be used to support clinical trial and commercialization efforts or to bolster novel findings that may serve as the basis for future research projects. This initiative gives us an opportunity to hear directly from researchers and the CROs performing the replication studies about how effective this approach is to replicate research, to identify common roadblocks in replicating experiments, and to learn best practices in experimental design to make research more robust. We are excited for the opportunity to contribute to the work NIH is doing to promote rigor and reproducibility as part of the important mission to produce transparent and reproducible work.
If you would like to learn more about the Replication Initiative or get involved, visit the Replication Initiative Website or reach out to CF-Replication@od.nih.gov.