Every year, millions of scientists publish research papers in biotech and life sciences journals. However, there is an industry-wide problem — the results aren’t easily reproducible. When scientists tried to reproduce their peers’ work, 70% of them failed.
In the United States alone, a study estimates that approximately $28 billion per year is spent on preclinical research that can not be reproduced. These numbers highlight the need for a better connection between academia, biotech labs, and the greater scientific community, and a need for more replicable experiments. Scientists also need to prioritize publishing research along with recommendations for others who may run the experiment in their own lab.
Along with a culture of showcasing research results in a good light, scientific research has been in a “reproducibility crisis” for many years. Scientists have “curated” their results, and the community has encouraged results over process. By celebrating failures, or results that may not be glamorous, and establishing an expectation of providing others with tools and resources to conduct their own research, the greater population will benefit.
Here are other ways that research can fail to be reproduced, limiting its potential to the greater scientific community.
For scientists to be able to reproduce academic studies, they must be able to access all the data and research material relevant (and often, also irrelevant) to the study. A lack of access to proper data hinders reproduction, and in the past, was understandable as collecting and storing data was challenging. Now, technology has made it easy for researchers to store and share unpublished data, research design, and discarded hypotheses so that it doesn’t impact experimental design.
A significant contributor to the non-reproducibility of scientific data is poor experimental design. Studies with undefined experimental parameters are not reported clearly and affect the ability to analytically replicate the data.
Cognitive bias refers to the ways that subjective thoughts and opinions impact decision making. Researchers strive for impartiality and avoid cognitive bias from impacting the research outcome, but it’s often difficult to eliminate it. Personal beliefs and perceptions can obstruct scientists from evaluating the data objectively and providing a bias-free template for reproducing the results.
To tackle the problem at its root, many scientific efforts have been directed towards conducting better research that can be translated into thorough experiments. Here are a few ways that more efficient experiment planning and design can impact research outcomes.
Following a standardized experiment planning process ensures uniformity across scientific research and brings innovation. Researchers following standard processes are more productive and save costs. Using a set of patterns while performing a scientific experiment allows them time to document their findings, data, deviating factors, and any other learnings that they find relevant. This optimized process allows them to function smoothly, safely and efficiently.
All the raw data from research studies need to be readily available for fellow researchers and scientists who aim to reproduce a study. Robust sharing of data, material, software, processes and other minute details reduce the likelihood of misinterpretation and significantly increase the probability of a scientific study coming to life.
To allow seamless sharing of data, some well-known journals have started providing additional space for expanded information. Making use of this space is voluntary and allows researchers to share more data if they’re willing to.
It’s important to include a thorough description of processes and research designs to improve reproducibility. Often, the ‘negative’ data that doesn’t support the hypotheses is discarded and not made a part of the final result. Surprisingly, this negative data can sometimes help to design experiments or interpret positive results from related studies. Therefore, researchers are encouraged to store all the data related to a study and share it with fellow researchers and scientists.
One of the most critical tasks in biotech research is to design experiments that can be reproduced. The National Institute of Health (NIH) now requires training in experimental design, and there have been efforts to improve the grant funding process to review experiments in more detail.
To meet industry standards and contribute to scientific progress, design experiments that are efficient, reproducible, and easily documented.
Document the aim of the experiment, the best method to achieve the goal, and the expected results. Remember to record all the information and expectations in the cloud so they are accessible to you at any time and don’t run the risk of being lost.
Documenting the protocol helps identify the reagents required for the experiment and the time taken to carry out the procedure. It’s best to draw up tentative deadlines and a buffer period, in case things go south. Create detailed timelines for each action along with responsible parties and any vendors or supplies involved. Cramming too much in one day may compromise efficiency and make your work more error-prone. Focus on one action at a time to ensure your research is reproducible in real time.
At the end of the experiment, record all your observations. Document your processes, tools, hypotheses, results, deviations from the protocol, difficulties you faced while carrying out the experiments, or any additional techniques that helped you get there. If the results deviate from your hypotheses, check back on the progress and try to identify where the study could have faltered — sometimes called a postmortem analysis. This information can prove valuable while troubleshooting the study and optimizing for future attempts.
It’s often noted that researchers only share with their peers the data and procedures they deem critical for their study to be reproduced. However, this process sometimes leads to the omission of minor pieces of information that may look otherwise, but are significant to the overall results.
Scientific reproduction is heavily dependent on the minutest of details that need to be recorded and shared with every researcher and scientist who has a stake in the process.
Recent developments in technology enable scientists to make data collection easy and error-free. Not only can this data be used to reproduce scientific studies, but is also significant to parallel or future research. The foundation of scientific experiments lies in the reproducibility of experiments. This is where all the stakeholders from across the biomedical industry must come together to refine strategies and develop solutions that bridge the gap between studies and experiments.
Getting this right will change the future of science.