Reproducibility is an important feature of scientific research, and reproducibility may be the core reason why science is accepted by humans as a methodology. It means that completing the same task is no longer limited by the execution subject, but a content or method that can be taught and copied. In other words, without repeatability, there is no science.
On December 7, 2021, researchers from the US Open Science Center published a research paper titled "Investigating the replicability of preclinical cancer biology" in the journal "eLife".
The study repeated experiments on cancer biology papers published in the top journals "Cell", "Science" and "Nature" from 2010 to 2012 and found that 54% of cancer studies could not be repeated!
Researchers launched the project eight years ago to try to replicate findings from top cancer labs. The original goal of the project was to select 30 of the 53 papers and replicate their experiments, but vague agreements, uncooperative authors, and other reasons hindered the achievement of this goal.
In the end, it was reduced to 23 papers, and all experimental replications of 18 of them were completed, as well as some experimental replications of the remaining papers.
The study found that results from only five papers could be fully replicated, with other replicates producing mixed results, some negative or inconclusive.
Overall, 54% of cancer studies could not be replicated. Of the 112 reported experimental effects, only 46% met at least 3 of the 5 replication criteria.
It's worth noting that the magnitude of the change in repeated experiments is usually much smaller, averaging only 15% of the original result.
Copy effect size compared to original effect size
The researchers emphasized that the lower effect size was not surprising and was consistent with publication bias. This does not mean that all scientific research cannot be trusted.
In fact, the president of the National Academy of Sciences said, the researchers were unwilling to share methods and data so that others could verify the work. Researchers lose credibility if their findings don't stand up to scrutiny. In contrast, if data, protocols, and reagents are readily available, experiments should be completed very quickly.
The researchers say it would be a waste to move forward without reproducing the experiments.
He also said that NIH will require funded institutions to share data in 2023 in an effort to improve data sharing among scientists.
Taken together, this new study reflects an early flaw in the scientific process rather than an established treatment approach. When cancer drugs hit the market, they have been rigorously tested on large groups of people to ensure they are safe and effective.
