The classic model of scientific progress is that the field advances when new findings contradict or replace old ones. But a new study reveals that this process doesn’t work today — at least not in scientific journals, where most of the data is shared with peers. Indeed, the researchers found that “refutations hardly change scholarly perceptions about the original papers.”
For the study, a group of researchers looked at the citation rates of seven marine biology papers on fisheries. Citation rates are often used as a measure of the “importance” of a scientific article, with the idea that the more an article is cited, the more influential it is. Each article had been the subject of a refutation, also published in a scientific journal. The researchers wanted to know if these rebuttals affected the citation levels of the original papers — and, perhaps more importantly, if they convinced people to question the interpretation of the data in the original papers.
As it turns out, rebuttals don’t seem to affect the scientific community’s understanding of the original papers in any way. “The original articles were cited 17 times more often than the rebuttals, an order of magnitude difference that overwhelms other factors,” the study authors write in the paper. Ecosphere. “Our test score results emphasize that rebuttals have little impact: even the rare few authors who encountered the rebuttals were only influenced enough to move from full support of the original paper (a score of five) to neutrality (a score of three). ), despite all the rebuttals claiming that the interpretations of data in the originals were incorrect. Astonishingly, 8 percent of articles citing a refutation suggested that the refutation supported the original paper’s claims, an observation that might give pause to those considering writing a rebuttal in the future.”
Clearly, the sample size here is modest. The study authors looked only at the effects of rebuttals on seven marine conservation biology papers. However, their findings are still quite damning. They suggest that once a paper is published and widely cited, it is almost impossible to contradict it with new evidence or data analysis.
The authors believe that one way to address this issue would be to link to rebuttals in the original articles so that readers understand that the article is contested. This is especially important in fields like conservation biology, where highly cited papers are often used to make public policy. Scientific journals should make it easy for readers to find rebuttals in the interest of transparency and the advancement of scientific knowledge.
Ecosphere2016. DOI: 10.1890/ES10-00142.1