Sat. Sep 24th, 2022
Image of a basketball player whose shot is blocked.
enlarge Dikembe Mutombo rejects your flawed publication.

Well, that took a while. Five years after Ars’ Chris Lee pointed out that the authors of a homeopathic article did little more than offer “magic” as an explanation for their results, the editors of the journal in which it was published retracted it. The retraction comes over extensive objections from the article’s authors, who continued to believe their work was solid. But really, the back and forth between the editors and authors is bogged down in details that miss the real problem with the original paper.

The work described in the now-withdrawn article involved a small clinical trial for the treatment of depression with three groups of participants. One group received standard treatment, the other a placebo. The third group was given a homeopathic remedy, which means that they were given water. According to the analysis in the article, the water was more effective than the placebo or the standard treatment. But as Chris noted in his original critique, the authors conclude that treating people with water must therefore be effective.

The problem with this is that it ignores some equally feasible explanations, such as a statistical fluke in a very small study (only about 45 people per group) or that it was the time spent with the homeopathic doctor that made the difference, not the water. . These are problems with the interpretation of the results rather than with the data. (This probably explains why the paper was eventually published by PLOS ONE, where reviewers are asked to simply look at the quality of the data rather than the significance of the results.)

That doesn’t mean there weren’t potential issues with the data. According to the retraction notice, other researchers criticized specific aspects of the study, prompting the journal to convene a panel of three editors, an outside academic and a statistical expert. They considered one of the problems Chris noted: the inability to exclude a placebo effect from the homeopathic process. But they also examined how the authors chose different “treatments” that involved variations in how the water was prepared. There were also questions about how the participants had been diagnosed in the first place.

The authors of the original article were given an opportunity to respond, and they have. But the PLOS ONE committee found their response to be unsatisfactory, leading to the withdrawal.

In response to their rejection, the authors say they have provided more than enough information to anyone skilled in homeopathy to replicate the study — all of whom would apparently know exactly how to prepare water based on a patient’s symptoms.

But their reaction also betrays the game a bit. “The PLOS ONE Editors did not explain the ways in which they considered our study design inadequate,” they wrote. “On the contrary, they just said that because the homeopathic treatments included, [different] potencies of the homeopathic medicines [aka water]any positive effect observed must have been a placebo effect.” For the authors, their study design was to rule out a placebo effect so that any difference would demonstrate the effect of homeopathy. They are angry that the editors did not see it things that way.

And that, rather than the specific complaints about the methodology, is the real problem here. Control groups don’t tell you anything about the specific mechanism that causes changes in the experimental group. They just let you identify when the experimental conditions yield a different result. The cause that difference is a matter of interpretation, based on what we know from other scientific studies. If you see a difference, you should consider all the scientifically plausible mechanisms to explain it.

Based on what we know from other work, the PLOS ONE editors rightly consider “homeopathy generates a stronger placebo effect than a pill” as a plausible mechanism. And they’re right that “water behaves magically” isn’t considered remotely plausible. The authors of the article prefer the latter, so their work has no place in a scientific journal.

What’s somewhat frustrating is that the editors don’t bother to pronounce that. Instead, their retraction notice focuses largely on experimental details, as if the paper could be fixed by working out the Materials and Methods section. It gives a misleading picture of the problems here. While there is some consolation in getting the right result – the paper has been officially retracted – it would be more helpful if the result came about for the right reasons.

By akfire1

Leave a Reply

Your email address will not be published.