Fri. Mar 24th, 2023
John Oliver's diatribe about scientific reporting should be taken seriously

In the course of the spring I had the opportunity to speak about science journalism at a number of universities. (Thanks to the folks at Washington University in St. Louis and the University of Illinois in Chicago for inviting me.) It took me about 45 minutes to discuss the issues involved and give some examples of things that went seriously wrong .

On Sunday, John Oliver managed to cover the same ground and more, and he did it with much more flair and wit – all in less than 20 minutes. If you have the time, it’s definitely worth checking out.

John Oliver talks about science in the media.

On Monday, all these science journalism problems were once again driven home. The University of Gothenburg has issued a press release suggesting that hunger affects our decision-making processes. This finding should come as no great surprise; anything that distracts us seems to affect our ability to make decisions. But the release itself is a perfect example of many of the issues Oliver pointed out.

Problems in science

For starters, the study was done with rats, not humans. But the very first line of the release – “Never make a decision when you’re hungry” – implies that the results clearly apply to us. The other problem is that the results were generated with a total of only nine or ten rats (depending on the experiment). The results were not very dramatic, making the conclusions extremely weak; there’s a good chance they were statistical flukes.

There’s nothing wrong with publishing exploratory data like this, of course. The problems stem from the fact that there is extreme pressure to publish anything at all if you want to get funding or promotion. And that pressure is combined with the strong tendency of magazines to accept only positive results – experiments that show that something has an effect, rather than that it has no effect. Combined, these factors will result in many publications on spurious effects. Add to that the fact that no one wants to finance or publish simple replication studies and you get a situation in which these ‘facts’ linger in the literature.

This is a problem for science, although probably not a serious one. Most small papers are ignored and the ones that get attention are followed up by scientists who generally know enough to be skeptical of the results until they can carefully examine the results.

The problem is that the researchers and the university weren’t content with letting scientists sort out the meaning of the results. As mentioned above, they issued a press release about it. And as usual, the press release was presented on various sites as if it were a real news item.

But the problem isn’t just that the scientists and press officers promoted some vague findings. The researcher behind the work in question actually suggested it could lead to a huge range of treatments for serious human health problems: “Our results indicate that the brain’s ghrelin receptors may be a potential target for future treatment of psychiatric disorders that are characterized by problems with impulsivity and even eating disorders.”

It’s hard to see this statement as anything other than irresponsible. And that researcher is not the only one; Oliver’s video shows a TED talk from another researcher who likes to overhype their favorite molecule.

Journalism has its own problems

In most cases, discussions about the problems with science journalism focus on the second half of that term: journalism. But all the problems we’ve come up with so far are either endemic to modern science itself or caused by the researchers and universities they work for. When it comes to issues of public understanding, the scientific community has a lot to answer for.

That’s not to say the press doesn’t have problems. As Oliver’s video makes clear, many places jump on these studies-of-the-week and report completely credulously. Part of the problem there is that many of those outlets don’t employ science reporters, so they don’t have anyone to carefully review the studies before the anchors start sharing them with their audiences.

(It’s no surprise they can’t afford academic staff, considering they face competition from places that draw audiences simply by re-running press releases as if they were original content.)

This is not to say that good science journalists are a cure-all. They make mistakes like all other people. They may write too far outside the fields in which they are knowledgeable or let the fact that they approve of a certain set of results cloud their judgment. But given the sheer amount of trouble that happens before journalists even get their hands on a story, they’re generally not doing too badly.

Damage to science

Buried between all the examples Oliver gives and a nice TED talk spoof is a bigger point that shouldn’t be missed: the constant back and forth of conflicting news stories leaves the public with a very confused impression of the state of scientific knowledge, as well as as the process of science itself.

Science is a process that gradually builds accurate pictures of the natural world. It is not always reliable in the details, but the more general conclusions drawn by science are usually quite accurate. By constantly focusing on the details, it’s easy to get the impression that anything is possible – this month’s conclusions are likely to be reversed in a few weeks.

But for many topics — evolution, vaccine safety, climate change — the evidence is comprehensive and highly internally consistent. If we want people to accept those conclusions, we can’t feed them a constant stream of stories that indicate that the process that led us to them is unreliable nonsense. Unfortunately, we continue to do just that.

By akfire1

Leave a Reply

Your email address will not be published.