Decisive
Pundits and activists have long blamed the “mainstream media” for having too much influence on public perception. Whichever side of the political spectrum you’re on, some people say, it seems like major media outlets like the New York Times or FOX News exerts too much power over the national conversation. Ideas from non-mainstream media are drowned out according to this logic. But a new long-term study reveals that small media outlets have a far greater effect on public discussion than anyone realized.
More precisely, it only takes three or more small news stories on the same topic to increase discussions on that topic on Twitter by 62.7 percent.
It took a group of Harvard researchers five years to come to this conclusion. They did this by tracking the effects of stories covered by 48 small media outlets and measuring how they influenced conversations on Twitter. Harvard political scientist Gary King and his colleagues explain in the journal Science that they honed 11 broad topics in public policy, ranging from refugees and race to food policy and domestic energy production.
“If we had done this research 100 years ago, we would have gone to city squares and listened to what people were saying on soapboxes,” King said. “Today it’s Twitter.”
The public sphere
To conduct their experiment, the researchers asked editors if they would be interested in publishing stories on any of the policy topics. Journalists could write anything they wanted, from any political perspective, as long as it was relevant to the broad category of, say, food policy. “They would choose the [story] corner and we would approve it for the experiment, so there was total control on both sides,” said King.
Then the researchers randomly chose two consecutive weeks that were expected to be “slow news” weeks. One week would be the experiment week, when the stories came out, and the other would be a control week. The experiment lasted five years, King explained, because it was so hard to coordinate. If there were major news related to the topic, it would introduce too many confounding factors to gauge the effects of their media outlets. They would have to cancel the experiment that week and start again at another time.
Ethics was a big problem for the researchers. They wanted to interfere as little as possible with the normal functioning of media organizations. “It was like the Prime Directive on Star Trekjoked King. But he was also deadly serious. He and his colleagues worked with news organizations for months to make sure the only thing that changed was when the stories ran, not what the stories were. (For a fascinating account of study the design and ethical issues, see the team’s supplemental materials.)
The hardest part was analyzing the impact these stories had. Using an algorithm from data analytics firm Crimson Hexagon, the group mined Twitter for any public tweet that might be relevant to the topic during the experiment and control weeks. Working with a team of researchers, many of them Harvard students, the group combed through those two weeks’ worth of tweets to make sure they were actually relevant. In other words, they wanted tweets about food policy, not just people who wrote “food is good nom nom nom.”
Study co-author Ariel White, now a political scientist at MIT, was one of the first to analyze the sheer volume of tweets for relevance. She said it was a good lesson on how the media can influence the public conversation. While the study revealed little about people’s personal beliefs, it revealed their “participatory opinion, their willingness to talk to each other in the public sphere.” Today’s policymakers look to Twitter to see what people are talking about, she added, which is why people’s willingness to tweet about a topic becomes politically significant.
Fake news
White also learned some unexpected lessons. When she started the project more than five years ago, she was not yet a Twitter user and was overwhelmed by the “insult” of what she found. She had to monitor the students analyzing tweets to make sure they weren’t “overburdened by the awful things people write on social media.”
After years of this sometimes harrowing data analysis, King and White found that it only took three stories in small outlets to boost public discussion of a given topic on Twitter by 62.7 percent. The same number of stories also swayed public opinion toward the stories’ ideological positions by about 2.3 percent. So the media may not be able to change our opinion much, but it certainly can change what we are talking about.
This research also sheds some light on the phenomenon of fake news on social media. “One thing we measure firmly is the media’s ability to set the agenda,” White said. Small outlets publishing fake news could do that too. “The things [the media] talk about being what we as Americans talk about,” White concluded — for better or for worse.
Science2017. DOI: 10.1126/science.aao1100