Sat. Jan 28th, 2023
Evaluating data about schools does not always have much to do with numbers.
Enlarge / Evaluating data about schools does not always have much to do with numbers.

There are few types of people we complain about more than politicians. They are often portrayed as stupid two-faced liars – with the possible exception of the ones you voted for. But politicians’ views tend to coincide with the majority of people who identify with their party, and it’s not always clear whether the politicians lead their party or whether everyone is just marching to the beat of the same drum.

After all, the thought processes of politicians should not be much different from those of voters. We are all subject to biases and cognitive cheats that block information we don’t like. Is it fair to hold politicians to a higher standard, given that they are the people we choose to carefully process decisions for the rest of us?

Ask a Dane

Politicians are not the easiest group to study, but a group of researchers from Aarhus University, led by Martin Baekgaard, took a chance by sending surveys to Danish city councilors – a group large enough (950 people) for a meaningful study . The surveys (which were also completed by a representative sample of about 1,000 Danish citizens) were designed to evaluate the subjects’ propensity for motivated reasoning.

Motivated reasoning is what we all do with information that either confirms or threatens our previous opinion: we tend to believe the corroborating evidence and explain away any conflicting evidence. It would of course be nice if politicians were particularly adept at processing information objectively. But the researchers’ hypothesis (and probably your hypothesis at this point) was that the politicians would not be the shining paragons of reason, like the citizens they polled.

In the first experiment, subjects were asked to evaluate fictitious customer satisfaction data for parents rating schools or citizens for road maintenance. Some surveys (for the first example) simply identified schools as “A” and “B”. But the other surveys identified one school as public and the other as private.

This was the crux of the experiment, as the merits of public services versus private suppliers are a source of lively political debate. The subjects were asked to evaluate which supplier performed better based on the data provided. When this was just A vs. B, politicians and citizens alike had little trouble identifying the winner in the data. But when the question became public versus private, interpretations were distorted by how people thought about public versus private suppliers in general.

When the data showed that the public school got better ratings, almost all who preferred public suppliers had no problem getting the correct answer, but half or more fans of private suppliers claimed the data showed the opposite. The same was true (conversely) if the data tended towards the private school.

Losing motivation

The next experiment is where things get a little weirder. A follow-up study included the same kind of data for post-surgery rehabilitation providers, but with different ones amounts From the data. Some participants received data for patients in one type of rehab, some received three sets of data, and others received five sets. Research has shown our reasoned reasoning can reaching a breaking point (in the right situation) when we have a large enough pile of unambiguous information in front of us. At some point it becomes too obvious that we are trying to explain away inconvenient facts and we process the information without ideological blinkers.

We hope that politicians will do this, and the researchers actually hypothesized that they would see this reflected in the research results. That is, reasoned reasoning would be slightly weaker for those who were given three or five sets of data that all pointed to the same conclusion.

But that’s… not what happened. The politicians showed no improvement with additional information. If anything, they might even have done the opposite. Reasoned reasoning was somewhat stronger for the subjects who received more data. If the public provider (for example) received better ratings, those who favored public providers were even more likely to give the correct answer if they had multiple data sets in front of them, while supporters of private providers were slightly fewer probably interpret the data correctly.

The shift was small – statistically significant by some but not all measures – but it’s at least safe to say the politicians certainly didn’t better when they got more information.

Before you take a shot at the politicians, it’s worth noting that citizens didn’t fare much better with this survey experiment. In their case, there was no indication that reasoned reasoning would have increased with more information, but the results were more or less the same.

Data versus opinion

So maybe people just reacted the way they would react whether they had little or a lot of data. It wouldn’t be unheard of for more data to fail, causing the person to do double duty despite opposition.

The researchers also point to a study that suggests highly partisan people will interpret questions about data as sincere opinion questions if they are motivated to do so. (For example, when asked to rate economic trends, the answers essentially reflect whether a person’s political party is in power, rather than the data provided.) And it’s worth noting that survey respondents knew that the data they were getting were fictitious in this case, so maybe felt free to ignore the numbers they had to interpret.

Whatever the reason, these Danish politicians did not cover themselves with glory. At best, they behaved just like the citizens who put them in office. At worst, they stuck to their positions more stubbornly than the average person, even in the face of unequivocal math.

British political science magazine2017. DOI: 10.1017/S0007123417000084 (About DOIs).

By akfire1

Leave a Reply

Your email address will not be published.