Wed. Sep 28th, 2022
Senator John Cornyn (center, black suit) poses with the 2017 astronaut class.
enlarge Senator John Cornyn (center, black suit) poses with the 2017 astronaut class.

On Friday, Texas Senator John Cornyn went to Twitter with some advice for scientists: models are not part of the scientific method. Scientists have responded immediately to blend from bewilderment and annoyance† And Cornyn’s misconception is widespread — and important enough — that it’s worth investigating.

Cornyn’s argument with models reflects a topic of conversation often brought up by people who want to reject uncomfortable conclusions of systems science. In reality, “you can make a model say anything you want” is just as powerful an argument as “all swans are white”. The latter is either an unfair argument, or you have an embarrassingly limited familiarity with swans.

Models are not perfect. They can generate inaccurate predictions. They can generate highly uncertain predictions when the science is uncertain. And some models can are really bad and produce useless and ill-supported predictions. But the idea that models are not central to science is deeply and thoroughly wrong. It’s true that most of the criticism focuses on mathematical simulations, but this is only one type of model in a spectrum – and there is no science without models.

What is a model to do?

There is something fundamental about scientific thinking – and indeed most of the things we navigate in everyday life: the conceptual model. This is the image that exists in your head of how something works. Whether you’re studying a bacteria or microwaving a burrito, you’re referencing your conceptual model to get what you’re looking for. Conceptual models can be extremely simplistic (turn key, engine start) or extremely detailed (hands-on knowledge of every part in your car’s ignition system), but they are useful nonetheless.

Because science is a pursuit of knowledge, it’s about building better and better conceptual models. While the interaction between model and data can take many forms, most of us learn some kind of laboratory-based scientific method that consists of hypothesis, experiment, data, and revised hypotheses.

In a now famous lecture, quantum physicist Richard Feynman similarly described to his students the process of discovering a new law of nature: “First we guess. Then we calculate the consequences of the gamble to see what… it would entail. And then we compare those calculation results with nature… If it doesn’t match the experiment, it’s wrong. In that simple statement lies the key to science.”

To “calculate the consequences of the gamble”, one needs a model. For some phenomena, a good conceptual model is sufficient. For example, one of the fundamental principles taught to young geologists is TC Chamberlin’s “Method of Multiple Working Hypotheses.” observations.

That way, instead of simply summing up all the observations consistent with your favorite hypothesis, the data can more objectively emphasize the hypothesis closest to reality. The more detailed your conceptual model, the easier it is for an observation to show that it is incorrect. If you know where you expect a certain rock layer to appear and it isn’t there, then there is a problem with your hypothesis.

Mathematics is involved

But at some point, the system under study becomes too complex for a human to “calculate” the consequences in his own head. Enter the mathematical model. This can be as simple as a single equation solved in a spreadsheet or as complex as a multi-layer global simulation that requires supercomputer time.

And this is where the modeller adage coined by George EP Box comes in: “All models are wrong, but some are useful.” Any mathematical model is necessarily a simplification of reality and thus is unlikely to be complete and perfect in every way. But perfection is not his job. His job is to be more useful than no model

Take an example of a science that generates few partisan arguments: hydrogeology. Imagine that a leak has been discovered in a storage tank under a gas station. The groundwater level here is so close to the surface that gasoline has contaminated the groundwater. That contamination needs to be mapped to see how far it has gone and (ideally) to facilitate a cleanup.

If money and effort were no object, you would could drill a thousand monitoring wells in a grid to find out where it went. Obviously no one is doing this. Instead, you could drill three wells close to the tank to determine the characteristics of the soil or bedrock, the direction of groundwater flow, and the concentration of contaminants near the source. That information can be plugged into a groundwater model simple enough to run on your laptop, simulating likely flow rates, chemical reactions and microbial activity that break down the contaminants and so on, spewing out the likely location and size of the contaminant. That’s just too much math to do it all in your head, but we can quantify the relevant physics and chemistry and let the computer do the heavy lifting.

For a really perfect model prediction, more or less the position of every grain of sand and every rock fracture under the station should be known. But a simplified model can generate a useful hypothesis that can be easily tested with just a few extra monitoring wells – certainly more effective than drilling on a conjecture.

Don’t shoot the modeller

Of course, Senator Cornyn probably didn’t have groundwater models in mind. The tweet was prompted by work with epidemiological models projecting the effects of COVID-19 in the United States. Recent models incorporating the social distancing, testing and treatment measures used so far predict fewer deaths than previous projections. Rather than welcome this sign of progress, some have inexplicably attacked the models, claiming that these downward revisions show that previous warnings exaggerated the threat and led to disproportionate economic consequences.

There’s a blindingly clear fact that’s ignored in that argument: previous projections showed what would happen if we didn’t react vigorously (as well as other scenarios), while new projections show where our current path sends us. The downward revision doesn’t mean the models were bad; it means we did something.

Often the societal value of scientific “what if?” models is that we might want to change the ‘if’. Calculating how quickly your bank account will go to zero if you buy a new pair of pants every day can lead to a change in your over-ambitious wardrobe purchasing plan. That’s why you cracked the numbers in the first place.

Still, complaints about “exaggerating models” are unfortunately predictable. All that fuss about a hole in the ozone layer, and it turns out it doesn’t grow anymore! (Because we’ve banned the production of the pollutants responsible.) Acid rain would be a catastrophe, but I haven’t heard from it in years! (Because we needed pollution control on sulfur-emitting chimneys.) In the past, the worst-case climate change scenario was warming more than 4°C by 2100, and now they’re getting closer to 3°C! (Because we’ve taken stopping measures to reduce emissions.)

These complaints seem to see models as crystal balls or psychic visions of a future event. But they are not. Models simply take a scenario or hypothesis you are interested in and “calculate the consequences of the gamble”. The result can be used to increase scientific understanding of how things work or to make important decisions.

After all, what’s the alternative? Can science reject models in favor of another method? Imagine what would happen if NASA looked at Mars in a telescope, aimed the rocket, hit the launch button and hoped for the best. Or maybe humanity could base its response to climate change on someone waving their hand to the atmosphere and saying, “I don’t know, 600 parts per million carbon dioxide doesn’t sound like much.”

Obviously, these are not alternatives that any reasonable individual should seriously consider.

The spread of COVID-19 is an incredibly complex process and difficult to predict. It depends on a number of things that have been well studied (such as how pathogens can spread between humans), some that are partially understood (such as the characteristics of the SARS-CoV-2 virus and its lethality), and some that are unknowable. (such as the precise movements and actions of every American). And it needs to be simulated on a fairly small scale across the country if we are to understand the extent to which hospitals can meet local healthcare demands.

Without computer models, we’d be spitting behind the envelope – and even That would require conceptual and mathematical models for individual variables. The reality is that great science requires great models. Those who claim otherwise are not defending a “pure” scientific method. They just don’t understand the science.

We can no more rid science of models than we can rid it of knowledge.

By akfire1

Leave a Reply

Your email address will not be published.