Demystifying modeling: How quantitative models can–and can’t–explain the world | McKinsey

not-actually-wrong from McKinsey!


Demystifying modeling: How quantitative models can—and can’t—explain the world

June 25, 2020 | Article By Sarun Charumilind, Anas El Turabi, Patrick Finn, and Ophelia Usher Open interactive popup Demystifying modeling: How quantitative models can—and can’t—explain the world Open interactive popup The COVID-19 crisis has brought quantitative models to the forefront. Here are some ways that modeling helps us—as long as we avoid its pitfalls.

One of the many impacts of the COVID-19 crisis has been to highlight the role of quantitative models in our lives. Ideas associated with modeling, such as flattening the curve of disease transmission, are now regularly discussed in the media and among families and friends. Across the globe, we are trying to understand the numbers and what they mean for us.

Forward-looking models aren’t new. They have long played an important but unseen role in day-to-day life—for instance, in pricing homeowners’ insurance, anticipating the weather, and deciding how many iPhones to manufacture. However, in the COVID-19 pandemic, the scale of impact and the level of uncertainty have introduced new challenges—and notoriety—for modelers.

Used properly, models provide information that can present a framework for understanding a situation. But they aren’t crystal balls that state with certainty what will happen, and they don’t in themselves answer the difficult question of what to do. The eminent British statistician George Box summarized the point with his famous aphorism: “All models are wrong, but some are useful.” And he refined it by saying, “Since all models are wrong, the scientist must be alert to what is importantly wrong. It is inappropriate to be concerned about mice when there are tigers abroad.” Sidebar

What is a model?

This article explains how models can help us make sense of the world and why they behave the way they do (see sidebar “What is a model?”). We also discuss the most common modeling pitfalls and how to avoid them.

The power of models

Making decisions in the face of uncertainty is challenging, particularly during a pandemic. Quantitative models can help us understand systems and behaviors in a number of useful ways that help navigate this ambiguous environment.

Clarifying which drivers matter

Models structure data in support of reasoned decision making by restricting variables to those that matter for a particular question. For example, when developing a demographic model to help civic leaders plan future community needs, key drivers could be birth rates, death rates, and new-job creation. Models can help users understand what is known about each element and identify the areas of continuing uncertainty.

Determining how much an input can matter

Models are well suited to exposing sensitivities: they show how even small changes in key assumptions can produce large variations in outcomes, helping decision makers establish priorities. An obvious case in point related to the COVID-19 pandemic is the massive impact of even small adjustments in the transmission rate of infection. By establishing sensitivities, models pinpoint areas for investment of effort or money to reduce uncertainty.

Facilitating discussions about the future

Models expose how different assumptions lead to different outcomes. Through discussion of modeling results, decision makers can form a collective judgement on scenarios to plan for, based on the multiple variables considered, and thus reach practical decisions (see sidebar “Building a quantitative model while using it”). For example, models were used to enable policy makers to weigh the benefits of requiring seatbelts against the moral hazard of encouraging people to drive faster. Not only do models trigger discussion, but they may force a more nuanced and evidence-based approach to decision making. In many cases, that is more important than the specific output itself. Sidebar

Building a quantitative model while using it

Pitfalls to avoid when using models

Overlooking the fact that a model can’t fix bad data

Taking assumptions and simplifications for granted

The risks of bias in modeling

Expecting too much certainty

Modeling philosophy for the COVID-19 pandemic

continues in source: