Don’t put too much faith in Covid-19 metrics – UnHerd – Tom Chivers

via Don’t put too much faith in Covid-19 metrics – UnHerd

Don’t put too much faith in Covid-19 metrics

Facts and figures are vital to fighting the pandemic, but they can also be hugely misleading


April 14, 2020

In this new era, we’re all becoming data nerds, or hobby-level epidemiologists. We’re all suddenly conversant in things like case fatality rates and R0.

It makes for an attractive amateur pastime because lots of the things we are trying to know — such as how many people are infected, or how deadly the disease is — are hugely uncertain. But there’s another problem, which is that the things we measure are affected by the simple fact that we’re measuring them — and that the right things to measure change with every passing day.

For instance, we’re all wondering about the “exit strategy” — how, now that we’re all in lockdown, we’re going to get out of it. It’s going to involve some combination of testing, contact tracing, perhaps (eventually) immunity passports, and hopefully in the not-too-distant future vaccines and treatments.

But it’s also going to involve — probably — see-sawing back and forth between tight controls and more relaxed ones, trying to eke out the cases over months to avoid overwhelming the NHS. And those controls will have to be imposed and relaxed, to some degree, on the basis of metrics.

In the UK, it might look a bit like this. The 16 March Imperial modelproposed a sequence of automated triggers: specifically, when the number of ICU cases in a week reaches a certain number, say 100, the lockdown (school closures, social distancing, etc) is imposed; when they drop below another certain number, say 60, they are relaxed. The outcome — hopefully — will be a saw-toothed line on the chart: ICU cases jagging up, coming down, jagging up, coming down, but never breaching the line of health service capacity.

But once you start using ICU beds as a metric, you hit a problem. There’s this thing, “Goodhart’s law”. It’s named after the economist Charles Goodhart, and is usually formulated as “When a measure becomes a target, it ceases to be a good measure.” Goodhart proposed it (in more technical language) when discussing Margaret Thatcher’s economic policies, but it applies everywhere.


Continues in source: Don’t put too much faith in Covid-19 metrics – UnHerd


compose new post
go to top
go to the next post or comment
go to the previous post or comment
toggle comment visibility
cancel edit post or comment
%d bloggers like this: