Prof Talk | “We are taking decisions based on bad numbers”
Many decisions taken to combat the new corona virus rely heavily on mathematical models. Maria Vlasiou, associate professor at TU/e’s Department of Mathematics and Computer Science and part-time full professor at the University of Twente, teaches epidemiology on the Bachelor’s course Healthcare Management & Modeling. She stresses that the rather restricted testing policy in the Netherlands, as well as many other countries, leaves scientists with a lack of data. As a result, policy decisions are having to be made based on little more than educated guesses.
Any pandemic will only come to an end once a large proportion of the population has become immune, Maria Vlasiou says. “This immunity can be due either to vaccination or to having recovered. The hope that we will manage to contain all infected people ‘immediately’, thus stopping the disease is in my view a futile one.” This means that we will have to take measures to mitigate the impact of the virus as far as possible, this expert in applied probability and operations research thinks.
“Policy decisions are being made based on models like the ones I teach, and that Jacco Wallinga and his modeling team at Dutch healthcare organisation RIVM are building. However, these models are based on parameters that we need to know, such as the transmission rates and recovery rates of groups that determine the dynamics. These parameter numbers are in turn based on knowing real-time data reflecting how many people are infected, have recovered or have died. When the Netherlands does not actively test, or does not even take enough random samples, how much stock can you put in the numbers given for any of these categories? We need better data for these models, and thus for these decisions.”
Epidemiology
Vlasiou stresses that she is not an epidemiologist, but she certainly has some experience: as a postdoc researcher she was involved in an epidemiological study of the spread of HIV among men in the United States who have sex with other men. “When I came to TU/e, I started teaching the epidemiology part of the course Healthcare Management & Modeling. In short, from a mathematical perspective, in terms of models in epidemiology, I’m an expert. However, I’m not an epidemiologist. The best part of that science is not the statistics or the modeling, but the domain knowledge.”
This domain knowledge includes, for example, how to divide the population into groups sharing relevant characteristics: such as people aged over sixty or people with lung conditions. “Choosing which groups to include so as to produce a meaningful model that captures the dynamics of the disease, but does not depend on too many parameters that need to be estimated is the art of the epidemiologist.”
Concerning the question whether we would have been better off now had the Dutch authorities been more proactive, this Greek scientist is clear. “I think the data on confirmed COVID-19 deaths speaks for itself; the Netherlands has a very sad position if you normalise the data per million people. Why should Austria, Greece, or Portugal be doing better? In the Netherlands, other than taking more relaxed measures than most Western countries, we also chose not to test.” However, she does not think that individual patients would benefit from more testing. “At the end of the day, doctors treat your respiratory problems regardless, and measures are being taken that hopefully isolate everybody with a cough.”
However, not testing aggressively creates two problems, Vlasiou says. “Firstly, it does not isolate infected people who don’t show any symptoms, thus helping to spread the virus. But as a mathematician, the biggest problem I have with it is that it means we lack good data. We are taking decisions blindly. I must say that I do not put much stock in data approaches to this situation, such as the predictions made by my colleague Edwin van den Heuvel. My view is that if you start with bad data, you'll end up with a bad answer. It is clear by now that many deaths due to COVID-19 are being missed, as well as many infections. I am recovering from pneumonia right now, which might just be a coincidence, but I don’t know. We really don’t know if we are off by a factor of 2 or 100. Is the death rate 0.05% or is it 40 times higher? If this pandemic looks like the 1918 flu pandemic, we could be talking about 40 million deaths globally.”
Dangerously naive
Reacting to an article about the Dutch situation published in Science Magazine, Vlasiou stresses that what worries her is the certainty expressed about the parameter values in the models. “For a mathematician, this seems dangerously naive. NO parameter exists that is “fairly known” in any real-life model. I accept that at RIVM there are highly trained specialists who know their models and parameters and analyses inside out, and as well or even better than me. Maybe the parameter uncertainty has indeed been taken into consideration, for example through an extensive sensitivity analysis. But given a model, I can come up with any outcome you like provided I tweak the parameters appropriately. Every line of that article that was saying how “fairly sure” they are about this or that only increased my anxiety. To me, it seems fairly clear that we are taking decisions based on bad numbers.”
Discussion