Setting policy in a pandemic is about trading off between alternative bad outcomes: coronavirus deaths, chronic virus-related illness, postponed medical procedures, economic devastation, missed education, increased mental illness, and on and on. There is no way out of a pandemic without incurring costs on some, or all, of these dimensions. But to chart the best course, we need, at a minimum, a good way of counting the toll for each.

Locally, most public schools remain closed and two recent Gazette reports highlighted one way of measuring the impact of this. As feared, children from low-income and racial/ethnic minority demographics have experienced a disproportionate rise in absenteeism under remote education. Doubtless, this will widen the already unacceptable gap between the most and least fortunate children in our society. Other negative effects of non-pharmaceutical interventions (NPIs) have impacted millions of adults: devastating social isolation in people who live alone, economic ruin for small business owners, financial and mental health crises in the unemployed, increased domestic violence against women, and more. These negative impacts of pandemic-mitigation measures will take years to measure; counting their toll as the pandemic unfolds is almost impossible.

What is certain is that both the virus itself and the NPIs it has invited have caused public health crises of unprecedented magnitude; both COVID and the lockdowns will lead to significantly reduced life expectancy in the worst afflicted groups. In deciding how to balance them, it is critical that we have an accurate estimate of the current and likely future impact of this devastating viral disease. How many people is COVID killing, now, in Massachusetts, and how many more virus-caused deaths can we expect to see under different policies?

The most straightforward way to do this is to look at COVID-related deaths in the comprehensive daily data summaries provided by the Massachusetts DPH (published on Mass.gov, the New York Times, and Worldometers.info). These data show COVID-related deaths at a steady level of 12-15 per day during the summer, followed by an increase, albeit not yet an exponential one, since October.

COVID deaths in Massachusetts this Fall have therefore been lower than in many states, but certainly not at zero. In our attempts to minimize the harsh effects of lockdowns, how should we set a threshold for determining the virus to be sufficiently “under control” to allow businesses and, most importantly, schools to open? A related question: are we sure that our measures of COVID-caused deaths are accurate? Are they too conservative (missing many cases) or too lax (chalking up deaths to COVID inappropriately)?

A hint that the counting of deaths is a difficult business comes from large variability in Case Fatality Rates in different regions ? that is, the number of deaths divided by the number of detected infections. (This is always an over-estimate of the Infection Fatality Rate, the number of deaths per number of infections, because we will always miss a greater proportion of COVID infections than of COVID deaths). However, if we examine two regions in which the per capita testing rate is similar, and hospital care can be assumed similarly effective, we’d expect the CFR to be about the same. In mid-September, Massachusetts was recording about 14 daily deaths against a backdrop of 355 daily detected cases 2-3 weeks earlier (assuming a 2-3 week mortality lag). This equates to a CFR of 3.9%. These data should be interpreted in the context of the daily testing rate of 638 per 100,000 people, keeping in mind that as the testing rate goes up, we catch more cases, and so the CFR should go down. In New York state on the same dates, the CFR was 1.3%, despite a testing rate of 443 tests per 100,000. That is, even though the testing rate was almost 1.5 times higher in Massachusetts than New York, the CFR in Massachusetts was 3 times greater. In Connecticut, the CFR was 1.3%, and the testing rate was 438 per 100,000: again, MA was reporting a death rate 3x higher despite a testing rate 1.5x greater. It seems implausible that Massachusetts, with some of the best hospitals in the world, was delivering significantly poorer clinical care than these other states. Therefore, either NY and CT were undercounting their COVID deaths (perhaps not testing enough), or Massachusetts was over-counting them (perhaps ascribing COVID as the cause of death in individuals who had multiple acute or chronic health problems at the time of their passing, of which COVID was not the most problematic). How can we determine which was true?

A reliable way to assess the accuracy of the recorded death toll is to look at excess mortality from all causes, by comparing deaths in 2020 to deaths in recent years. The CDC provide these data in week-by-week plots. Excess deaths tell us indirectly about the number of deaths attributable to this extraordinary disease by allowing us to factor out the number of deaths we should expect from other “ordinary” causes ? a number which is seasonal, increasing reliably every Fall. The plot below shows weekly deaths in Massachusetts, from 2017 to 2020, with the orange line representing the threshold for excess deaths (relative to the same week in years 2013 – 2019, see technical notes here). The devastating impact of the first COVID wave in MA is evident in the 11 weeks marked with a red cross (+ means deaths exceeded the threshold). But since early June, only a single week in October reported excess deaths, with a total of 4 deaths above the excess death threshold (the plot includes data reported up to the week ending Nov 21).

The same plots for New York and Connecticut (find them here by selecting the relevant jurisdiction) show a similarly devastating Spring wave, followed by a very modest second wave, but one that does include some excess deaths in recent weeks (although in NY this “second wave” resembles the 2018 seasonal flu in terms of excess mortality). The overall picture ? with CFR higher in MA than in NY and CT, but excess deaths lower ? suggests that, rather than NY and CT under-counting deaths to produce a falsely low CFR, Massachusetts may be over-attributing deaths to COVID, resulting in an over-estimated CFR. In line with this, the cumulative number of COVID deaths had reached ~10,600 in Massachusetts by the end of November, but this number does not square with the cumulative excess mortality in MA from February to the end of November, which stands at around 7,500 (calculated from data here).

To be clear, I am far from suggesting fraud or scientific malpractice in the counting of deaths in Massachusetts. The attributing of deaths to COVID is tricky. The discrepancy of around 3,000 deaths (between excess mortality and COVID mortality) can perhaps be explained by the fact that the average age of COVID deaths is 81 years in Massachusetts, with 98% of deaths occurring in individuals with underlying conditions. In other words, many deaths are occurring in frail individuals with multiple co-morbidities, and it must often be impossible to distinguish between people who die of COVID (the virus is unambiguously the cause of death) and people who die with COVID (having had a positive test, but with the cause of death being a complex mixture).

In view of these analyses, it is clear that excess mortality is the best measure of the impact of COVID on society (even though it may even overestimate the true count, if the extraordinary circumstances of 2020 have increased other kinds of deaths, e.g., from dementia, untreated heart disease or unscreened cancer).

At the present time, we have no good measure of the long-term impact of school closures on children’s health and cognitive development, or of the economic ruin delivered by lockdowns on adults’ life expectancy. We know that the impact is likely, at least for some, to be devastating. As measured by excess mortality, the current impact of COVID on the Massachusetts population seems insufficient to justify the complete or even partial closure of elementary schools. It’s time to count the impact of COVID accurately and weigh it more carefully against the impact of our mitigation measures.

Rosie Cowell (rcowell@umass.edu)

Thanks to Carlo Dallapiccola, Dave Huber and Adrian Staub for helpful discussion and comments.