On January 20^th 2021, the WHO issued new guidance for the use of gold-standard PCR testing for detection of SARS-CoV-2 (COVID-19). Considering this new guidance, and in light of the massive degree of testing currently taking place in the U.S., and Massachusetts in particular, I discuss what it means to receive a positive COVID test result.

Most medical tests are performed after a patient shows up in a doctor’s office or the emergency room and the test is chosen based on the patient’s symptoms. In such cases, a doctor uses the test as a diagnostic tool, attempting to identify the root cause of the symptoms. But in some cases, medical tests are used in the absence of symptoms, as a screening tool. This is common with cancer screening because the prognosis is better if cancers are identified before they cause symptoms. Asymptomatic screening for infectious diseases is less common, with HIV screening a notable exception. However, even HIV screening fails to come close to the level of screening that is currently deployed for COVID-19. The numbers screened for COVID-19 are staggering. For instance, among countries with a population greater than 10 million, the United States ranks second in the world in its rate of testing, with 878,000 tests per million (the UK ranks first, with 968,000 per million). In other words, there has been nearly one test for every individual in the U.S. (retrieved from worldometers on January 20^th 2021). Among states, Massachusetts ranks third (after Rhode Island and Alaska), with 1.819 million tests per million. Hence, in the state of Massachusetts, everyone has been tested nearly twice on average. However, screening for COVID-19 is not uniformly applied; in truth, many people have never been tested, while others (e.g., college students) are tested every week, or even twice per week.

Because this level of screening is unprecedented, it is worth considering the two types of errors that can occur with a medical test. Most doctors and labs are concerned with the false negative rate (which is one minus the ‘specificity’). The false negative rate is the probability that the test fails to detect the disease when the disease is present. This is considered the bad kind of error because a failure to detect would allow the cancer to grow, or, in the case of COVID-19, allow the individual to continue about their daily activities, potentially infecting others. The other kind of error is a false positive, which is a positive test even though the individual does not have the disease. In this case, the cost is financial and emotional, including: 1) an unnecessary quarantine, with consequences for employment; 2) a cost to friends and family who will also undergo contact tracing and testing, and likely quarantine; and 3) a cost to mental health owing to fear, social isolation, and unwarranted shame if the local community blames the individual for perceived reckless behavior.

These different kinds of errors are defined in terms of known states of the world (the “ground truth” of whether a person is infected or not). Therefore, determination of the false negative and false positive rates requires testing of people who have been independently verified as having or not having the disease, respectively. In the mathematical language of conditional probabilities, these are expressed as p(negative test | disease) or p(positive test | no disease), read as the probability (p) of a negative test result given that you have the disease or the probability of a positive test result given that you do not have the disease. But in truth, this is not what we want to know. Instead, what we want to know is p(disease | positive test) – i.e., the probability that an individual has the disease if they receive a positive test result. Fortunately, we can use a mathematical trick termed ‘Bayes rule’ to reverse the conditional probabilities. Without going into the mathematics of it, what we require for doing this are three things: 1) the false negative rate; 2) the false positive rate; and 3) the prevalence of the disease in the population.

Paradoxically, Bayes rule says that the thing we want to know (i.e., how to interpret a positive test result), depends on how many people have the disease in general. If a disease is highly prevalent (e.g., 25% of the population has herpes), then there is greater certainty that a positive test result is an indicator of infection. But if the prevalence is low, even a highly accurate test – in the sense of delivering low false positive and false negative rates – can be misleading. In their new guidance, the WHO reminds PCR users that:

“disease prevalence alters the predictive value of test results; as disease prevalence decreases, the risk of false positive increases (2). This means that the probability that a person who has a positive result (SARS-CoV-2 detected) is truly infected with SARS-CoV-2 decreases as prevalence decreases, irrespective of the claimed specificity.“

Consider an exceptionally accurate and sensitive test; one with a 0% false negative rate and only a 1% false positive rate (of note, many PCR tests appear to have a false positive rate lower than 1% — this value is used for illustrative purposes). In other words, a test that always detects the disease when it exists and rarely produces a false positive. Now consider what will occur if this exceptionally accurate test is massively deployed in the context of 1% prevalence (which is a prevalence that that is likely higher than the current prevalence of active COVID-19 infections). Paradoxically, when applied to everyone in the population (e.g., when used for asymptomatic screening, rather than as a diagnostic tool), a positive result with this highly accurate test means that an individual has only a 50% chance of truly being infected. In other words, if they do not have any symptoms or reason to expect that they are infected, and if they receive a positive COVID test, they are just as likely to be disease free as they are to have the disease. To make this concrete, consider applying this test to 100 people who do not have symptoms. Because the prevalence of disease is 1%, you can expect 1 true positive. However, because the test has a 1% false positive rate, you can also expect 1 false positive. So, among the 2 positive tests, one is true and one is false (i.e., only a 50/50 chance of the disease for any person who tests positive).

The solution to this problem is quite simple. Test again. This does not mean that you take the same specimen and run it through the test machine a second time; the false positive might have occurred owing to contamination of the specimen or from mis-labeling of the specimen. Instead, what it means is that you bring the individual back and collect a new specimen for a second independent test. Because this second test is independent, the probability of two false positives is the multiplication (the “square’) of the false positive rate, i.e., 1 in 10,000 for a test with a 1% false positive rate. Performing two tests lowers the probability of a misdiagnosis considerably, even with a low prevalence of the disease. The WHO writes:

“Where test results do not correspond with the clinical presentation, a new specimen should be taken and retested using the same or different NAT technology.“

Of note, the asymptomatic testing program at the University of Massachusetts Amherst does not advise a second test currently. Instead, after a positive test result, the prescription is an immediate quarantine, which serves to block the individual from seeking a second test (and furthermore, many testing sites will not test an individual who has already received a positive test result). For the Spring 2021 semester, UMass undergraduates are expected to take 2 PCR tests every week during the 15 weeks of the semester. In recent weeks, as the new in-house UMass test has been introduced, the asymptomatic testing program at UMass has seen its positivity rate rise to around 1% and sometimes higher – it is unknown whether this recent increase reflects an increase in true positives or in false positives. If the test has a 1% false positive rate, the chances that all thirty tests deliver a negative result as applied to a disease-free individual are .99³⁰ = .74. In other words, the chances of at least one positive test for this COVID-free individual are 1-.74 = .26; every COVID-free student would nevertheless suffer a 26% chance of receiving at least one false positive test result (a 26% chance of at least one needless quarantine)! This will produce a massive disruption to the community and entail serious consequences for mental health if 26% of the population can expect a false positive diagnosis at some point.

Considering that the goal of COVID-19 screening is to identify those who are infected before they spread the disease, there is in fact a third kind of error, which is a true positive result that comes too late. This can occur either if the test result is too slow in arriving or if the test is administered after the individual has already gone through their infectious stage of the disease, but still has enough virus to yield a low viral load positive test. For most diseases this third kind of error is unlikely to occur because individuals typically experience symptoms at the height of infectiousness (i.e., the disease is likely to be caught prior to this post-infectiousness stage). However, a sizable proportion of individual infected with COVID-19 never experience symptoms and may have progressed beyond the point of infectiousness before being screened for the disease.

This third kind of error is more likely with a highly sensitive test, such as PCR. A recent paper in The New England Journal of Medicine (Nov. 26, 2020) indicated that PCR tests applied to an individual with COVID-19 are just as likely to give a positive result after their period of infectiousness as compared to before or during their period of infectiousness. This is because the virus has a “long tail” and a low level of the virus remains for a long time after infectiousness, even though the individual has ostensibly recovered during this time. PCR tests use cycles (Ct) to amplify the signal to a set threshold and after many cycles even a very low viral load will be detected. This lowers the false negative rate, but it also raises the false positive rate and raises the rate for this third kind of error: a true positive that occurs after infectiousness. On the issue of viral load, the WHO writes:

“careful interpretation of weak positive results is needed (1). The cycle threshold (Ct) needed to detect virus is inversely proportional to the patient’s viral load…Action to be taken: Provide the Ct value in the report to the requesting health care provider.”

In conclusion, it is not always clear what a positive test result means, particularly when the test is used for mass screening of the population, and when the test is used repeatedly on the same individuals. If the prevalence in the community low, then the test may be a false positive even if the test is highly accurate, and the chances of this occurring grow with each additional test of the same individual. The WHO concludes their guidance by writing:

“Most PCR assays are indicated as an aid for diagnosis, therefore, health care providers must consider any result in combination with timing of sampling, specimen type, assay specifics, clinical observations, patient history, confirmed status of any contacts, and epidemiological information.“

Finally, I note that this discussion pertains to the use of tests for screening of asymptomatic individuals, rather than for patients experiencing symptoms, or for contact tracing where there is good reason to expect that an individual has been exposed to the virus. If there are other indicators of disease, then Bayes rule tells us that there is a much greater probability that a positive test accurately indicates the presence of disease. This occurs because the prevalence of the disease is higher among the population experiencing symptoms and higher for the population who have had a recent exposure to someone with the disease. In brief, this discussion concerns the ways in which things might go wrong when a test designed as a diagnostic tool is instead used for mass screening of the entire population.

David Huber: dehuber@umass.edu

Thanks to Adrian Staub, Carlo Dallapiccola, Rosemary Cowell, and William Cowell for helpful discussion and comments.

UPDATE 2/8/2021: The positivity rate for UMass asymptomatic screening has recently surged to above 2%. This seems to indicate an alarming increase in local infections. However, this positivity rate is approaching the positivity rate for symptomatic testing for the state of Massachusetts, but in general the positivity rate for symptomatic testing should be lower than the positivity rate for asymptomatic testing. UMass uses a mixture of two different PCR tests each day (an in-house version and the Broad institute’s PCR test). What’s needed is a breakdown by the two types of tests to ascertain whether they yield the same positivity rate. In any case, I still urge the asymptomatic testing program to follow WHO guidelines to obtain a second test following a positive test for asymptomatic individuals.

UPDATE 3/22/3021: William Cowell posted a comment in regard to asymptomatic screening in UK schools. In his comment he compares two different kinds of second tests that might be used after a positive first test. The graphic shown here provides some supporting information for his comment.