By Nicholson Price, Rachel Sachs, Jacob Sherkow, and Lisa Larrimore Ouellette
Over the last few weeks, dozens of companies have begun marketing tests intended to determine whether someone has antibodies directed to SARS-CoV-2, the virus that causes COVID-19. These tests indicate likely immunity to COVID-19 and are different from diagnostic tests to determine whether someone is currently infected. However, recent evaluations have shown that these antibody tests are often failing to deliver accurate results. Last week, the House Oversight Committee called for more information from these companies about the accuracy of their tests, and the FDA has now responded by increasing its oversight over antibody tests. But why did the FDA’s initial stance toward these tests differ so strongly from its stance toward COVID-19 diagnostics, and what lessons should policymakers draw from these experiences going forward?
What are antibody tests, and why are they important?
As we discussed two weeks ago, there are generally two types of COVID-19 tests: RT-PCR tests, which test for the presence of the SARS-CoV-2 viral genome in a patient’s mucus; and serological or antibody tests, which test for the presence of antibodies directed against SARS-CoV-2 in a patient’s blood. The RT-PCR tests are used to tell if someone is currently infected; antibody tests tell whether someone was infected in the past. For that reason, antibody tests are not useful as a point-of-care diagnostic, but are useful for assessing a patient’s COVID-19 history and for public health research on the extent of COVID-19’s spread.
As governments face pressure to reopen their economies, there’s been an increased interest in using antibody tests to determine how many people have been infected and, importantly, who can move about more freely. Antibody tests, according to this policy, would become the basis for “immunity certificates” determining who must continue to shelter-in-place and who is safe to rejoin society. Although there is real potential for immunity certificates to be used in a discriminatory fashion, as in New Orleans’ nineteenth-century outbreak of yellow fever, if designed well and used better they may one day become a tool in the public health toolbox for combatting COVID-19.
Currently, however, SARS-CoV-2 antibody tests suffer from two major deficiencies: First, it is unclear whether the presence of antibodies against the virus actually means someone is immune to reinfection. Despite the WHO’s proclamation that “[t]here is currently no evidence that people who have recovered from COVID-19 and have antibodies are protected from a second infection,” there is strong circumstantial evidence suggesting this is, in fact, the case. But the truth remains murky, including about how long immunity might last. And second, as we note below, the antibody tests themselves aren’t terribly accurate. While false negatives are relatively rare, false positives are persistently high—and could outstrip true positives in a population, giving patients a false sense of security that they are immune from infection. There is some hope on the horizon, though, that antibody tests will improve—as seems to be the case with a test by Roche recently green-lighted by FDA. Nonetheless, the sheer prevalence of underwhelming antibody tests remains concerning.
What has the FDA’s position been on antibody testing so far?
Initially, the FDA took a highly permissive approach to serology tests, essentially using discretion not to enforce its normal rules and permitting developers to enter the market where (1) the test has been validated, (2) the manufacturer has notified the FDA of its intention to enter the market, and (3) a series of disclaimers accompany the test results. (See Section IV.D of the FDA’s guidance document here for more details about these disclaimers.) The validation studies recommended by the FDA include studies designed to determine the sensitivity and specificity of the test, as described below. Likely as a result of the FDA’s lack of review process, serology test manufacturers rushed to enter the market. As of April 30, 170 commercial manufacturers and 24 separate laboratories had notified the FDA that they have validated and are offering serology tests.
Manufacturers were also permitted to submit their validation data to the FDA for Emergency Use Authorization (EUA), in which the FDA performs a limited review of the relevant information and more formally authorizes the manufacturer’s entrance into the market. Although the EUA process is less formal and rigorous than the FDA’s typical review pathway, in this context it can still provide greater assurance that there are no major concerns with a company’s validation tests. As of April 30, 10 manufacturers and laboratories had received such authorization.
In short, the FDA was allowing antibody tests to quickly enter the market with little regulatory oversight. The agency has also been collaborating with the NIH and CDC to “establish a capability at NIH to evaluate serological tests for developers,” which “may complement and inform the determination of whether FDA issues an EUA for a particular test” in the future. But through April, the agency’s focus was on expanding antibody testing capacity rather than assessing the quality of the proliferating array of tests.
Today, the FDA changed its policy and announced that companies marketing antibody tests would need to apply for an EUA within ten business days of beginning to market their products. Companies who do not receive such authorization must cease marketing their tests.
Why might the FDA’s initial position have differed from the FDA’s view of diagnostic testing?
As we have previously explained, FDA regulators face a fundamental tradeoff between risk and access. Regulatory requirements can reduce safety risks for patients, but these requirements can also delay and limit access. The FDA was slow to relax restrictions on diagnostic tests, leading to widespread criticism of its role in delaying the expansion of testing capacity. Some observers have argued that because of this criticism, the agency “has swung in the opposite direction in overseeing tests for coronavirus immunity.”
We think overcorrection from its diagnostic response might help explain the FDA’s different initial approach for antibody testing, but it is also worth noting that the agency might reasonably have wanted to make the risk–access tradeoff differently here. Diagnostic tests for whether patients are currently infected are important for informing medical decisions. Antibody tests, in contrast, have little clinical significance—at least for now.
The risks of erroneous antibody test results may change quickly, however, if people incorrectly believe they are immune from COVID-19 as states open up. As Umair Irfan has explained, this concern means that antibody test regulations should emphasize specificity (having a low false positive rate) over sensitivity (having a low false negative rate). And the relatively low prevalence of COVID-19 in the U.S. population so far means that even a highly specific antibody test would not be that informative about which individuals are immune. For example, if 1% of people are known to have been infected (as determined by an RT-PCR test) and a test is 99% specific, then testing 100 people would likely yield one true positive and one false positive—meaning that an individual who tests positive would have a 50% chance of not actually having antibodies against SARS-CoV-2. Because of this concern, on April 28 the White House updated its testing blueprint to suggest “the use of two antibody tests rather than one” to “dramatically improve[] the predictive value of a testing program, particularly in low prevalence environments.”
What considerations should policymakers keep in mind going forward?
The problem of regulating functional but problematic technologies is a tricky one, especially when the existing technology has entered relatively widespread use. What we would like is to get as much value from the technologies as we can—here, to know how the pandemic has spread and potentially who may be safer from reinfection—but still to advance the technology so that the next iterations give us better information.
We may be seeing a version of this innovation story play out that is different and unusual in the FDA space. In a typical technology improvement story, there’s one well-validated product that is approved, and years later another may be, which is (hopefully) better. Here, though, rather one or a few developers demonstrating that their product works well and then winning approval, we have seen a large number of developers put products on the market—and many of them do not work well at all. Imposing a more stringent performance requirement at this stage of the process by imposing EUA requirements would cull the poor performers from the market, leaving us with a smaller number of remaining tests that meet the newly imposed performance standard (which could, for instance, have unusually stringent specificity and sensitivity standards). This story would reflect—ex post—a more competitive development process than normal.
It’s worth taking a moment to note the role of CLIA certification. Under the Clinical Laboratory Improvement Amendments, the Centers for Medicare & Medicaid Services (CMS) regulates all non-research human laboratory testing, and thus might be thought to play a significant part here. CMS could, for example, conclude that the risk of misinterpreting test results outweighs their limited clinical benefit, and impose special proficiency testing requirements for COVID-19 serology tests. But CMS more commonly focuses on the performance of the laboratory conducting the tests, not the quality of the tests themselves. The latter is squarely within FDA’s bailiwick.
The biggest question for policymakers is this: how should we make sure people know what these tests are good for? Right now, wide variations in test quality, combined with a relatively low prevalence of COVID-19 in the population (we think), means that results are often shrouded in uncertainty. This leads to a problem: if lots of people get antibody tests, and lots of them get poor-quality false positives, many people will incorrectly think they are not at risk of contracting or spreading the virus. Social distancing is still crucially important to keep viral spread down, but social distancing is compromised if groups of people stop distancing based on the false belief that they are safe. Policymakers should make clear to the users of antibody-based tests that a positive result does not mean they are safe. It provides a clue, but not enough to throw caution to the wind, at least until the tests are substantially better—and substantially better validated.
No comments:
Post a Comment