By Rachel Sachs, Jacob S. Sherkow, Lisa Larrimore Ouellette, and Nicholson Price
We have written before about the challenges of making decisions under scientific uncertainty and the simultaneous importance and difficulty of developing high-quality clinical evidence under pandemic circumstances. To address these problems, scientists and regulators in the UK developed a national-scale trial, the Randomised Evaluation of COVID-19 Therapy (RECOVERY) trial, with the goal of rigorously testing the most promising potential therapies for patients who have been hospitalized with COVID-19. In this post, we consider how the design and success of the RECOVERY trial yield important lessons for U.S. policymakers to consider going forward.
How was the RECOVERY trial designed?
In early March 2020, Oxford University scientists Martin Landray and Peter Horby recognized the crucial role well-designed clinical trials would play in combating the emerging pandemic and avoiding the mistakes of past health crises, where “everyone runs around like headless chickens with scientists and doctors acting alone, each testing different treatments in small numbers of patients; creating lots of noise, but no answers.” Landray and Horby consequently focused on four “keys” to the RECOVERY trial: size, speed, randomization, and simplicity. Without an easy way to enroll patients in a randomized trial, doctors would be left to make individual choices about experimental treatments and would not be able to generate good evidence of whether those treatments actually worked.
Broadly speaking, RECOVERY is a “platform” trial, a clinical trial “defined by the broad goal of finding the best treatment for a disease by simultaneously investigating multiple treatments . . . [where the] focus is on the disease rather than any particular experimental therapy.” (In other instances, “platform” trials are called “master protocol trials,” “basket trials, or “umbrella trials.”)
To achieve its aims, RECOVERY was designed to make randomizing treatments fast and simple for doctors. It is “multi-armed” with a single control, meaning that instead of inefficiently testing each intervention against its own control group, only a single control is needed to test multiple interventions. It is also “adaptive,” meaning that the trial design is modified based on interim data analyses, such as by randomizing fewer patients to less promising treatments and shutting down treatment arms that have been shown to be ineffective, such as hydroxychloroquine, lopinavir/ritonavir, and azithromycin. The RECOVERY trial can also add arms; for example, this fall it added aspirin, colchicine, and Regeneron’s antibody cocktail. Thus far, the trial has investigated 13 products, with positive results reported for dexamethasone and tocilizumab.
The RECOVERY trial benefits from the UK’s single-payer National Health Service (NHS), which has made it easy for 176 NHS hospitals to participate in the trial. Because all NHS hospitals use the same electronic medical record platform, gathering the resulting data is comparatively easy. The trial has also expanded internationally to include selected hospitals in Indonesia, Nepal, and Vietnam, increasing the diversity of the patient population. To date, RECOVERY has enrolled nearly 40,000 participants.
RECOVERY isn’t the only COVID-19 clinical trial to have this adaptive, multi-arm design. Similar trials include the WHO’s Solidarity, the European Discovery, and the NIH’s ACTIV. But those trials faced other challenges: Solidarity was delayed by the need to navigate regulatory approval in numerous countries, Discovery failed to meet recruitment goals due to lack of coordinated international funding, and ACTIV was hampered by the fragmentation of the U.S. healthcare system. Despite delays, these efforts are catching up, if slowly. For example, ACTIV has numerous trials underway, and as of October 2020, the Solidarity trial had recruited over 12,000 patients and reported interim results that “all 4 treatments evaluated (remdesivir, hydroxychloroquine, lopinavir/ritonavir and interferon) had little or no effect on overall mortality, initiation of ventilation and duration of hospital stay in hospitalized patients.”
What did the RECOVERY trial achieve?
Now, with roughly a year of hindsight, it appears that the RECOVERY trial was able to deliver on many of its principal aims. First and foremost, size. Other COVID-19 clinical trials frequently enrolled too few patients to provide robust statistical analysis of their investigations. RECOVERY, by drawing upon “all major hospitals in the UK on an unprecedented scale,” was able to overcome this deficiency in statistical power and answer, definitively answering pressing questions about the efficacy of treatments like dexamethasone and hydroxychloroquine. Aggregating many patients via RECOVERY also allowed the UK to avoid other nations’ wasteful duplication of research in the form of many small, inconclusive trials.
Because its size allowed randomization into numerous treatment arms, RECOVERY also delivered on the generation of high-quality clinical information about potential treatments. One can compare these results to the U.S.’s experience with convalescent plasma, which was authorized by the FDA even though an existing emergency access program was already underway; this made it difficult to enroll enough patients in the randomized plasma clinical trials and consequently determine whether convalescent plasma was truly effective. RECOVERY, by contrast, demonstrated the lack of benefit of plasma in its trials (later confirmed, although questions remain about plasma’s efficacy in other circumstances). The RECOVERY trial highlights, in many ways, the need for sound clinical trial data over stabs in the dark, “re-emphasizing the importance of high-quality clinical research” even during a pandemic.
RECOVERY was also able to deliver much of this information rapidly. By June of last year, the RECOVERY trial gave good results about a number of potential treatments, including dexamethasone (works), HCQ (doesn’t), and lopinavir/ritonavir (also doesn’t!). Later trial data similarly quickly confirmed the efficacy of other treatments, including tocilizumab (works) and azithromycin (doesn’t). This speed allowed important, life-saving treatments to quickly become the standard of care under the ticking clock of the pandemic, and to save—in a possibly hyperbolic estimate from the BBC—millions of lives. Doing so also saved patients by redirecting them to more effective treatment where many would have otherwise been enrolled in failing treatment arms. In the words of Professor Martin Landray, “It’s very, very rare that you announce results at lunchtime, and it becomes policy and practice by tea time, and probably starts to save lives by the weekend.”
At same time, RECOVERY has not been perfect. Criticisms have surfaced concerning the trial’s early release of data; a lack of transparency in the choices of treatment arms; and even that some arms were allocated more patients than necessary before being closed. Nonetheless, RECOVERY—perhaps superlative among other platform trials during the pandemic—has blazed a path forward for conducting such trials in the future.
What lessons can U.S. policymakers learn from the RECOVERY trial?
RECOVERY accomplished a lot; if U.S. policymakers want to replicate some of those successes, what should they consider? The fact that we might not need to run a U.S. version of RECOVERY this year shouldn’t get in the way of planning for the future.
First, health infrastructure matters. The combination of a single-payer health system and integrated electronic medical records (EMRs) made the creation and running of RECOVERY substantially more feasible. In the United States, neither of those factors exists; health care and health insurance are highly fragmented, including rampant incompatibility among different EMRs. To be sure, health care and health data fragmentation are much larger issues than the ability to run large, adaptive, multi-arm platform clinical trials—the RECOVERY tail probably won’t wag the fragmentation dog—but it’s worth noting that efforts at defragmentation might come with that benefit among others.
In the meantime, there are systems that start to come closer to an integrated system like the UK, including the U.S. Department of Veterans Affairs, Kaiser Permanente, or other integrated health systems. CMS, which operates Medicare and Medicaid, is also more integrated than the norm. Policymakers should consider leveraging such organizations’ capacities—including through exhortation, funding, and regulatory facilitation—to help run RECOVERY-like knowledge-generation efforts in the future. Defragmenting health data systems in other ways, such as requirements for interoperability between different EMR systems, should also help.
Second, national-level coordination and messaging are both important. RECOVERY’s fundamental strength came from coordination between many providers; while it is easier if those providers are part of a larger integrated health system, coordination is even more important if providers are independent. Parallel to coordination among providers is coordination between agencies (like FDA, CMS, and NIH), which would let agencies know who’s doing what and what sorts of evidence are being produced or can be developed under current conditions. Related to coordination is messaging: helping both providers and patients know what trials are ongoing, and encouraging them to participate, helps those trials succeed. For RECOVERY, the UK’s five most senior doctors wrote a letter to all NHS hospitals encouraging enrollment and saying: “Use of treatments outside of a trial, where participation was possible, is a wasted opportunity to create information that will benefit others.” Such arguments tie into the push for a learning health system, where treatment is viewed as an opportunity to learn—and participation in the creation of knowledge as a responsibility of both providers and patients.
Third, administrative barriers matter. The planners of RECOVERY put substantial efforts into ensuring that it was simple for hospitals to enroll patients and for doctors to participate. Lowering barriers to entry is important—especially for situations like COVID-19, when providers were overwhelmed dealing with the onset of the pandemic. Clinical trials in the United States have encountered problems enrolling diverse patients, including during COVID-19; policymakers should make it simple and easy to participate. Along similar lines, especially in a pandemic, tying clinical trials to insurance coverage is likely to bias enrollment, since insurance coverage is lower for minority patients. Given the substantial social goods of representative, high-quality clinical trials, administrative hurdles should not stand in the way.
RECOVERY represents an outstanding example of coordination allowing the effective, efficient, and fast generation of high-quality information about high-stakes treatment. Given that such a task is endemically challenging in biomedical innovation generally, and novel diseases in particular, policymakers shouldn’t waste the chance to learn from what RECOVERY got right—and how its success can be followed in the future.
This post is part of a series on COVID-19 innovation law and policy. Author order is rotated with each post.
