As patients enroll in the study, and new data comes in, more knowledge about which biomarkers are associated with positive or negative outcomes is fed back into the assignment of subsequent patients to particular therapy arms. Instead of performing the statistical equivalent of tossing a coin to decide whether a patient gets the drug on study arm A or B, randomizing them to one of two arms of a study, when a new patient is enrolled in this kind of Bayesian trial, their biomarkers are used as a way to preferentially randomize them to therapies that have already been successful for patients like them.
This should sound familiar. Berry's approach is very much like building a steam table and defining the phase transitions that define what will more likely be a successful treatment for any given individual. The dimensions here are the biomarkers, not temperature and pressure. And although we can't—practically or ethically—test every therapy on every combination of biomarkers (which would be the experimental approach to generating a steam table), we can use Bayesian statistics to start with some assumptions about what those phase transitions look like (the initial probability distribution), and with each patient treated we can refine that function. In other words, we can refine the patient equation represented in the study as we go along.
The phase transition visualization of the way I‐SPY 2 works is mine, not Berry's, but the thinking that led to this book truly starts with Don Berry's advocacy and mentorship for this kind of trial design. Even without an appreciation for the mathematical advantages, simply realizing that adaptive designs result in more patients being exposed to therapies that are beneficial to them should be enough to see their advantages. In Berry's words, “You learn and confirm. And you see if your predictions can be reproduced.”
Breaking the Barrier
The I‐SPY 2 trial doesn't have a fixed set of therapies. There are 19 therapies that have been incorporated so far. Six have “graduated” as of this writing, and more will. Once there is enough data to confirm that a particular drug works well for patients with certain biomarker profiles, that drug is removed from the trial. The company that makes it can use the data generated for regulatory approvals, having gained further ground as far as the precision application of their therapy than would be possible in a traditional phase I–phase II–phase III clinical development program. So not only are there benefits to patients in the trial in terms of having likelier access to better therapies, but those therapies can be brought to market faster to patients waiting for them everywhere.
In the United States, the FDA has been supportive of Bayesian trial designs14,15—but there are hurdles because of the inherently (and in many ways responsibly) conservative culture of the pharmaceutical industry, as well as industry incentive structures that are designed to support the machinery of running studies in traditional designs. Plus, there are very practical limitations to this kind of study design. It requires significant coordination and collaboration. Recalling the protocol that describes every aspect of the therapeutic administration, there is the need for a “master protocol” that governs the overall design and operation of the trial. All of the participating companies, and all the organizations with drugs being evaluated in the adaptive design, need to work within that master protocol's framework. The requirements for regulatory and scientific rigor are no different than in a traditional study design, only there are more experimental therapies, more patients, and the trial runs over an even longer time frame. This adds significant complexity. Although it is very difficult to argue against the ethical and financial benefits of adaptive, Bayesian‐style trial designs like I‐SPY 2, the costs of the individual studies and the complexity of coordinating them remain barriers.
These barriers, however, can, should, and will be overcome, because there is huge benefit. These types of studies break the two‐patients‐per‐unit‐of‐evidence limitation we've operated with for centuries. In the simplest way, sharing a single control arm across multiple study arms with a range of new drugs means that we're able to reuse those control patients. With seven experimental arms, we effectively have 1.125 patients required per unit of evidence. And that is just the simplest view of a study like I‐SPY 2. Arguably, due to the learning nature of the study and the virtuous cycle of its design, we are creating even more evidence per patient.
Ultimately, that increase in evidence‐generating power will result in more and more trials like this. GBM AGILE (Glioblastoma Adaptive Global Innovative Learning Environment) is an ambitious new adaptive trial design conceived in 2015 to help speed knowledge about treatments for glioblastoma, an aggressive form of brain cancer responsible for the deaths of Senators Ted Kennedy and John McCain, among many others.16
Like I‐SPY 2, the GBM AGILE trial is designed to evaluate many therapies at once, with just one control group—meaning patients are more likely to get an experimental therapy, and, even more critically, more likely to get an experimental therapy that is right for them. Columbia University has been among the first institutions to enroll patients in the trial. Their news release explains, “Throughout the trial, tumor tissue from participants will undergo analyses to identify biomarkers that may be associated with a patient's response. As the trial accumulates data, its algorithm refines the randomization process, so that patients have a better chance of getting a treatment that appears to show benefit.”17
Columbia's Dr. Andrew Lassman, chief of neuro‐oncology, says, “This trial design offers a way to lower the cost, time, and number of patients needed to test new therapies for newly diagnosed or recurrent glioblastoma.”18 Given the poor prognosis for patients diagnosed with glioblastoma and the lack of effective treatment options, the need for programs like GBM‐AGILE is clear.
Having established the value of collaborative, adaptive designs like I‐SPY 2 and GBM‐AGILE, we now have to ask whether there are other barriers that can be broken down, or other ways for life science companies to collaborate that can go beyond even these studies in terms of generating evidence, getting safe and effective precision therapies to market more quickly, and creating breakthrough value for patients. I believe there are, and the discussion continues with the idea of synthetic control arms.
Synthetic Control Arms
The idea of a synthetic control was first described in a paper in 1976 by Stuart J. Pocock in the Journal of Chronic Diseases. “In many clinical trials the objective is to compare a new treatment with a standard control treatment, the design being to randomize equal numbers of patients onto the two treatments. However, there often exist acceptable historical data on the control treatment,” states the article's abstract.19
The idea is that we can synthesize historical patient data into a hypothetical control group that will function just as well as a randomized control group. As long as these two hypothetical sets of patients are equivalent based on the definition of the study—that is, if they share the same characteristics and meet the right inclusion/exclusion criteria—they should function just the same. The idea isn't dissimilar to sharing patients across the arms of a multi‐arm Bayesian design like the I‐SPY 2 trial. We have a protocol being rigorously followed by the investigators, and patients who all meet the criteria for being part of the study—should we not be able to reuse the data?
If we are looking for, say, a group of patients with heart disease who will be given a particular study drug, we already have data from many other trials with heart disease patients who have been given the standard‐of‐care treatment as part of a control group. Why not reuse their results? Or, at the very least, why not reuse the data generated in those previous studies to supplement the new data we're obtaining in the trial? The word “synthetic” can be confusing here. It's not that the control patients are somehow synthesized. These are very real participants in clinical trials, just not the clinical trial currently being performed. The synthesis is of their experiences as control subjects into a new control cohort, a new arm synthesized from data deriving from other rigorous and scientific clinical trials.
Mathematically, Pocock made the case in his paper that we can—and, from the perspectives of cost, time, and ethics, we should—approach control groups in this way. Think about those two patients, one experimental and one control—tens, hundreds, or thousands of times over—whose data is finally assembled into a data set that shows the difference in outcomes for those who are getting the experimental therapy and those who are not. If we can reuse control patients from previous studies, we should be able to save half the cost. We only need to enroll patients who are to get the experimental therapy, because the control group is already taken care of.
Time becomes an issue here as well. Assuming there are no shortcuts in how long it takes to evaluate whether or not a particular therapy works, the idea of saving time with synthetic controls may seem unintuitive to those who don't work in clinical research. If it takes 12 months to go through the course of therapy and see if, for instance, a tumor stops growing, we're not going to save any time by reusing data from other trials. However: the number of patients we have to find goes down—in the most extreme example, by half.
The time it takes to recruit patients into a study is typically one of the key rate‐limiting parts of the process. Assume we need to enroll 120 patients in a theoretical study, and are able to find 10 participants a month. (Those numbers are reasonable for many studies. In some cases, it can be even harder to recruit, and the trend as we move toward more precise therapies—as we've already discussed—is going to be in that direction.)
Between the time we've enrolled the first and last patients into the study is a full year. Then add a year for the last patient recruited to get through the complete course of therapy, and then the time it takes to evaluate the effectiveness and safety. If we could reduce the number of patients we need to enroll by half, six months in this case gets shaved off the timeline. A regulatory submission could happen six months earlier. Six additional months of patients diagnosed with the condition the drug addresses could have broad access to the new treatment.
Even if the treatment doesn't work better than the standard of care, we've avoided asking 60 additional patients to take the chance to get randomized to a therapy that isn't going to help them, and give up the opportunity to be enrolled in studies that could serve them better. Particularly in therapeutic areas where there are no effective therapies currently on the market, we've created more chances for success. We've minimized the number of patients exposed to something harmful, and maximized—during and beyond the study—the number of patients who can benefit from more effective medications.
So why—with what appears to be a huge set of advantages—isn't this done more? It's certainly on the radar screen of many. Julian Jenkins—the former GlaxoSmithKline executive who worked on Flumoji—says that pharma is absolutely looking at this. “If I know the old drug worked against a particular target,” he says, “then, in looking at whether a new drug is going to work, many companies are trying to use secondary analysis, looking back to test hypotheses, to validate targets. It speeds things up, and is a huge enabler for the industry.”20
And yet, the standard of care—the gold standard—for clinical evidence is still the randomized, prospectively‐controlled trial. Why? The answer is partly because of the conservatism of the life sciences industry (always important to point out that this is a good thing that protects us all), and partly because creating synthetic control arms with previous clinical trial data isn't an easy thing to do. Most trials—virtually all trials, unless they are part of a master protocol like GBM‐AGILE or I‐SPY 2—have their own unique combination and cadence of visits to the clinic, lab tests, imaging, and so on. The way that studies are designed and run—one at a time, on one particular drug—means that every study's data set has its own unique design and particularities. It's not just a matter of taking data from studies, pooling it, and using it again. We need to ensure that the data is high‐quality, standardized properly, and consistent. That consistency across trials isn't always possible to find right now.
Even once the data is standardized, the hard work isn't over. Although we try to eliminate biases from clinical trials, there can still be inherent issues. Take the simple example of age. A clinical trial (and, again, this is a very reasonable example) may have inclusion criteria that patients must be over 18 and under 65. With a standardized data set, it is simple enough to find patients who meet those criteria. But what is the distribution of age across the synthetic controls? Do we use a normal distribution, centered around age 42? Perhaps our synthetic control group skews younger, and perhaps the distribution of ages among the prospectively enrolled patients skews older. Should this matter? Might this create a confounding element in our synthetically controlled study that makes analysis harder, not easier?
The answer in this case is that we don't know, and therefore we must do anything we can to make the characteristics of the patients—at least all of the characteristics that we know about—as consistent as possible with the prospectively enrolled patients. We'd like the inputs of the synthetic controls to look as much like the inputs of prospectively enrolled patients in the study as possible.
The work doesn't end there. The outputs of previous studies need to emerge in a way that is matched to the new study being performed. Raw data collected in the previous trials needs to be recleaned and the results recalculated. And before you even get to all that standardization and cleaning, you need to actually find all of the raw data from those previous studies. This data may be in deeply‐buried folders in servers across dozens of pharmaceutical companies. This data may not be well‐cataloged or indexed in a way that makes finding all of it possible.
Our Synthetic Control Model
The effort needed to successfully reuse clinical trial data is why, over the four‐plus decades since Pocock wrote his article in 1976, the idea of synthetic controls hasn't been scaled. This is where my company, Medidata, comes into the story. Barbara Elashoff, who broke her leg back in Chapter 2, along with her husband Michael Elashoff and their former colleague at the FDA, Ruthanna Davi, all reassembled at Medidata and began to work on the idea of synthetic controls with an advantage that nobody previously had: a platform where clinical trials had been run for over a decade with consistent data definitions, all the data in one place—in the cloud, where the studies themselves were run—and an organization placed centrally and able to ask more than one thousand life sciences companies if they would like to pool their control data for this greater good. Not to mention, we also had a commercially‐sustainable business model, where the costs of standardizing data and managing the complexities of synthetic controls would be an incentive, not a disincentive, to make them work at scale.
Every patient who is in the volunteer pool—with consent and permission from the patients and from the companies involved—is treated as equal: as members of one giant clinical trial data set. For any given indication, relevant patients are selected and those exposed to experimental therapies are excluded. Matching algorithms—ways to look at each individual as a matrix of data—are used to ensure that the types of skews present in different variables won't impact downstream analyses, and, ultimately, from a data set synthesized across what is effectively the breadth of the life sciences industry, a synthetic control arm emerges.
It is still, to be sure, not a trivial process. This is another example where the execution of analyses is fast, but there are painstaking months—sometimes years—of necessary preparation and the testing of techniques. However, along with some of the life sciences companies that participate in the project with us, we've paved the way for these data sets to be used—both for planning purposes and as a supplemental data source to benchmark trial results. Soon—perhaps by the time you are reading this book—synthetic controls will be used as part of the statistical package submitted to regulators in the approval process for a new medication.
The inherent, incredible, game‐changing value of being able to reuse patient data to create more evidence will probably offer f
urther surprises—in a good way—for those creating therapeutic value for patients, and for the patients waiting to receive it. For a pharmaceutical or biotechnology executive, even if a synthetic control isn't part of the data they submit to a regulator, or used to prove the value of a new therapeutic to a payer or a provider, imagine the value of knowing that your control data “looks” like control data in other studies.
The need to eliminate biases in clinical trials should be clear. A drug that appears to be safe and effective in research, but has had biases introduced into that research, can result in bringing a drug to market with elevated expectations. Whether it's because of the criteria by which patients were selected, the geographies where investigators were chosen, or something else about the investigators or institutions, having elements that skew not the inputs but the outputs of the study—the endpoints related to survival or quality of life—can have tremendous consequences. Comparing a control arm to a synthetic control and seeing that the standard of care or placebo arm in a randomized, controlled trial matches your results will mean more confidence that you have guarded against that risk.
That's just the starting point. The value of generating high‐quality evidence faster and more efficiently has been discussed. Once therapies have been approved using synthetic controls as a replacement (or at least as a supplement) for prospectively enrolled control subjects, I expect the life sciences industry to embrace the idea at scale. The current exception—the occasionally‐presented idea at scientific meetings—will become the new standard by which therapeutics go from theoretical value in a laboratory to therapies available to the public.
The Patient Equation Page 18