We can start to predict—to rewrite the history of my electronic medical records experience—which patients will benefit the most from which therapies. And we can do that with algorithms built on the fact that we can compare—benchmark—those patients we are trying to predict for against the global set of patients who have been treated before them, and who are like them based on biology, physiology, and behavior.
As you grasp how fundamental this progression is in data science—as fundamental as gravity is to physics—an incredibly important aspect of scaling data science emerges. You can see that the relationship between effort and value is not linear as you advance through the progression of milestones. Cleaning and standardizing data is hard. It took Dr. David Fajgenbaum years to collect data, months for Medidata to work with his team to clean and standardize it, and ultimately just minutes to run the algorithms used to subtype Castleman disease patients. Artificial intelligence algorithms create huge value, they are what will power so much of our future, and they are the part of the process that—if you are a scientist—is truly sexy. But they don't work without tremendous effort to make sure that they are built on the right foundations. Much as we remember the failed Mars probe, the argument can be made that we can see this lesson in the business failure of Watson for cancer treatment recommendations.
Once the data is robust, accurate, and reliable, we can expand our thinking into more creative uses for it, especially as applied specifically to the world of clinical trials, which is where we turn in the next chapter. As I've learned more and more about what our clients are trying to achieve, I've realized how much more we can do to build upon what a standard clinical trial has looked like for a generation. To take data from one trial and use it in another—synthetic controls—or to adjust trials on the fly based on the results—adaptive trials—can make it orders of magnitude easier (and cheaper) to gain new knowledge.
And, once we incorporate this new clinical trial knowledge with our advancements in understanding behavioral and cognitive phenotypes, as well as all of the other layers of data we've discussed in the book thus far, we can empower the life sciences industry with enough information to create medications and treatments with more and more specificity, and ultimately make real gains for patients, ones that were impossible to envision when I first started on my own data journey a quarter of a century ago.
Notes
1. Ian Steadman, “IBM's Watson Is Better at Diagnosing Cancer than Human Doctors,” Wired UK, February 11, 2013, https://www.wired.co.uk/article/ibm-watson-medical-doctor.
2. Casey Ross and Ike Swetlitz, “IBM's Watson Supercomputer Recommended ‘Unsafe and Incorrect’ Cancer Treatments, Internal Documents Show,” STAT, 2018, https://www.statnews.com/wp-content/uploads/2018/09/IBMs-Watson-recommended-unsafe-and-incorrect-cancer-treatments-STAT.pdf.
3. Julie Spitzer, “IBM's Watson Recommended ‘Unsafe and Incorrect’ Cancer Treatments, STAT Report Finds,” Becker's Hospital Review, July 25, 2018, https://www.beckershospitalreview.com/artificial-intelligence/ibm-s-watson-recommended-unsafe-and-incorrect-cancer-treatments-stat-report-finds.html.
4. Heather Landi, “IBM Watson Health Touts Recent Studies Showing AI Improves How Physicians Treat Cancer,” FierceHealthcare, June 4, 2019, https://www.fiercehealthcare.com/tech/ibm-watson-health-says-ai-making-progress-clinical-decision-support-for-cancer-care.
5. Eliza Strickland, “How IBM Watson Overpromised and Underdelivered on AI Health Care,” IEEE Spectrum: Technology, Engineering, and Science News, April 2, 2019, https://spectrum.ieee.org/biomedical/diagnostics/how-ibm-watson-overpromised-and-underdelivered-on-ai-health-care.
6. Robert Lee Hotz, “Mars Probe Lost Due to Simple Math Error,” Los Angeles Times, October 1999, https://www.latimes.com/archives/la-xpm-1999-oct-01-mn-17288-story.html.
7. Lisa Grossman, “Nov. 10, 1999: Metric Math Mistake Muffed Mars Meteorology Mission,” Wired, November 10, 2010), https://www.wired.com/2010/11/1110mars-climate-observer-report/.
8. Robert Lee Hotz, “Mars Probe Lost Due to Simple Math Error.”
11
Changing Clinical Trials
The way we run clinical trials hasn't changed significantly in decades. Yes, we're doing things online instead of on paper, and, yes, most organizations are starting to be open to new types of measurements, in the home instead of in the clinic, with wearables and other new‐generation devices, and, sure, there is a growing sense that we can enhance the data set with new approaches and ideas.
But it is still early days, and trials are still on the fringes of the data revolution—for now, partly because of the huge expense involved in launching a trial—and thus the fear of taking an expensive risk on the unknown—and partly because there hasn't been much reason or industry impetus to move trial design, trial collection, or trial access forward. That has to change. It's that simple.
To reach the full potential that these new patient equations offer, clinical trials need to evolve. And they need to evolve in three ways. We need to change the way patients find and participate in trials (what people call “access”), we need to become comfortable collecting new kinds of data in new ways (going all the way in terms of scale from DNA to behavior and our environment), and we need to move to new mathematical designs. We need to innovate with new trial frameworks and techniques that will help us more effectively discover the inputs and outputs of the equations that will lead to maximum benefit for patients. In this chapter, we cover all of these issues in depth.
Expanding Access to Trials
According to the New York Times, fewer than 5% of adult cancer patients in America enroll in clinical trials, when greater rates would likely save lives—not just the lives of patients who enroll but the lives of future generations who would benefit from the increased research opportunities.1 And while part of the reason is the necessary eligibility requirements of certain research studies, it is also the case that patients aren't always (or even often) steered toward trials that might benefit them, or made aware that even those who will not receive the study drug will still receive the same standard of care treatment that they would outside the trial—and for free.
The New York Times piece pushes for clinical trial navigators to help patients find the trials that might benefit them most—but the education needs to spread throughout the health care system, with many clinicians needing just as much information and guidance as patients. While it is true that trials won't help everyone—another New York Times article cites numbers above 90% as the failure rate of precision medicine studies—it's also the case that the patients in these trials are generally the ones whose diseases have proven most resistant to standard treatments—and the only way to improve these numbers is going to be with further trial research.2
One advocate who has been fighting for increased trial access for years is T. J. Sharpe, who was diagnosed with stage 4 melanoma in 2012. “I was originally diagnosed with stage 1b melanoma that I had removed when I was 25 years old, and then 12 years later I get diagnosed with stage 4.…[M]y son was only 4 weeks old at the time,” Sharpe told CURE magazine, a consumer publication focused on cancer.3
Faced with his diagnosis—and an oncologist telling him he'd be surprised if Sharpe survived even two years—Sharpe sought out immunotherapy trials, and after one failed trial, he tried again—and after years on the drug Keytruda, has been cancer‐free since August 2017.4 I talked to Sharpe, and he emphasized how difficult it still is for patients to even be referred to trials.5 Since diagnosis, he has worked as a patient advocate and consultant to the research industry, and emphasizes the need for more information—and more tools, for doctors and patients.
Run by the National Institutes of Health (NIH), ClinicalTrials.gov is a publicly available resource that describes itself as “a database of privately and publicly funded clinical studies conducted around the world.”6 I receive emails and calls regularly from those who have received unfortunate diagnoses (or know people who have), and see me as their connection
to the clinical trial industry. They have heard about the potential to access cutting‐edge therapies through clinical trials, and they want advice about what trials they might be candidates for, and which might be of the highest potential benefit to them. ClinicalTrials.gov is inevitably the tool I use to respond. It is not, however, without its flaws.
“ClinicalTrials.gov is not a user‐friendly tool, and wasn't really intended to be,” T. J. Sharpe explains. “It was designed to be a repository for results, but it's being used to find trials because right now it's the only way.…There often isn't a clear way to find the right trials for the patient, to match their health with the criteria in the database, to evaluate whether it might work well for them.”
Sharpe sees three primary problems with the current system: access to general knowledge about clinical trials, access to a useful database that can effectively match the right trial to the right person at the right time, and access to understandable results. “How do you evaluate apples to apples?” he asks. “If you have three different companies announcing trial results that all show efficacy, how can you compare them, when they might all have different endpoints, different populations, and more?”
Recalling our phase‐diagram ideal of matching the right patient to the right therapy at the right time, Sharpe is highlighting the problem that there is no view into the available set of clinical trials that gets us anywhere close. There's no way to see where an individual might map across the necessary dimensions to be effectively matched to the selection of therapies that could treat them. Of course, we might not know yet if a particular experimental therapy will work for a given individual. That's why we do the research. However, even seeing where those potential matches might be—in one comprehensive, organized, accessible place—could help trial volunteers have the best possible chance of finding something that might cure or slow the progression of their disease.
It's not just patients who don't have the information they need to compare trials—it's physicians, too. There's no guide to interpretation, no way to separate out subsets of the population so someone can see how a particular trial worked for patients like them—with similar genetics, general health, and comorbidities. There are simply not enough meaningful insights being generated from the data and exposed to either practitioners or patients.
Sharpe sees the patient equation challenge as a big one right now. The data is in many different silos, and the industry isn't as motivated to evolve as he would have hoped. Being patient‐centric is a buzzword, he fears, but doesn't actually translate into creating trial designs that let patients compare one drug to another, or figure out what trial might be best. His hope is that the life sciences companies will start to understand this and see that it is better business to get effective and actionable information to patients and doctors, and enable patients to become better advocates for themselves.
What might that translate into, from a practical perspective? Sharpe imagines a trusted source of trial information—far more user‐friendly than the ClinicalTrials.gov site—where life sciences companies and other health care providers can contribute to creating a one‐stop shop for patients, doctors, and researchers. If someone gets a new diagnosis, they or their doctor could log on and figure out what knowledge is out there, what the latest therapies are, what possible trials exist, and what the results along the range of therapies have been, not just for patients in general but for patients as similar to our new diagnosee as possible.
Pharma's Lack of Connection to Clinical Care
Alicia Staley is Medidata's senior director of patient engagement and a three‐time cancer survivor in her thirtieth year of survivorship. Staley was diagnosed with Hodgkin's lymphoma as a young adult, and then breast cancer in both 2004 and 2008. She agrees with T. J. Sharpe's take on the industry, and believes that pharma's lack of connection to clinical care is a primary source of the difficulty.7 “For too long,” Staley says, “pharma has operated in one lane and clinical care in another—with no crossover between the two.” This is confusing for patients, Staley argues, and holds back the ability of researchers to recruit patients into trials and then keep them in the clinical research system. It is all transactional—there is no long‐term relationship building, not with patients, not with patient advocates, and not with the health care industry at large.
In fact, it's too often the case that even doctors don't really know how clinical trials work. It's simply not relevant to most of their professional lives and what they do day to day. This ought to change if we want true industry collaboration. The current lack of such collaboration plays out at every level, from education to treatment. Staley believes the life sciences industry needs to realize that patient education and the creation of long‐term relationships with advocacy groups helps them—not just when it comes to selling them the next blockbuster drug, but for trial recruitment and data collection, and to truly get us on the road toward richer disease models. Just like Sharpe argues, Staley insists the data is far too siloed and that there needs to be cooperation. “The value isn't in data blocking or data hoarding,” says Staley, “because data that isn't being analyzed, shared, and used productively has no value. Data without action is valueless.”
And yet, Staley sees all kinds of things happening in the world of clinical trials that don't serve patients at all, particularly doctors refusing to share trial opportunities with patients just because they're based at another health system. Clinical trials are operated around physicians who are chosen to be the “investigators” in each individual study. If a physician thinks that a study might help a particular patient, but it's not a study that physician is an investigator in, the process will include a referral to a study investigator. That means the patient's treatment will be with another doctor, possibly at a different facility, and perhaps in a different health care system. The patient may benefit, but the doctor and hospital may lose a paying customer. This can unfortunately create disincentives for referrals out of the system. “You think it's about the loss of a potential revenue stream, but it can actually turn out to be the loss of a life,” Staley laments.
Unlike Sharpe, who sees the most promise in a technological solution to educate patients—the perfect trials database—Staley worries that an overreliance on technology can leave out lots of patients, at least right now, who aren't plugged in. “The industry needs to meet the patient where they are,” she says. At the same time, she sees a huge place for technology in collecting richer sets of patient data—as long as someone is listening to the signal. To instrument patients, “so that they can walk through life throwing off valuable data nuggets,” Staley says, “makes it easier for patients to live their lives, to avoid being constrained by constantly visiting doctors and wasting time.”
There's a mindset shift that needs to happen throughout the industry, Staley says, and it's the same kind of shift I talked about earlier, where patient equations can move us from a reactive system to a proactive one. Can we build systems that aren't just addressing things that have already happened, but can in fact educate patients even before there's a diagnosis? Can we instrument people to catch things earlier, when there are more treatment options, or to tell that a patient is becoming refractory to their treatment before it's too late?
“Pharma is still a mature industry that can make a lot of money without changing much,” Staley fears, “but they have to realize that by collaborating with patients, with advocates, with the health care industry, and with each other, they can really make a bigger difference in people's lives.”
Truly Patient‐centric Trials
The frustrations expressed by both Sharpe and Staley—about the difficulties of finding trials and navigating the problem of access—stem from a particular cause: trials are designed around investigators, not around patients. This is not an accusation against those who design, sponsor, and (like me) help to run them. It is a practical limitation, imposed by important scientific and regulatory requirements. Trials are, by definition, experime
nts. They need to be controlled—by which I'm referring not just to the usual controls where some patients receive a new drug and others receive some standard of care or placebo treatment, but also controlled in terms of consistency.
If some aspect of care is inconsistently handled for different patients in a study, it can create a confounding effect as far as determining whether or not a particular treatment is beneficial. In any good scientific endeavor, it is always easier—and by definition more reliable—to keep as many variables as consistent as possible. The worst case is an inconsistent variable that isn't part of the ultimate analysis. It's just creating the potential for inaccurate results. Keeping a trial limited to a particular set of investigators, through whom you know that the protocol will be executed consistently (meaning that the treatments and tests that define how the therapies will be evaluated for safety, efficacy, and value to the patient will be reliably performed and recorded) is a way to meet our ethical obligation to create a valuable result.
There are also regulatory requirements, sensibly and responsibly imposed by organizations like the FDA, to ensure that the protocol is being followed consistently. Experimental medications can often lead to unknown and dangerous side effects. The data sets that are produced in studies are not only the summation of the outcomes for the volunteer subjects, but they are also the information regulators use to make assessments for approvals.
Imagine a typical phase III trial, comparing a new medicine to the current standard of care, hoping for results that lead to drug approval. Not only are lives at stake for the hundreds of patients in the study, but also for the thousands, tens of thousands, and tens of millions more future patients for whom that drug may or may not be available down the road. Care is absolutely required to ensure the most reliable result, and that care is achieved through the current investigator‐centric paradigm.
The Patient Equation Page 16