SCIENTIFIC PROGRESS AND MEDICAL REVERSAL
Some medical reversals are anomalies in the Kuhnian sense of the word. Reversal occurs because doctors adopt a therapy based on their understanding of the prevailing paradigm. When that treatment is proved ineffective, it not only is an anomaly, upsetting the paradigm; it also becomes a reversal. Other reversals do not represent anomalies. In these cases, doctors have adopted a therapy that does not really fit with the prevailing paradigm. When this therapy is reversed, it does not raise doubts about the paradigm; instead, it forces doctors to acknowledge that their practice made little sense within the paradigm.
In chapter 1 we discussed the COURAGE trial. COURAGE showed that even though chest tightness (angina) is caused by blockages in the coronary arteries that deprive the heart of oxygen, and even though those blockages can be remedied with stents, those stents do not save lives and offer minimal (if any) subjective benefits. Was this reversal an example of an anomaly, or an example of a result alerting doctors that their therapies were at odds with the paradigm? To answer this question, it is worth first briefly considering the history of the two cardiac processes that stenting addresses: angina and heart attacks (myocardial infarction, or MI).
The first published descriptions of angina appeared in the late 1700s. Some of the most accurate descriptions of this syndrome have been attributed to Dr. William Heberden. At the time, the exact cause was unknown, and treatments were symptomatic—“quiet and warmth, and spirituous liquors,” as well as opium. Great strides in the understanding of angina came in the late 19th century, and by the early 20th century, angina had gone from a syndromic description of chest pain to a symptom clearly related to blockages of the coronary arteries—a symptom that carried a poor prognosis.
The 20th century also saw the coalescing of a paradigm concerning the understanding of the MI. Although the first accurate description of the process may date back to 1844, it has only been in the past 25 years that plaque rupture has been accepted as the causative event in most MIs. The idea is that the walls of diseased, atherosclerotic, coronary arteries are unstable. If an atherosclerotic lesion ulcerates, or ruptures, it releases factors that can cause thrombosis, blood clotting, within the vessel. This blockage then leads to the death of heart muscle, the definition of an MI. The process of plaque rupture can occur in narrowed arteries, those that are causing angina, or in the walls of vessels that are diseased but not narrowed to the extent that they would cause symptoms.
Reviewing the current paradigm of angina and MIs informs how the results of the COURAGE trial can be viewed. As far as MIs are concerned, the idea that stenting stable lesions would be preventive made little sense. Plaque rupture can occur in any vessel, not necessarily the ones that are narrowed enough to cause angina. The finding that stenting does not save lives is not an anomaly; it just reinforces that the hypothesis made little sense within the paradigm. (Shockingly, however, a recent study found that 90 percent of patients believed the procedure would extend their lives and 88 percent believed it would prevent a future heart attack.)
For angina, the interpretation is more complicated. Placing stents does improve symptoms—this finding was expected, given the century-old paradigm holding that narrowed vessels cause angina. The fact that the benefit was as small as it was, compared with modern medical therapy, may someday be seen as an anomaly. The sham studies that we propose (chapter 2) would lend clarity to this issue.
HOW SCIENCE AND MEDICAL SCIENCE DIFFER
There is a critical point to make when we apply Kuhn’s thinking to medical sciences. Unlike pure science, medicine directly affects the lives of human beings. No one was really hurt when we believed in Ptolemy’s astronomy—we do not think anybody got lost on a misdirected space voyage in the third century. When a medical hypothesis turns out to be wrong, there is the potential for injury. Medical paradigms are the theories that help us understand the biology of health and disease. These theories are supported by diverse data—laboratory research, case-control studies, and observational studies. Medical science is a relatively new field that operates on well-developed scientific method; because of this, our paradigms have been robust. Revolutions in the modern medical sciences have been rare and limited, affecting minor paradigms in limited areas of the field. The greatest advances of the past century replaced preparadigmatic thinking and established paradigms.
When medical science functions properly, medical paradigms suggest hypotheses: for example, the hypothesis that tight blood-sugar control benefits diabetics. The next step in the scientific method is to test this hypothesis, in this case by a randomized trial testing whether strict blood-sugar control is better than lenient control. If the hypothesis is proved true, the paradigm is strengthened. If false, the paradigm is adjusted or an anomaly is recorded. Reversals happen when we jump the gun, when doctors start acting based on the paradigm rather than on the experimental results. They act as if the hypothesis is true, and the experiment is performed only years later, if at all.
Anomalies become reversals when we have already implemented them broadly, believing the theory to be the truth. Had people not yet been treated, we could say, “Back to the drawing board; let’s think more about that paradigm.” Thus, anomalies can be reversals, but they do not have to be. If a medical intervention is thoroughly tested before it is implemented and found to be in error, the theory is adjusted and no one is harmed.* This is the process every time a drug in development fails to make it to the pharmacy. This is the appropriate way to address anomalies. However, an anomaly that becomes a reversal can have a direct and negative effect on a human being who is ill. This does not happen in the nonmedical sciences.
WHY IT IS WORTH THINKING ABOUT KUHN
Before moving on to discuss more concrete causes of reversal, let’s consider three more lessons from Thomas Kuhn. First, Kuhn had the wisdom to realize that history books whitewash the missteps. In the modern world, there is better documentation of the past, but medical books still underemphasize how wrong we have been at times. It is hard to believe that just 15 years ago there was nearly universal enthusiasm for hormone-replacement therapy in postmenopausal women. Some practitioners’ current embrace of treating age-related testosterone decline in men suggests how completely we have forgotten our error. Because history underemphasizes how fervently smart people believed things that turned out to be wrong, we find it hard to believe that things we currently believe might someday be proved wrong as well.
Second, Kuhn saw experiments as tests of our current worldview, the current paradigm. This is how we should look at clinical trials in medicine. Our understanding that narrowed coronary arteries cause angina is supported by countless studies, but the COURAGE trial is an experiment that may have demonstrated an anomaly. In the future, we may reach a better understanding of how pain is caused by narrowed arteries, and this knowledge might suggest a different intervention that saves lives and improves symptoms.
Third, Kuhn reminds us that we have to design experiments that challenge our worldview. We should not only do experiments meant to support the paradigm. Although these experiments are important, they do not offer the potential advances that experiments questioning key assumptions do. This was the foundation of Kuhn’s contempt for “normal science.” In doing any type of medical science, though, be it normal or paradigm-questioning, you have to wait and see whether the proposed intervention works before you implement it. No theory in the history of science has been ironclad; the only way to know something works is to know it works.
JOHN MCKINLAY
In the final stretches of putting this book together, we came across the work of Dr. John McKinlay, whom we referred to in the introduction. McKinlay is a sociologist of medicine, who, before hormone-replacement therapy and antiarrhythmics for heart attacks, provided an outline for how medical practices come into vogue and fall out of favor; his outline is remarkably similar to our own research and understanding. McKinlay described “seven stages in the career of
a medical innovation,” and it is worth revisiting those stages, highlighting what has changed and what remains the same.
In 1981 McKinlay suggested that innovations are first announced with a “promising report,” often noting success in a patient and published in a prominent medical journal. In the second stage, the profession adopts the innovation, motivated by the belief that the innovation will benefit patients (as well as by some peer pressure and financial incentives). Stage three occurs when patients and payers accept the innovation as standard. In the fourth stage, “data” begin to enter the story. However, the data supporting the innovation come only from insubstantial studies that support the innovation in the most superficial way. In stage five, we see the first randomized controlled trial assessing the intervention. These studies may either support the innovation or prove that it is ineffective (medical reversal).* Stages six and seven are the reactions doctors have to the data; first denial, as entrenched interests deny that the innovation may not be effective and then, finally, acceptance. McKinlay’s conclusion about the solution to the problem is the same as ours: “All services must be evaluated objectively (preferably and where appropriate by RCTs) before they are introduced.”
So what has changed? Well, first, today a promising report takes many forms. McKinlay described it as a small pilot study, while today a promising report could be anything from a basic science or anatomical paper, an observational study, or a randomized trial that uses an inappropriate control arm, an unimportant end point, inadequate follow-up, or poor blinding. In fact, with the rise of the influence of the biopharmaceutical industry, more and more often, promising reports come in what appear to be well-done randomized trials, whose flaws are revealed only upon careful scrutiny.
Second, the role of perverse financial incentives likely plays a greater role in health care today than in 1981. In 1981 health-care spending in America as a percentage of GDP was 8 to 9 percent, while in 2014 that number is 18 to 20 percent. As many have noted, all these dollars have not yielded a proportionate increase in life expectancy, and in that sense the United States lags behind many other industrialized nations who spend less. These trends have also changed McKinlay’s narrative. Professional organizations and pharmaceutical companies have a strong interest in more medicine and a greater reluctance to abandon what has been disproved. In the next chapter we explore these forces in more depth. For now, we must marvel that John McKinlay saw so much of what was to come when evidence-based medicine was still in its infancy.
12 SOURCES OF FLAWED DATA
EVEN THE BEST RANDOMIZED controlled trials, or practice guidelines based on such trials, can mislead us. Doctors can read the literature, carefully analyze the trials, adopt only therapies that seem well-founded, and still offer treatments that turn out to be wanting. This is not only because sometimes we discover anomalies or work from incomplete paradigms. It is also because, in today’s world of biomedical research, many studies are funded by the pharmaceutical industry. There is a great temptation to create bias when the very companies that develop the drugs, devices, and infusions also design the studies that test whether these products work. How this bias is introduced varies. It may occur within individual studies in which the design is subtly skewed to favor one outcome. It may be present in how the results are promulgated—companies decide how and when to report studies. When companies hold back evidence, the medical literature becomes just the tip of the trial iceberg—a handful of trials, selectively drawn from a much larger pool. Each year we discover new ways that bias has been introduced, new ways that we have been fooled—reasons why trials we believed were accurate were, in fact, not.
There are three key ways doctors are actively misled into practicing medicine that is at high risk of reversal. First, the pharmaceutical and medical-device industry can manipulate studies. Second, treatment guidelines, often considered helpful and unbiased by physicians, can endorse treatments that are not evidence-based. And, third, the approval process for medical therapies often sets the bar too low.
INDUSTRY-SPONSORED TRIALS
Just days after the story broke, the Internet coined a term for the whole affair: “Scamiflu.” In the spring of 2014, a meta-analysis that appeared in the British Medical Journal presented data that showed that oseltamivir (Tamiflu), a medication used widely to treat the flu (influenza), provides very little benefit—even less than was previously thought. This drug, which had been on the market for more than a decade, had been studied in the past, but unlike previous studies, this meta-analysis was performed by independent researchers who had unrestricted access to full study reports of every oseltamivir trial (published and unpublished). Although Tamiflu had been thought to prevent transmission of the flu virus, decrease hospitalizations, and save lives, the study found it did no such thing. Tamiflu decreased flu symptoms by less than a day, from an average of seven days to just over six. It did this while causing nausea and vomiting. The drug did nothing to prevent transmission of the virus or reduce hospitalization. Finally, there was no evidence that it decreased deaths.
These findings were particularly unfortunate because countries around the world had stockpiled Tamiflu for years in preparation for a potential epidemic. Between the United States and the United Kingdom, more than $2 billion was spent to amass Tamiflu stockpiles. Government stockpiles were justified because officials believed that the drug could be used to slow the spread of a future influenza epidemic and save lives. It makes no sense to stockpile a drug (at tremendous cost) that is no better than Tylenol. The case of Tamiflu is another example of medical reversal, a therapy adopted into widespread use that is later found to be no better than our previous less expensive and safer therapy. It is a particularly costly, visible, and painful example that deserves deeper analysis because it illustrates ways that industry-sponsored trials can mislead physicians.
Before we can delve into the details of the Tamiflu case, there are some basics you need to know about the flu. First, when people talk about “the flu,” most of us are really talking about influenza-like illnesses. You have probably experienced at least one of these—you develop fever, cough, aches, and pain. Among people with influenza-like illnesses, some are actually infected with a virus called influenza, while other patients have been infected with other viruses—rhinovirus, coronavirus, and others. These viruses can cause illnesses that mimic true influenza. There is a test to distinguish influenza from influenza-like illnesses; however, for practical reasons, doctors have traditionally not used this test very often. If the treatment is to go home, eat some soup, take some Tylenol, call in sick to work, and get some sleep—does it matter if you have rhinovirus or influenza?
Influenza can be a terrible infection. It accounts for more than 100,000 hospitalizations each year in the United States and more than 50,000 deaths. In the worst years, the numbers can be terrifying. The 1918 Spanish Flu pandemic killed between 20 and 100 million people. Influenza is especially dangerous for people at the extremes of age and for women who are pregnant. Influenza can kill by filling the lungs with secretions, by damaging the lung tissue, or by causing a massive inflammatory reaction. The patients with severe influenza can also develop secondary infections, most dangerously, bacterial pneumonia.
With these facts in mind, let us turn to Tamiflu. When doctors and policymakers set out to judge this new pill to fight “the flu,” they studied patients with influenza-like illness. The appropriate end points they needed to study were hospitalization, pneumonia, transmission of the virus, and death. Most of the trials considered in the 2014 meta-analysis by the British Medical Journal group studied these very outcomes in these very patients.
Roche, the company that manufactures Tamiflu, seemingly working to help us make our point that companies can successfully introduce bias into studies, published its own meta-analysis just one month before the BMJ article. The Roche-sponsored study found that Tamiflu had a large benefit, reducing deaths among patients who were later proved to have influenza, especially if the
y took the drug early in the course of their illness.
Why were Roche’s results so different from those of the BMJ authors? Roche’s study combined data from observational studies, case-control studies, and randomized trials. It included only people who were proved to have influenza (rather than those with influenza-like illness) and focused on people who received the drug early. These are not unimportant data, but they do not pertain to how the drug is used in the real world.*
To know the real-world utility of Tamiflu, we need to know whether it helps all the people who were told to take it—not just those who later turn out to have true influenza. The question facing doctors is, By giving Tamiflu to all patients that you suspect have “the flu,” do you improve outcomes? Simply studying patients who are later confirmed to have influenza—and not all the patients you treat—is misleading. Not to mention that observational data are terribly unreliable for this question. Patients who did not receive Tamiflu (or did not receive it early) were likely very different from those who received the drug early. Even if you considered all the differences you could think of (patients’ other medical conditions, socioeconomic status, access to care . . . ) and adjusted for them, it is still unlikely that you would have controlled for all the factors that predict worse outcomes for the untreated people.
Ending Medical Reversal Page 15