Returning to gown-and-glove precautions, it is clear that these precautions should work. If the problem is that resistant bugs are being spread by clinging to doctors’ and nurses’ clothing, wearing a disposable cover should help. In before-and-after, single-center studies, gowns did decrease the rate of transmission. And yet, when the policy was tested in a randomized fashion, it did not work. The problem with systems interventions is twofold. First, when they do not work, we spend precious time, energy, and money to keep them going anyway. Second, the intervention adopted without evidence potentially distracts us from truly promising alternatives. In the gown-and-glove case, a different approach may actually decrease the spread of these bugs. Others have tested in a randomized trial whether the use of decontamination wipes decreases transmission of these resistant bugs, and it appears to work. It also appears to decrease hospital-acquired bloodstream infections. There are still questions, of course, but this strategy has already cleared a hurdle (the randomized trial) that gown-and-glove precautions avoided for years, before studies proved the technique ineffective.
Another example of a costly, widely implemented, but ultimately contradicted systems intervention is the rapid-response team, or RRT. Every day, in hospitals around the world, alerts go off that there has been a cardiac arrest. Unexpectedly, a patient who had been doing all right, in a standard hospital room, is now near death. In retrospect, when we look at the patient’s vital data in the hours leading up to the cardiac arrest, it often appears that we could have seen it coming. The person’s heart rate may have begun to climb hours before; her blood pressure may have dropped transiently two hours earlier; or her temperature might have been rising. You might assume that if we had only acted sooner, we could have prevented the cardiac arrest.
This belief led to the implementation of the RRT. The RRT is usually a group of well-trained nurses, occasionally in collaboration with physicians, who monitor vital signs remotely or are at the ready if called on by a patient’s primary doctors and nurses. The hope is that this team will identify patients who are heading for trouble, or they can be called on when the primary team first becomes concerned. The RRT then swoops into a patient’s room, makes an assessment, and intervenes before things get worse. The hope, of course, is that the presence of this team will decrease the number of cardiac arrests and improve survival.
The history of the RRT is similar to that of the gown-and-glove intervention. First, a serious problem was noted—in this case, too many seemingly preventable cardiac arrests. Second, a perfectly sensible intervention was suggested, the RRT. Third, single-center, before-and-after studies demonstrated that the RRT decreases cardiac arrests and improves survival. Soon, nearly every hospital in America had an RRT and it was considered a mark of quality.
Then in 2005, the MERIT trial was conducted at 23 hospitals in Australia. The investigators randomized hospitals to either adopt the RRT model or not. They found that although the RRT was well utilized, the number of cardiac arrests, unplanned ICU admissions, and unplanned deaths did not change. Interestingly, both groups had a reduction in cardiac arrests over time. The MERIT trial demonstrated both that RRTs do not work and that we are getting better at treating these patients. The MERIT trial also proved that a systems intervention could be an example of medical reversal.
NEITHER PROVED NOR DISPROVED, THE GRAY ZONE OF SYSTEMS
Gown-and-glove precautions and the RRT are systems interventions that became widely accepted, and good evidence now shows that they do not work.* Certainly, there have been systems interventions within health care that have improved patient outcomes. One recent example is an intervention that instituted careful discharge planning and home visits for a population of elderly patients leaving the hospital. This intervention decreased multiple outcomes, including rehospitalization. Besides interventions being proved effective or proved ineffective, a third outcome might be most common: the systems intervention is adopted but is neither really proved nor contradicted. The intervention is adopted based on weak data, it persists in practice for years, being increasingly questioned, but it is never conclusively rejected. Over time, equally weak evidence accumulates that the intervention is not achieving its intended goal or is having unintended consequences that outweigh its benefits. These are not medical reversals in the strongest sense, but they are examples of probably ineffective practices being continued for years, until they fall out of favor. There are two informative examples of this sort of practice.
Ten to fifteen years ago, observational studies suggested that among people who go to the emergency room with pneumonia, those who receive antibiotics within four to eight hours have better outcomes than those who receive antibiotics later. In an ideal world, the logical reaction to these data would be to design a randomized controlled trial. You would compare one group of hospitals, where doctors were incentivized to treat appropriate patients with antibiotics as soon as possible, with another group of hospitals that would provide the current standard of care. Your outcomes would include measures of success in treating pneumonia (mortality, length of stay in the hospital) as well as potential adverse effects of the intervention. We do not live in this ideal world, of course. Instead, what actually happened was that without real evidence, the four-hour goal was widely implemented, became a national quality measure, and served as the basis for pilot programs that paid hospitals more based on how well they met this mark. Soon after, people began to recognize unintended consequences. Patients who did not have pneumonia were given antibiotics, and doctors felt pressured to act before there was reasonable diagnostic certainty. Guidelines regarding this “four-hour rule” have been relaxed, but we still have no idea whether any such rule is helpful.
The second example of an unproven systems intervention coming into question is the use of “door-to-balloon time” as an incentivized quality marker. From the late 1990s to the very early 2000s, observational studies showed that when patients come to the emergency room with a heart attack, time matters. The specific time that mattered was the time from when the patient walked into the ER to the time when the blocked artery was opened by a balloon deployed by an interventional cardiologist. We call this time “reperfusion time.” Opening arteries in a timely fashion makes perfect sense, and it is associated with better outcomes. There is, however, an important difference between saying that patients who undergo timely reperfusion do well and saying we ought to create a national campaign to improve reperfusion times. By 2006, however, we were well into the latter—a massive initiative to bring the slowest hospitals up to grade.
5.1 Declining door-to-balloon times without associated declines in mortality.
Source: Menees DS, Peterson ED, Wang Y, et al., Door-to-balloon time and mortality among patients undergoing primary PCI. N Engl J Med. 2013;369:901–909. Copyright © 2013, Massachusetts Medical Society. Used with permission from the Massachusetts Medical Society.
In 2013 a study was published that raised doubts about our efforts. The study found that between 2005 and 2009, door-to-balloon times were reduced across the nation; the number of patients waiting longer than 90 minutes decreased from 40.3 percent to 16.9 percent. Unfortunately, mortality was completely unchanged over this time period. Figure 5.1 displays in a striking way the decline in door-to-balloon time graphed against the unchanging mortality. Our tremendous and costly efforts improved door-to-balloon time but did not yield improvements in survival.
Why was the intervention to reduce door-to-balloon time ineffective? There are several possible, but only speculative, explanations. The first is that we were wrong all along that door-to-balloon time matters. Maybe a shorter door-to-balloon time is unimportant in and of itself but is a marker of better hospitals that do everything better. This is akin to noting that lawyers who wear Armani suits win more court cases. An intervention to clothe more lawyers in Armani would be unlikely to improve the performance of the better-suited attorneys. Another explanation is that door-to-balloon time is not all that matters. Logically, y
ou would think the time from symptom onset (which happens at home) to balloon is what matters and that (emergency room) door-to-balloon is just one part of this. By this logic, we are not targeting the right metric. We need people to respond faster to their symptoms, 911 systems to run more efficiently, and ambulances to move faster. Lastly, decreasing door-to-balloon time might be an example of diminishing returns. Going from 180 minutes to 120 minutes provides a large benefit, 120 to 100 may also make a difference, but going from 99 to 90 minutes does not matter all that much. A final possibility— one that proponents of the intervention favor—is that door-to-balloon time reductions did help, but, over time, sicker people underwent the procedure. While it looks like no net improvement, the reality is that everyone is improving, and people are benefiting who previously did not even receive the procedure. The truth is that we do not know which explanation is right. Only a randomized trial can tell us whether or not asking hospitals to reduce their door-to-balloon time saves lives. Instead, door-to-balloon-time efforts were widely implemented and remain untested. Of course, it must be acknowledged that the study we describe is not the final word on the subject, but it does highlight a not-uncommon outcome of systems interventions—whether proved or disproved, they cost society billions of dollars.
THE SCIENCE OF SYSTEMS
We end with a major and conclusive reversal of a systems intervention. In 2001 a randomized controlled trial found that using insulin to lower patients’ blood-sugar levels to normal levels could improve survival in a surgical intensive care unit. This intervention was more novel than it might seem, because one of the body’s responses to critical illness is actually to raise blood-sugar levels. This trial was not of the highest level. It was a single-center study and was unblinded (more about that in chapter 9). In 2006 the same center reproduced the results in patients in the medical intensive care units. Professional societies embraced the strict blood-sugar targets, and entire hospitals instituted quality interventions aimed at meeting and enforcing the strict standard.
Fortunately, this systems intervention was ultimately rigorously tested. In 2009 a multicenter (42-hospital) randomized trial of more than 6,000 patients tested the intensive blood-sugar-control strategy against a more permissive one. This study found that strict blood-sugar reduction actually increased deaths by 2.6 percentage points at 90 days. This dramatic result meant that for every 40 patients treated with the intensive strategy, a patient died.
The story of blood-sugar control in the ICU is a classic in the medical-reversal genre. It is a story of poorly designed studies, single-center trials, and surrogate end points. We include it here, in our discussion of systems interventions, because as clinicians who worked in hospitals during the years the intervention was embraced, we remember the efforts hospitals made to ensure the strict target was met. Nurses received special training, pharmacies were ready to keep insulin running, and it was on the daily checklist of ICU rounds. In the end, all this effort helped no one and hurt a few.
These days we hear statements like: “The 20th century was about the science of medicine; the 21st will be about the science of systems” and “The major problem in medicine is to take what we know and ensure it gets done right.” These catchy phrases might be true, but so far it seems that when it comes to systems interventions, we have failed to learn the lessons of reversals taught by medical and surgical interventions gone awry. Systems interventions are often adopted based on scant evidence, and when this happens, they are probably just as likely to be overturned as any other medical treatment—no matter how much sense they seem to make. Being on the losing end of a systems intervention is just as bad as being treated with a medication that is later shown to be ineffective or harmful.
6 FINDING FLAWED THERAPIES ON OUR OWN
THE PRECEDING CHAPTERS have been pretty hard on doctors. The one thing every practice we discussed had in common is that they were recommended by physicians or performed under their guidance. These examples show that, despite years of training and the best intentions, doctors often recommend treatments that do not work. We like to think of this review less as an opportunity to scold our profession and more as a reminder to be humble. Before we move on from our long list of reversals, let us leave doctors’ offices and hospitals and go into our homes. As it turns out, we do not need doctors to suggest flawed therapies; we do quite well finding them all by ourselves.
William Osler, easily the most quoted physician of the 20th century, wrote, “The desire to take medicine is perhaps the greatest feature which distinguishes man from animals.” By this measure, we are becoming progressively more human. Americans use more self-prescribed treatments every year. These treatments come in many forms. There are vitamins and minerals, dietary supplements, and procedures of all kinds—acupuncture, chiropractic manipulation, and intravenous vitamin therapy, to name just a few. Many of these interventions either have no evidence supporting their benefit (and are thus potentially open to reversal) or have already been shown to be ineffective. For clarity, throughout this chapter we will refer to these self-prescribed remedies as complementary and refer to those directly sanctioned and prescribed by doctors as traditional.
6.1 Percentage of adults using dietary supplements. Source: Centers for Disease Control and Prevention, www.cdc.gov/nchs/data/databriefs/db61.htm.
We are all particularly susceptible to these sorts of complementary treatments. A recent survey by the Centers for Disease Control and Prevention revealed that more than half of all Americans use some sort of dietary supplement. The survey results are displayed graphically in figure 6.1. We admit that a search through our own medicine cabinets revealed two or three of these products (or at least did before we researched this chapter). What is so important about the reversals that involve complementary treatments is that they potentially affect a huge number of people. To be affected by the reversals that we dwelled on in the past few chapters, you had to have a disease and then see a doctor who chose to prescribe an unsubstantiated treatment. Even the screening tests we discussed only affect people for whom the test is recommended, and then only if they seek it out. By definition, the complementary therapies we discuss below are available to anyone and everyone.
To reflect the ubiquity of these treatments, instead of using one of our patients as an example, we will use a friend. Joanne happily volunteered the details of her case, though she did ask us to change her name.
Joanne knows that some of what she does probably does not make much of a difference. She eats low-fat cottage cheese and yogurt even though she has heard that reduced-fat dairy products may be no better for you than the luxurious full-fat versions. She uses low-fat salad dressing and almond milk. At this point she cannot even remember why, but she figures they help keep her weight down. She makes sure to have a glass of red wine and a small square of dark chocolate each night—she says these are good for her heart, but she also smiles when she says this. She eats only brown rice. She briefly followed a low-carb and then a gluten-free diet, but those interventions were too painful to last long. She is pretty sure that she has heard about data, at one time or another, supporting all these dietary quirks, but she is also pretty sure that she has heard data suggesting the opposite.
Joanne is more confident about some of her other health habits. She takes a multivitamin, specially balanced for women, every day. She also takes calcium and vitamin D, daily. She is 57, after all, and although she has no reason to think she has osteoporosis, a little extra boost for her bones cannot hurt. She also takes glucosamine because her knees feel a bit creaky these days and (with prompting) admits to having once visited an acupuncture practitioner for the same pain. When she is sick, she uses echinacea to speed her recovery.
Like most Americans, Joanne has chosen a range of practices in pursuit of health. Like many of us, she admits to being a little confused. She has heard various practices supported at one time or by one news source while the same practice is dismissed as ineffective, or even harmful, at another
time or by another source. One day fat is the enemy, the next it is sugar. In January you are supposed to do aerobic exercise; in February weight-training is in vogue. Studies extol the virtues of blackberries one week and peaches the next.
The story of how products and procedures become popular complementary health practices is interesting and a nice prelude to future chapters in which we delve into the causes of reversals in medicine. But before we get into that story, let us specifically consider some of Joanne’s chosen interventions and why they should be considered reversals.
To begin, we will limit the discussion to the “interventions” Joanne has chosen and steer away from her diet. Joanne says that these interventions are the things in which she is most confident. She began using a multivitamin after learning that the body needs a small quantity of a wide range of vitamins and minerals to thrive. She believes these are important because she thinks she does not “eat right.” She began the calcium and vitamin D after menopause to improve the strength of her bones. Her mother had a knee replacement, so the glucosamine seemed like a no-brainer as a preventive therapy. She tried acupuncture in an effort to stay away from pharmaceutical painkillers. The echinacea just works for colds. Period.
Ending Medical Reversal Page 8