All things considered, WHC performed well. But for Craig Feied (pronounced FEE-ed), an emergency-medicine specialist there, the incident confirmed his greatest fears. If the hospital nearly went haywire with just a few extra burn patients, what would happen during a major disaster, when the ER was most needed?
Even before September 11, Feied had spent thousands of hours thinking such grim thoughts. He was the chief architect of a federally funded pilot program called ER One, which was meant to drag the emergency room into the modern era.
Until the 1960s, hospitals simply weren’t designed to treat emergencies. “If you brought someone to a hospital at night,” Feied says, “the doors would be locked. You’d ring the bell, a nurse would come down to see what you wanted. She might let you in, then she’d call the doctor at home, and he might or might not come in.” Ambulances were often run by the local mortuary. It is hard to think of a better example of misaligned incentives: a funeral director who is put in charge of helping a patient not die!
Today, emergency medicine ranks as the seventh-largest physician specialty (out of thirty-eight), with a fivefold increase in practitioners since 1980. It is a master-of-all-trades endeavor, performed at lightning speed, and the emergency room has become the linchpin of public health. In a given year, there are roughly 115 million ER visits in the United States. Excluding pregnancies, 56 percent of all people admitted to U.S. hospitals come through the ER, up from 46 percent in 1993. And yet, Feied says, “you could drive a truck through the gaps in our protocols.”
September 11 brought home the point that emergency rooms are painfully limited in their surge capacity. If there had been a thousand victims at WHC, would they even have gotten inside?
Such a prospect makes Feied grimace. Most ERs have an ambulance bay that can fit only a few vehicles at a time. The docks are also built too high—“because the people who designed them were used to building loading docks,” Feied says. Rooftop helipads are similarly problematic because of the time and space constraints of a single elevator. Feied’s idea for getting rid of such bottlenecks is to design an ER more like an airport, with a large convex intake area that could accommodate a multitude of ambulances, buses, or even helicopters.
But these intake issues aren’t what worry Feied the most. A hospital that gets hit with something serious and communicable—SARS or anthrax or Ebola or a new strain of lethal influenza—would soon cripple itself. Like most buildings, hospitals recirculate their air, which means that one sick patient could infect hundreds. “You don’t want to go to the hospital for a broken ankle and get SARS,” Feied says.
The answer is to build hospitals, and especially ERs, with rooms designed for isolation and zero air recirculation. But most hospitals, Feied notes, don’t want to spend money on such unsexy, non-revenue-generating features. “There were some nice emergency departments built in 2001, state-of-the-art, and they’re completely obsolete today. They were built with open bays, divided by curtains, but if you have a SARS patient in Bed 4, there’s not a patient or doctor in the world who will want to go into Bed 5.”
And don’t even get Feied started on all the hospital patients who die from a cause other than what brought them to the hospital: wrong diagnoses (the result of carelessness, hubris, or cognitive bias); medication errors (based, far too often, on sloppy handwriting); technical complications (reading an X-ray backward, for instance); and bacterial infections (the deadliest and most pervasive problem).
“The state of current medical practice is so bad right now that there’s not very much worth protecting about the old ways of doing things,” Feied says. “Nobody in medicine wants to admit this but it’s the truth.”
Feied grew up in Berkeley, California, during the very raucous 1960s, and he fit right in. He skateboarded everywhere; he occasionally jammed on drums with a local band called the Grateful Dead. He had an aptitude for mechanics, taking apart and reassembling whatever looked interesting, and he was enterprising: by eighteen, he had founded a small technology company. He studied biophysics and mathematics before going into medicine. He became a doctor, he says, because of “the lure of secret knowledge,” a desire to understand the human body as well as he understood machines.
Still, you sense that machines remain his first love. He is a fervent early adopter—he put a fax machine in the ER and started riding a Segway when both were novelties—and he excitedly recalls hearing a lecture by the computer scientist Alan Kay more than thirty-five years ago on object-oriented programming. Kay’s idea—to encapsulate each chunk of code with logic that enabled it to interact with any other piece—was a miracle of streamlining, making programmers’ lives easier and helping turn computers into more robust and flexible tools.
Feied arrived at Washington Hospital Center in 1995, recruited by his longtime colleague Mark Smith to help fix its emergency department. (Smith was also a true believer in technology. He had a master’s degree in computer science from Stanford, where his thesis adviser was none other than Alan Kay.) Although some of WHC’s specialty departments were well regarded, the ER consistently ranked last in the D.C. area. It was crowded, slow, and disorganized; it ran through a new director every year or so, and the hospital’s own medical director called the ER “a pretty undesirable place.”
By this time, Feied and Smith had between them treated more than a hundred thousand patients in various emergency rooms. They found one commodity was always in short supply: information. A patient would come in—conscious or unconscious, cooperative or not, sober or high, with a limitless array of potential problems—and the doctor had to decide quickly how to treat him. But there were usually more questions than answers: Was the patient on medication? What was his medical history? Did a low blood count mean acute internal bleeding or just chronic anemia? And where was the CT scan that was supposedly done two hours ago?
“For years, I treated patients with no more information than the patients could tell me,” Feied says. “Any other information took too long, so you couldn’t factor it in. We often knew what information we needed, and even knew where it was, but it just wasn’t available in time. The critical piece of data might have been two hours away or two weeks away. In a busy emergency department, even two minutes away is too much. You can’t do that when you have forty patients and half of them are trying to die.”
The problem agitated Feied so badly that he turned himself into the world’s first emergency-medicine informaticist. (He made up the phrase, based on the European term for computer science.) He believed that the best way to improve clinical care in the ER was to improve the flow of information.
Even before taking over at WHC, Feied and Smith hired a bunch of medical students to follow doctors and nurses around the ER and pepper them with questions. Much like Sudhir Venkatesh hired trackers to interview Chicago street prostitutes, they wanted to gather reliable, real-time data that were otherwise hard to get. Here are some of the questions the students asked:
Since I last talked to you, what information did you need?
How long did it take to get it?
What was the source: Did you make a phone call? Use a reference book? Talk to a medical librarian?*
Did you get a satisfactory answer to your query?
Did you make a medical decision based on that answer?
How did that decision impact patient care?
What was the financial impact of that decision on the hospital?
The diagnosis was clear: the WHC emergency department had a severe case of “datapenia,” or low data counts. (Feied invented this word as well, stealing the suffix from “leucopenia,” or low white-blood-cell counts.) Doctors were spending about 60 percent of their time on “information management,” and only 15 percent on direct patient care. This was a sickening ratio. “Emergency medicine is a specialty defined not by an organ of the body or by an age group but by time,” says Mark Smith. “It’s about what you do in the first sixty minutes.”
Smith and Feied discovered more than
three hundred data sources in the hospital that didn’t talk to one another, including a mainframe system, handwritten notes, scanned images, lab results, streaming video from cardiac angiograms, and an infection-control tracking system that lived on one person’s computer on an Excel spreadsheet. “And if she went on vacation, God help you if you’re trying to track a TB outbreak,” says Feied.
To give the ER doctors and nurses what they really needed, a computer system had to be built from the ground up. It had to be encyclopedic (one missing piece of key data would defeat the purpose); it had to be muscular (a single MRI, for instance, ate up a massive amount of data capacity); and it had to be flexible (a system that couldn’t incorporate any data from any department in any hospital in the past, present, or future was useless).
It also had to be really, really fast. Not only because slowness kills in an ER but because, as Feied had learned from the scientific literature, a person using a computer experiences “cognitive drift” if more than one second elapses between clicking the mouse and seeing new data on the screen. If ten seconds pass, the person’s mind is somewhere else entirely. That’s how medical errors are made.
To build this fast, flexible, muscular, encyclopedic system, Feied and Smith turned to their old crush: object-oriented programming. They set to work using a new architecture that they called “data-centric” and “data-atomic.” Their system would deconstruct each piece of data from every department and store it in a way that allowed it to interact with any other single piece of data, or any other 1 billion pieces.
Alas, not everyone at WHC was enthusiastic. Institutions are by nature large and inflexible beasts with fiefdoms that must be protected and rules that must not be broken. Some departments considered their data proprietary and wouldn’t surrender it. The hospital’s strict purchasing codes wouldn’t let Feied and Smith buy the computer equipment they needed. One top administrator “hated us,” Feied recalls, “and missed no opportunity to try to stonewall and prevent people from working with us. He used to go into the service-request system at night and delete our service requests.”
It probably didn’t help that Feied was such an odd duck—the contrarianism, the Segway, the original Miró prints on his office wall—or that, when challenged, he wouldn’t rest until he found a way to charm or, if need be, threaten his way to victory. Even the name he gave his new computer system seemed grandiose: Azyxxi (uh-ZICK-see), which he told people came from the Phoenician for “one who is capable of seeing far”—but which really, he admits with a laugh, “we just made up.”
In the end, Feied won—or, really, the data won. Azyxxi went live on a single desktop computer in the WHC emergency room. Feied put a sign on it: “Beta Test: Do Not Use.” (No one ever said he wasn’t clever.) Like so many Adams and Eves, doctors and nurses began to peck at the forbidden fruit and found it nothing short of miraculous. In a few seconds they could locate practically any information they needed. Within a week, the Azyxxi computer had a waiting line. And it wasn’t just ER docs: they came from all over the hospital to drink up the data. At first glance, it seemed like the product of genius. But no, says Feied. It was “a triumph of doggedness.”
Within a few years, the WHC emergency department went from worst to first in the Washington region. Even though Azyxxi quadrupled the amount of information that was actually being seen, doctors were spending 25 percent less time on “information management,” and more than twice as much time directly treating patients. The old ER wait time averaged eight hours; now, 60 percent of patients were in and out in less than two hours. Patient outcomes were better and doctors were happier (and less error-prone). Annual patient volume doubled, from 40,000 to 80,000, with only a 30 percent increase in staffing. Efficiencies abounded, and this was good for the hospital’s bottom line.
As Azyxxi’s benefits became clear, many other hospitals came calling. So did, eventually, Microsoft, which bought it, Craig Feied and all. Microsoft renamed it Amalga and, within the first year, installed the system in fourteen major hospitals, including Johns Hopkins, New York–Presbyterian, and the Mayo Clinic. Although it was developed in an ER, more than 90 percent of its use is currently in other hospital departments. As of this writing, Amalga covers roughly 10 million patients at 350 care sites; for those of you keeping score at home, that’s more than 150 terabytes of data.
It would have been enough if Amalga merely improved patient outcomes and made doctors more efficient. But such a massive accumulation of data creates other opportunities. It lets doctors seek out markers for diseases in patients who haven’t been diagnosed. It makes billing more efficient. It makes the dream of electronic medical records a straightforward reality. And, because it collects data in real time from all over the country, the system can serve as a Distant Early Warning Line for disease outbreaks or even bioterrorism.
It also allows other, non-medical people—people like us, for instance—to repurpose its data to answer other kinds of questions, such as: who are the best and worst doctors in the ER?
For a variety of reasons, measuring doctor skill is a tricky affair.
The first is selection bias: patients aren’t randomly assigned to doctors. Two cardiologists will have two sets of clientele who may differ on many dimensions. The better doctor’s patients may even have a higher death rate. Why? Perhaps the sicker patients seek out the best cardiologist, so even if he does a good job, his patients are more likely to die than the other doctor’s.
It can therefore be misleading to measure doctor skill solely by looking at patient outcomes. That is generally what doctor “report cards” do and, though the idea has obvious appeal, it can produce some undesirable consequences. A doctor who knows he is being graded on patient outcomes may “cream-skim,” turning down the high-risk patients who most need treatment so as to not tarnish his score. Indeed, studies have shown that hospital report cards have actually hurt patients precisely because of this kind of perverse physician incentive.
Measuring doctor skill is also tricky because the impact of a doctor’s decisions may not be detectable until long after the patient is treated. When a doctor reads a mammogram, for instance, she can’t be sure if there is breast cancer or not. She may find out weeks later, if a biopsy is ordered—or, if she missed a tumor that later kills the patient, she may never find out. Even when a doctor gets a diagnosis just right and forestalls a potentially serious problem, it’s hard to make sure the patient follows directions. Did he take the prescribed medication? Did he change his diet and exercise program as directed? Did he stop scarfing down entire bags of pork rinds?
The data culled by Craig Feied’s team from the WHC emergency room turn out to be just the thing to answer some questions about doctor skill. For starters, the data set is huge, recording some 620,000 visits by roughly 240,000 different patients over nearly eight years, and the more than 300 doctors who treated them.
It contains everything you might want to know about a given patient—anonymized, of course, for our analysis—from the moment she walks, rolls, or is carried through the ER door until the time she leaves the hospital, alive or otherwise. The data include demographic information; the patient’s complaint upon entering the ER; how long it took to see a doctor; how the patient was diagnosed and treated; whether the patient was admitted to the hospital, and the length of stay; whether the patient was later readmitted; the total cost of the treatment; and if or when the patient died. (Even if the patient died two years later outside the hospital, the death would still be included in our analysis as a result of cross-linking the hospital data with the Social Security Death Index.)
The data also show which doctor treated which patients, and we know a good bit about each doctor as well, including age, gender, medical school attended, hospital where residency was served, and years of experience.
When most people think of ERs, they envision a steady stream of gunshot wounds and accident victims. In reality, dramatic incidents like these represent a tiny fraction of ER traffic and, bec
ause WHC has a separate Level I trauma center, such cases are especially rare in our ER data. That said, the main emergency room has an extraordinary array of patient complaints, from the life-threatening to the entirely imaginary.
On average, about 160 patients showed up each day. The busiest day is Monday, and weekend days are the slowest. (This is a good clue that many ailments aren’t so serious that they can’t wait until the weekend’s activities are over.) The peak hour is 11:00 A.M., which is five times busier than the slowest hour, which is 5:00 A.M. Six of every ten patients are female; the average age is forty-seven.
The first thing a patient does upon arrival is tell the triage nurse what’s wrong. Some complaints are common: “shortness of breath,” “chest pains,” “dehydration,” “flulike symptoms.” Others are far less so: “fish bone stuck in throat,” “hit over the head with book,” and a variety of bites, including a good number of dog bites (about 300) and insect or spider bites (200). Interestingly, there are more human bites (65) than rat bites and cat bites combined (30), including 1 instance of being “bitten by client at work.” (Alas, the intake form didn’t reveal the nature of this patient’s job.)
The vast majority of patients who come to the ER leave alive. Only 1 of every 250 patients dies within a week; 1 percent die within a month, and about 5 percent die within a year. But knowing whether a condition is life-threatening or not isn’t always obvious (especially to the patients themselves). Imagine you’re an ER doc with eight patients in the waiting room, one each with one of the following eight common complaints. Four of these conditions have a relatively high death rate while the other four are low. Can you tell which ones are which?
SuperFreakonomics Page 8