Black Box Thinking

by Matthew Syed

  Soon Martin started receiving e-mails from practicing doctors. The messages were from clinicians not just in the UK but from the United States, Asia, and the rest of the world. One doctor wrote: “. . . for the first time in my career, I was recently faced with an unexpected ‘can’t ventilate, can’t intubate’ situation. Despite the horror . . . we made the early decision to perform a surgical tracheotomy and the patient recovered with no neurological deficit of any kind.”

  A doctor in Texas wrote:

  After a 5 hour case today, my patient was turned supine . . . Because of the information I learned relating to your wife’s case I pursued a surgical airway. An emergency tracheotomy was completed . . . The patient was transferred to ICU and when sedation was discontinued he woke up and responded appropriately. The good outcome in this case is directly related to the information you are sharing with medical professionals. I wanted to thank you.

  Another wrote: “Were it not for the work that you have tirelessly done to improve training in my profession, I do not think that this patient would have had such a successful outcome [the doctor had just performed an emergency tracheotomy]. I am greatly indebted to you.”

  The final report into the death of Elaine Bromiley can be found via a simple search on Google.36 It contains eighteen pages of detailed medical information. For all the technical language, however, the report can be seen, above all, as a heartfelt tribute to a beloved wife and mother.

  At the bottom of the opening page, Martin, one of the most inspirational individuals I have ever interviewed, added a single, italicized sentence.

  So that others may learn, and even more may live.

  Part II

  COGNITIVE DISSONANCE

  Chapter 4

  Wrongful Convictions

  I

  On August 17, 1992, Holly Staker, an eleven-year-old girl living in Waukegan, a small town in Illinois, took the short walk from her home to the apartment of Dawn Engelbrecht, a neighbor. She was babysitting Dawn’s two young children, a daughter, aged two, and a son, aged five.*

  Dawn had met Holly’s mother, Nancy, at the bar where she worked just a few blocks away. Little Holly often babysat when Dawn, who was recently divorced, was working at the bar in the evenings. The two families had become good friends.

  Holly arrived at the two-story apartment building on a tree-lined road named Hickory Street as agreed, at 4 p.m. It was a fine day and Dawn greeted her warmly. A few minutes later, Dawn said good-bye to her children and Holly, and left for work. She had a long shift ahead.

  By 8 p.m. Holly was dead. An unidentified intruder broke into the apartment, locked the door, and then violently raped Holly, stabbing her twenty-seven times in a frenzied assault. The corpse of the youngster was almost unrecognizable.

  At just after 8 p.m. a neighbor went to the bar where Dawn worked to say that he had seen her son, who had been locked out of the apartment and couldn’t get back in. Dawn called the apartment, but there was no answer. She then called Holly’s mother.

  They met at the apartment, and Dawn unlocked the door. They saw that Dawn’s two-year-old daughter seemed to be alone, and immediately called the police. Officers found Holly’s bloodied corpse behind a bedroom door.

  The local community descended into panic. The local police force pursued 600 leads and interviewed 200 people, but within a few weeks the trail had run cold. Parents were paranoid about letting their children out. Journalists described the community as “traumatized.”

  Then, through the testimony of a jailhouse informant, police happened upon a new suspect: Juan Rivera, a nineteen-year-old who lived a few miles south of the murder scene. Over four days, Rivera, who had a history of psychological problems, was subjected to a grueling examination by the Lake County Major Crimes Task Force. At one point it seemed to get too much for him. He was seen by officers pulling out a clump of hair and banging his head on the wall.

  On the third day, when the interview became accusatory, Rivera finally nodded his head when asked if he had committed the crime. By this time he was hog-tied (his hands were cuffed between his legs and his legs were shackled and linked to his handcuffs) and confined to a padded cell. Mental health staff at the jail determined that he had undergone a psychotic episode.

  On the basis of his confession police prepared a statement for Rivera to sign. But the confession was so inconsistent with what was known about the crime that police had to go back the next day to obtain a new confession, with the inconsistencies removed. The final interrogation lasted almost twenty-four hours. Rivera signed the new confession as well.

  At the trial, a few months later, the rewritten confession, which Rivera retracted hours after signing it, would form the central plank of the prosecution’s case. There were no witnesses. Although Rivera had a history of psychological problems, there was nothing in his past suggesting that he was capable of violence. There was no physical evidence linking him to the attack, despite a crime scene rich with human tissue. There was blood, hair, skin fragments, and many unidentified fingerprints, none of which matched Juan Rivera.

  But there was a brutally murdered young girl, a community still in mourning, and that signed confession.

  The jury didn’t take long to make up its mind. Rivera was convicted of first-degree murder and sentenced to life in prison. The court declined a request to set the death penalty.

  Many observers, including a number of local reporters, were uneasy at the verdict. They could see that the case hinged on the confession of a disturbed young man. But the police and prosecutors felt vindicated. It had been a troubling crime. A man had been convicted and sentenced. Holly’s family could try to find closure. The panic had finally abated. The community could rest easy.

  Or could it?

  II

  One of the key objectives of the criminal justice system is to ensure that people aren’t punished for crimes they didn’t commit. The idea of an innocent person serving time behind bars, deprived of his liberty by the state, offends deep sensibilities. As the English jurist William Blackstone put it: “It is better that ten guilty persons escape than that one innocent suffer.”1

  But miscarriages of justice have a quite different significance: they also represent precious learning opportunities. We saw in the last chapter that the aviation industry has made dramatic improvements by learning from failure. Investigators have examined data from accidents and reformed procedures. As a result, the number of crashes has fallen. This is the anatomy of progress: adapting systems in the light of feedback.

  There is a rather obvious trade-off between two of the key objectives of the justice system: convicting the guilty and acquitting the innocent. If you wanted to eliminate wrongful convictions altogether you could, say, increase the burden of proof required by the prosecution to 100 percent. But this outcome would come at a hefty price. It would mean that many more criminals would walk free. How could a jury ever convict, even if it were virtually sure of guilt, with the requirement for 100 percent certainty?

  What we are interested in, then, is reducing the number of wrongful convictions without compromising rightful convictions, and vice versa. This would represent a win-win. It would please liberals worried about miscarriages of justice as well as conservatives worried about too many guilty people walking free. The question is: how to make it happen?

  Think back to radiology, which we looked at in the last chapter. Here there are also two kinds of error. The first is when a doctor diagnoses a tumor that isn’t actually there. This is sometimes called a Type One error: an error of commission. The second kind is when a doctor fails to diagnose a tumor that is there. This is called a Type Two error: an error of omission. It is possible to reduce one kind of error while simultaneously increasing the other kind by altering the “evidence threshold,” as in the criminal justice system. But this trade-off should not obscure the fact that it is possible to reduce both kinds of error at t
he same time. That is what progress is ultimately about.

  Wrongful convictions are, in many ways, like plane crashes. If they can be established conclusively (a far from easy task, it has to be said), they hint at serious system failure. They offer an opportunity to probe what went wrong in everything from the police investigation, to the way the evidence was presented in court, to the deliberations of the jury, to the activities of the judge. By learning from failure we can design reforms that ensure that similar mistakes don’t happen again.

  But, as we have seen, people don’t like to admit to failure. How are the police going to feel when they are told that all their hard work to find a brutal killer has served only to put an innocent man in jail? How will prosecutors, who often make the decisive difference in court, feel when all those efforts have ruined the life of an innocent man? And how are judges and law officers going to react when they come face-to-face with evidence that the system they preside over has failed?

  In Part 1, we interrogated the concept of failure through the contrast between aviation and health care. We found that in health care, professionals are so fearful of their mistakes that they cover them up in various ways, making it impossible to learn from them. We also noted that this tendency characterizes the response to failure in many areas of our world.

  In this section, we are going to ask, why? We are going to drill down into the precise psychological mechanisms that underpin error denial, investigate the contours of its subtle evasions, and see how closed loops are perpetrated by smart, honest people. The criminal justice system will provide the lens, but we will also look at some of the most breathtaking failures in politics, economics, and business—and how progress has been thwarted again and again. We cannot learn if we close our eyes to inconvenient truths, but we will see that this is precisely what the human mind is wired to do, often in astonishing ways.

  It is not difficult to see why, in psychological terms, miscarriages of justice have been a sore topic for the legal system. The history is revealing. In 1932, Edwin Borchard, a law professor at Yale, compiled a list of wrongful convictions in his seminal book Convicting the Innocent and State Indemnity for Errors of Criminal Justice.2 Many of the cases were unequivocal failures. Eight involved people convicted of murder when the “victim” was missing, presumed dead, but who later turned out to be alive and well.

  These examples offered an opportunity to identify error traps, to probe systemic weaknesses. But many prosecutors, police, and judges (if not defense lawyers) drew very different conclusions. They were dismissive. Many regarded the very idea that the system was anything other than faultless as impertinent. As the district attorney of Worcester County put it: “Innocent men are never convicted. Don’t worry about it . . . It is a physical impossibility.”3

  It is difficult to conceive of a more exquisite example of closed-loop thinking. After all, if miscarriages of justice are impossible, why spend any time learning from them?

  “Historically, the legal system has been incredibly complacent,” Barry Scheck, a defense lawyer from New York, told me. “When people were convicted, people took it as confirmation that the system was working just fine. There was very little serious work done on testing the system. In fact, the idea that wrongful conviction was common seemed outlandish.”

  It is noteworthy that when a court of criminal appeal was first proposed in England and Wales in the early nineteenth century, the strongest opponents were judges. The court had a simple rationale: to provide an opportunity for redress. It was an institutional acknowledgment that mistakes were possible. The judges were against it, in large part, because they denied the premise. The creation of the court turned out to be “one of the longest and hardest fought campaigns in the history of law reform” requiring “thirty-one parliamentary bills over a sixty year period.”4

  Over the next few decades, remarkably little changed. Well-attested miscarriages of justice were dismissed as “one-offs” or as the price worth paying for a system that, on the whole, got decisions right. Scarcely anyone conducted systematic tests on police methods, court procedures, forensic techniques, or anything else. Why would they when the system is near to perfect?

  As Edwin Meese, attorney general of the United States under President Reagan, put it: “The thing is, you don’t have many suspects who are innocent of a crime. That’s contradictory. If a person is innocent of a crime, then he is not a suspect.”

  Then, on the morning of Monday, September 10, 1984, everything changed.

  It was at precisely 9:05 a.m. in a lab in Leicester, England, that Alec Jeffreys, a research scientist, had a eureka moment while looking at an X-ray film of a DNA experiment. He realized that by examining variations in the genetic code it was possible to discover a genetic fingerprint, a unique marker that could provide almost definitive identification. Together with later work by Kary Mullis, a scientist who would go on to win the Nobel Prize, it set the stage for a revolution in criminology.5

  Up until the work of Jeffreys, blood analysis represented pretty much the most sophisticated aspect of courtroom science. There are four blood groups, which means that tissue found at a crime scene could narrow down the list of suspects, but not by much. In the UK, around 48 percent of the population have blood group O.6

  DNA evidence is quite different. In the absence of contamination, and provided the test is administered correctly, the odds of two unrelated people having matching DNA is roughly one in a billion. The ramifications were huge—and it didn’t take long for the legal system to see them.

  In a narrow group of cases it would be possible to identify conclusively the DNA of tissue at a crime scene. In a rape case, for example, if the police swabbed the sperm found in the victim, they could narrow down the number of potential suspects to just one. This is why DNA fingerprinting has helped to secure hundreds of convictions—it has a unique power in establishing guilt.

  But DNA also has profound implications for cases that have already been tried: the power to exonerate. After all, if the DNA from the sperm in a rape victim has been stored, and if it does not match the DNA of the person serving time in prison, the conclusion is difficult to deny: it came from a different man, the real criminal.

  “DNA testing is to justice what the telescope is for the stars: not a lesson in biochemistry, not a display of the wonders of magnifying optical glass, but a way to see things as they really are,” Scheck has said. “It is a revelation machine.”7

  DNA tests are not completely fail-safe, since they can be corrupted by human error, fraud, mislabeling, or flawed interpretations when there are only tiny fragments of human tissue.8 But when they are undertaken honestly and systematically, they are pretty much definitive. By early 1989, the laboratory techniques pioneered by Jeffreys were ready to use in forensic labs. It set the stage for the most breathtaking experiment in legal history. And it didn’t take long for the results to come rolling in.

  On August 14, 1989, Gary Dotson, who had been convicted of rape in Chicago, was released from jail having consistently proclaimed his innocence. Underwear worn by the victim had been sent for DNA testing, which revealed that the semen belonged to a different man. Dotson had served more than ten years in jail.9

  A few months later, Bruce Nelson, who had been convicted of rape and murder in Pennsylvania, had his sentence overturned after DNA testing eliminated him as the source of the saliva found on a cigarette and on the victim’s breast, bra, and hair. He had served nine years. Then Leonard Callace, convicted of the sexual assault of an eighteen-year-old in New York State, was released when DNA testing excluded him as the perpetrator. He had served almost six years.

  The first DNA exoneration in the UK involved Michael Shirley, a young sailor who had been convicted of the rape and murder of Linda Cook, a barmaid working in Portsmouth, in 1986. A number of swabs had been taken from the victim and the original jury had been informed the blood group matched Shirley’s (along with 23
.3 percent of the British adult male population).

  Shirley mounted rooftop protests and engaged in hunger strikes. A journalist who campaigned for his release was fired by his newspaper. The Home Secretary refused to refer his case to the Court of Appeal. The police claimed that the swabs containing the semen had been destroyed, but under pressure discovered the relevant material. A simple DNA test revealed that the semen found in the victim did not belong to Shirley. He had served sixteen years at the time of his release.10

  By 2005 more than three hundred people had had their convictions overturned following DNA tests.11 In situations where evidence had been stored, clients of the Innocence Project (a nonprofit group that helps prisoners protesting their innocence) were exonerated in almost half the cases.

  These exonerations raised dozens of questions. Why were police pursuing the wrong suspects? Why were eyewitnesses misidentifying criminals? Why were interrogation techniques used by the police leading to false conclusions? Why were the courts failing? And what could be done about it?

  There was a wider question, too: What about the system more generally? DNA is relevant in only a small number of cases (rapes, murders, etc., where human tissue had been found and stored). What about all the other cases, where convicted criminals had no recourse to DNA fingerprinting to establish their innocence? How many innocent people were behind bars in total?

  Estimates are difficult to establish, but a study led by Samuel R. Gross, a professor at the University of Michigan Law School, concluded: “If we reviewed prison sentences with the same level of care that we devote to death sentences, there would have been over 28,500 non-death-row exonerations [in the United States] in the past 15 years rather than the 255 that have in fact occurred.”12

  This should not surprise us. Systems that do not engage with failure struggle to learn. “The emerging picture is clear enough,” Barry Scheck, the lawyer, has written. “The criminal justice system, from the police precinct to the Supreme Court, is a near shambles . . . A study by Columbia University reported that nationally two out of three death sentences imposed between 1973 and 1995 were constitutionally flawed and overturned by the courts.”13