When Science Goes Wrong

by Simon Levay

  And considering that the police had two shots at finding a match (once with Josiah Sutton and once with Gregory Adams) the odds that one of the men would match the sample were twice as high – l in 7.5. Thus, even if one ignored the fact that Sutton was positively excluded as a suspect (with the caveats mentioned above), the match of his DNA to the vaginal sample could easily have been a mundane coincidence of the kind that a busy crime lab could expect to encounter every day.

  When Werner and Raziq received Thompson’s report, they aired Sutton’s case in a news special on KHOU-TV. A few days later, the Houston Police Department ordered a new test of the DNA evidence, this time from an independent laboratory named Identigene. The laboratory used a more recently developed test named Profiler Plus, which uses a special-purpose machine, combined with analytical software, to test nine different loci simultaneously. These loci are different from the ones tested by Christy Kim, and they are more informative because there are more possible alleles (7 to 14) per locus. The results proved that two men contributed to the vaginal sperm sample and that neither of them had the DNA profile of Josiah Sutton. Sutton was innocent.

  A few days later, on March 12, 2003, Sutton was released on bond. Now 21 years old, his life as a free man took an erratic course: he tried various jobs and even enrolled in college, but nothing seemed to stick. Meanwhile, the director of the Texas Innocence Network, David Dow, finally agreed to take on Sutton’s case, with the aim of getting the parole board to recommend to Governor Rick Perry that he should issue a pardon.

  Texas has two kinds of pardon. A ‘full pardon’ leaves the conviction standing: it simply terminates the sentence and restores the person’s civil rights, without saying anything about whether he actually committed the crime. A ‘pardon for innocence’ is much rarer: it annuls the original conviction and opens the door for the person to claim compensation for time spent in prison, at a rate of up to $25,000 per year. To obtain a pardon for innocence, the district attorney, the police chief and the judge must all send letters to the parole board recommending the pardon, and the letters much be accompanied by evidence that the person did not commit the crime for which he was convicted. In Sutton’s case, the district attorney, Chuck Rosenthal, seemed incapable of letting go of his belief that Sutton was one of the rapists, and he therefore petitioned for a full pardon rather than a pardon for innocence. When asked by journalists for his reasons, he mentioned the fact that the victim had positively identified Sutton as one of her attackers (though she had also positively identified Gregory Adams, who was unquestionably innocent). As for why Sutton’s DNA was not found in the evidence samples, Rosenthal suggested that he had used a condom or failed to ejaculate. This theory required the victim to have been raped by three men rather than the two she testified to. This kind of reasoning – adding extra, unknown perpetrators – has become a staple among prosecutors trying to rescue a case in the face of exculpatory DNA evidence. Peter Neufeld, co-founder of the New York Innocence Project, has famously dubbed it the ‘unindicted co-ejaculator hypothesis’.

  Dow’s efforts to obtain the pardon for innocence dragged on for months. Rosenthal initiated yet another independent round of testing of the DNA evidence on the Sutton case, and the results confirmed Identigene’s earlier report. Finally, in May 2004, Governor Rick Perry issued Sutton a pardon for innocence. Sutton petitioned for compensation and was declared as being entitled to $118,000. He received his first payment of $60,000 in 2004. He went through the money in six months by buying three cars, partying and giving money to his family, according to the Houston Chronicle. He told the newspaper that he planned to be more careful with the next instalment. ‘It was a learning experience, and I have grown up,’ he said.

  ‘After he was exonerated, I went down to testify before a grand jury hearing about the lab,’ said Thompson, ‘and I took him and his mother out to dinner. I was kind of relieved to meet him, because I was thinking he would be some thug, but he struck me as sort of a gentle soul. He’s into meditation; he’s very serious about religion. He seemed confused and rootless and he didn’t quite know what to do with himself.’

  While Sutton was seeking his pardon, the city of Houston turned its attention to the Crime Lab. In early 2003, the former police chief recommended that the lab’s director be fired, and he retired before that could happen. Nine employees of the lab were disciplined. Christy Kim and another analyst were given 14 day suspensions, but they appealed to the Civil Service Commission and the suspensions were overturned. Then, in December, the mayor fired Kim, but again the Commission reinstated her on the grounds that she had merely followed the laboratory’s standard practices. Eventually Kim resigned, and several other employees joined the director in retirement.

  In early 2005 the city of Houston commissioned a new and completely independent review of its Crime Lab by Michael Bromwich of the Fried Frank law firm. As US Inspector General during the Clinton administration, Bromwich had conducted an investigation of the FBI’s Crime Lab that led to the disciplining of several agents.

  His review of the Houston lab, published in several instalments in 2005 and 2006, was a damning indictment of the lab’s practices, not just in DNA work but in serology (blood group analysis) and other departments. Out of 67 DNA cases reviewed, Bromwich found that 40 per cent had deficiencies serious enough to raise doubt as to the reliability of the work, the validity of the results or the correctness of the analysts’ conclusions. Three of these were capital murder cases that had ended with the defendant being convicted and sentenced to death, though none had proceeded to actual execution.

  In numerous instances, the analysts seemed to have shaded their findings to support the prosecution, often by failing to report findings that contradicted the prosecution’s case. In one serology case, for example, a bloodstain found at the scene of a double murder failed to match either the victims or the accused – a finding that was clearly exculpatory – but the DNA lab chief, James Bolding, concealed the mismatch by marking the stain’s blood type as ‘inconclusive’, according to Bromwich’s report. Years later, a second person was accused in the case, and his blood did match the stain. According to Bromwich, Bolding did not retest the stain but simply altered the original report to show a positive match. Bromwich also wrote that Bolding was even untruthful about his own qualifications: in a sexual assault case he testified that he had a Ph.D. in biochemistry from Texas Southern University, but he later admitted to Bromwich that he did not have a Ph.D. degree at all.

  There were also DNA cases in which the analyst reported having run controls (such as reagent blanks) but the lab data showed that no controls had been run. Such cases of apparent ‘dry-labbing’ are said to be pervasive in crime labs around the United States.

  The analysts consistently misreported the statistics of their cases. Just as happened in the Sutton case, the analysts would cite the frequency of the accused’s DNA profile in the population (a very low but irrelevant number), but not the likelihood that the accused’s profile would be present in a mixed evidence sample purely by chance (a much higher number). ‘It is clear that the DNA analysts in the Crime Lab, including Mr. Bolding, did not fully understand the scientific basis of calculating frequency estimates,’ wrote Bromwich.

  Perhaps the overriding problem at the Houston Crime Lab was the lack of effective oversight. ‘We have found no semblance of an effective technical review program or quality assurance regime to detect and correct these problems,’ wrote Bromwich. ‘As a result, they continued unabated.’

  Bolding eventually ceased cooperating with the investigation, and Kim had refused to answer Bromwich’s questions from the beginning. Although this lack of co-operation may have been motivated by a fear of self-incrimination, no Crime Lab employees have in fact been indicted in connection with the investigations.

  Although many convictions have been brought into question by Bromwich’s review, so far only two persons have been released from prison. One was Sutton, and the other was a man who serve
d more than 17 years for rape and who, like Sutton, was exonerated by new tests.

  In the summer of 2006, after being shuttered for three years, the Houston Crime Lab reopened for business with official certification, a new director, new or retrained analysts, a quality-control programme and largely new facilities. A year later, Bromwich’s final report spoke approvingly of these changes, but also listed some persisting shortcomings in the techniques used to analyse and interpret DNA data.

  In fact, there are considerable differences of opinion about the nature of the underlying problems and how to solve them. According to Bill Thompson, the core problem is that police crime labs are instruments of the state, so their analysts will, consciously or unconsciously, favour the prosecution. The way to solve this problem, according to Thompson, is to close the police labs and turn all work over to independent agencies. In theory at least, these labs would be more likely to conduct scientifically rigorous tests and to present the results in an impartial manner.

  In addition, Thompson believes that all labs should be subject to periodic random testing in which known samples are provided for ‘blind’ analysis – that is to say, the laboratory would not know when they were being tested. Currently, if such quality-control tests are done at all, analysts are given advance notice. As a result, we may assume that they are put on their best behaviour. Thompson believes that blind testing could be used to derive some measure of a lab’s false-positive rate – the likelihood that an incriminating test result would be in error. Even if the false-positive rate were very low – say, one in a thousand – that would still make nonsense of the ‘one-in-a-billion’-type statistics that are commonly presented to juries in an effort to secure conviction.

  A committee of the US National Research Council has rejected these ideas. It would be impracticable and too expensive to conduct enough blind testing to obtain reliable estimates of false-positive rates, the committee believed. And if defence lawyers believed their case was affected by pro-prosecution bias among police-lab analysts, they should be encouraged to commission independent testing to counter it.

  That’s not realistic, says Thompson. The defence attorneys may not be familiar enough with DNA technology to perceive the need for a retest. There are often insufficient funds for independent testing. Or the police lab may have used up the entire evidence sample in their own testing – or they’ll say that they have.

  Most importantly, criminal defence lawyers are often reluctant to commission independent tests for fear of harming their clients. After all, defence lawyers generally assume that their clients are guilty – because they usually are. So they expect that a test conducted by an independent lab will confirm the findings of the police lab. For sure, they don’t have to mention in court that they had their own testing done or what the result was, but the prosecution often finds some devious way to bring up the subject, according to Thompson. ‘The prosecutor says to the expert, “Oh, I notice there are some missing pieces of this fabric sample, where did those go to?” The defence says, “Objection!” and the prosecutor says, “Your Honour, it’s important for continuity, to explain, et cetera…” The judge says, “Objection overruled,” and the expert’s answer is “Oh, I sent them to the defence lab for testing.”’ Once the jury knows that the defence had its own testing done but didn’t present the results, they immediately assume that the results were incriminating, so any attack on the police lab’s tests loses credibility. Thus, there are several real-life factors that reduce the defence’s ability or willingness to conduct independent tests.

  Thompson’s self-appointed mission is to educate defence lawyers about DNA testing so that they are in a better position to review tests conducted by police labs and spot the problems that call out for independent testing and for challenging the state’s experts. The lawyers and law students who staff the Innocence Projects are clearly hearing his message, because the existence of DNA evidence is no longer a bar to taking on a case the way it was when Josiah Sutton sought the Houston Innocence Network’s aid.

  Have errors in forensic science ever led to the execution of an innocent man? Given the sheer number of executions – more than a thousand since the death penalty was re-instated in the United States in 1976 – it seems likely that they have, but identifying a specific instance has proved difficult. One case that has drawn a lot of attention is that of Cameron Todd Willingham, executed in Texas in 2004 for setting a house fire that killed his three children. Willingham asserted his innocence before, during and after his trial, and he did so again in his final statement before execution. At his trial, investigator Manuel Vasquez reported finding scientific indicators of arson, such as the presence of crazed glass, but subsequently-published forensic guidelines have rejected them as mere superstition. Hot glass is easily crazed by contact with water from fire hoses, for example. ‘Each and everyone of the ‘indicators’ listed by Mr. Vasquez means absolutely nothing,’ reported a commission of nationally-respected arson investigators in 2006. Willingham certainly hasn’t been proven innocent, but the evidence for his guilt has largely evaporated.

  Josiah Sutton’s story isn’t over. When the independent lab did the DNA testing that ruled him out as a suspect, it was able to reconstruct a complete DNA profile for one of the actual rapists, as well as a partial profile for the other. The complete profile was used to search a database maintained by the Texas Department of Public Safety, but no match was found. Still, new DNA profiles are being entered into the database all the time, because in Texas all convicted criminals have to give a DNA sample. In 2005, a young black man named Donnie Lamon Young, who was serving time on a drug conviction, gave blood for testing. In May of 2006 the DPS found that Young’s DNA was a match to the complete profile from the Sutton case.

  The Houston Police Department was notified, and a new sample was taken from Young, who was by then out of prison. Again there was an exact match. In June, Young was arrested and charged with aggravated sexual assault in the case for which Sutton was wrongly convicted. He was held in the county jail after he failed to post a $150,000 bail bond. The victim was unable to pick Young out of an identification line-up, but in January 2007 he pleaded guilty and was sentenced to 10 years’ imprisonment. He also named his accomplice, a man who had died in prison.

  When asked by the Houston Chronicle for his reaction to Young’s arrest, Josiah Sutton expressed himself laconically. ‘Let’s just say, if he’s the one who did it, that I don’t think we would be two good people to put in a room together.’

  His mother was more philosophical. ‘My son had been pardoned,’ she said, ‘but it still weighed on my heart that no one had been arrested and that some people would not believe in Josiah’s innocence until someone was. Now, justice can be done for the victim, and we can really close the book and say he did not do it.’

  District Attorney Rosenthal said, ‘I still don’t know enough to know whether [the victim] was mistaken or not. I intend to look into it, but if Sutton is innocent, I will be the first to say he is.’

  To which Bill Thompson commented, ‘Chuck, you’re a little late.’*

  SPACE SCIENCE: Off Target

  ON SEPTEMBER 23, 1999, after a journey of 419 million miles, the Mars Climate Orbiter spacecraft made its final approach to the Red Planet. At one minute after two in the morning, a tired but excited group of engineers and scientists at NASA’s Jet Propulsion Laboratory (JPL) near Pasadena, California, broke into smiles and applause: a signal had arrived to indicate that the spacecraft’s main engine had begun firing. This event would reduce the spacecraft’s speed enough for it to be captured by Mars’ gravitational field and go into orbit. Shortly thereafter, as expected, signals from the spacecraft ceased as it passed into the radio shadow behind the planet. The JPL group waited impatiently for the spacecraft to re-emerge on the other side of the planet, an event that was predicted to occur 21 minutes later. But the silence stretched to 22, 23 and 24 minutes, and then to hours and days. In fact, no signal was ever received f
rom the spacecraft again. The Mars Climate Orbiter was lost, and the mission was a total failure.

  Losing a Mars mission was not exactly a new experience for NASA: three out of the 11 previous US missions had ended the same way. Just six years before the Mars Climate Orbiter mishap, the $800 million Mars Observer spacecraft was lost in rather similar circumstances: radio signals from the spacecraft mysteriously ceased during final approach.

  Still, the US Mars programme overall had a good track record, especially in comparison with the Soviet programme. Of the 18 Russian spacecraft sent to Mars before 1999, 15 had been total losses – often failing to reach space at all – and the remaining three were only partial successes. And of the successful earlier American missions, some had been extraordinarily complex. These included the Viking mission of 1975 – a fleet of two orbiters and two landers, all of which functioned as planned and for far longer than their nominal design lifetimes. By the time of the Mars Climate Orbiter launch, there was a real confidence that NASA and its industrial partners – Lockheed Martin Astronautics, in this case – knew how to get the job done. It may have been this very confidence that sank the mission.

  The root cause of the loss was a scientific blunder as old as science itself: the confusion of units. Such errors can be prevented. Yet, in a larger sense, the loss represented a failure in systems engineering – that is, a failure successfully to integrate thousands of individual technical contributions into a single cohesive whole: a product that would fulfil the objectives of the customer, the US government. In that sense, the Mars Climate Orbiter mishap represented what is probably the most common mode of failure in large and complex scientific enterprises, and one that is extremely difficult to eradicate.