Randomistas

by Andrew Leigh

  One of the leading academic investigators on the Minneapolis Domestic Violence Experiment was Lawrence Sherman, then in his early thirties. The child of a Baptist minister and a YMCA worker, Sherman says, ‘I was brought up with a strong sense of social justice.’24 He finished his undergraduate degree in Ohio in half the usual time – two years rather than four – then found himself in the late 1960s in front of the draft board, as a conscientious objector to the Vietnam War. While others in his position had fled to Canada or been jailed, Sherman confidently told the draft board that he could avoid military service by taking on any government job, and he planned to undertake a research fellowship with the New York Police Department. ‘The draft board wasn’t pleased,’ he recalled. ‘Then again, they weren’t so upset when the police headquarters was bombed the weekend before I started work, and they realised I was just as likely to be killed in New York as in Vietnam.’25

  Through his work with police departments, Sherman became fascinated with understanding why some policing strategies worked while others were ineffective. His answer was to forge a new discipline: experimental criminology. This broke fresh ground not only for police forces, but also for the discipline of criminology itself. ‘For most of its history,’ Sherman observes, ‘criminology has been essentially a descriptive or observational science, sort of like astronomy.’26 Astronomers study the movement of celestial bodies; they do not try to change their trajectories. Sherman thought criminology should be more like medicine – striving to make sick people healthier, or, better still, intervening before they became sick in the first place. A core focus of criminology, he believed, ought to be figuring out how to cut crime. Experimental criminology was a significant departure from the thinking of people like Frenchman Émile Durkheim, who took the view that crime was inevitable because it was ‘bound up with the fundamental conditions of all social life’.27 Durkheim wanted to explain crime. Sherman wanted to reduce it.

  In Kansas City, the early 1990s saw a boom in crack cocaine. Dealers relied on ‘crack houses’ – abandoned homes that could be used to sell their product. The city’s police force wanted to find out the effect of shutting down crack houses. So they worked with Lawrence Sherman to design an experiment.28

  To begin with, the police needed to verify complaints that a home was being used to sell drugs. An undercover officer or informant went to the house and bought crack cocaine using sequentially marked bank notes. Then Sherman’s team came into the picture. In the police station were a series of numbered envelopes, each containing a note that said either ‘raid’ or ‘no action’, according to the results of a random number generator. If the note said ‘raid’, the police filed a search warrant request with a local judge. If it said ‘no action’, the evidence was put to one side.

  The raid was conducted by an eight-man tactical team from the Street Narcotics Unit. To be sure they had the right house, they sometimes sent in another undercover agent to buy drugs. A few years earlier, a raid on the wrong house had led to bad publicity for the unit, and they wanted to be sure they had the correct home.

  Immediately after the confirmation buy, the Street Narcotics Unit began the raid. An unmarked van drove up to the front of the house, its side door opened and an officer used a battering ram to break down the front door. Officers poured into the house, sometimes using flash-bang grenades to disorient those inside. Everyone was ordered to lie face-down and was handcuffed. Outside the house, police were waiting to catch anyone who jumped out a window or dashed through the back door. Nearby residents came out of their homes to see what was going on, sometimes applauding and cheering the officers. Everyone in the crack house was taken to the police station for questioning, and the home was searched for drugs. Sometimes new buyers would arrive while the house was being searched, then quickly flee when they realised what was going on.

  At the end of the eight-month experiment, Kansas police had randomly raided ninety-eight crack houses, with about the same number left alone as a control group. Looking at crime reports for the two sets of neighbourhoods, Sherman and his team saw that the raids had an immediate effect on crime. Two days after the raids, there were half as many reported offences in the city blocks that had been raided than in the control neighbourhoods. But the impact quickly evaporated. Within a fortnight, crime rates in the raided areas were no different from those in places for which the randomised envelope had read ‘no action’.

  But while raids failed to produce a sustained reduction in crime, experimental criminologists found other evidence that ‘hot spots’ policing – concentrating resources on the areas that produce the most calls for help – could cut crime.29 In Jersey City, police cracked down on a subset of the most dangerous street corners. Intersections randomly chosen for the crackdowns saw larger reductions in crime than the otherwise identical corners randomly chosen to be the controls.30 In Philadelphia, an experiment found that two- to three-kilometre foot patrols, short enough so that police repeatedly passed the same points, massively cut crime in dangerous neighbourhoods. Shifting from traditional approaches such as car patrols to hot spots foot patrols could avert four violent crimes a week.31

  Does this kind of approach simply shift the crime elsewhere? One way to answer the question is to look at the areas surrounding the hot spot and see if crime has increased in those streets. In both Jersey City and Philadelphia, researchers saw little evidence that the criminals had simply shifted their operations a few blocks away. Carried out appropriately, hot spots policing does seem to reduce overall crime.32 A majority of US police forces currently use the approach, with a fresh wave of studies now looking at the kinds of hot spots policing that work best.33 One of the latest findings is that ‘problem oriented policing’ – an approach in which police work with community leaders to address the underlying causes of crime – may be a more effective form of hot spots policing than strategies that simply focus on arrests.34

  Perhaps you’re not surprised by the results on hot spots policing. After all, some street corners in Philadelphia produce ten times as many crimes as others, so it makes intuitive sense that a ‘flood the zone’ approach might reduce offending rates. But high-quality evaluations of policing programs don’t always produce the expected result. Take, for example, Neighbourhood Watch, which promotes citizen involvement in crime prevention through encouraging incident reporting, marking property and conducting surveys on security. Anecdotal reports from Neighbourhood Watch organisers almost invariably report that the program has a substantial impact on reducing crime. Low-quality evaluations told a similarly positive story. But the randomised trials show zero impact. As experimental criminologist Lawrence Sherman puts it: ‘One of the most consistent findings in the literature is also the least well-known to policymakers and the public. The oldest and best-known community policing program, Neighbourhood Watch, is ineffective at preventing crime.’35

  *

  We’ve looked at prevention and policing. Now let’s look at punishment. As we have seen, not every form of punishment that sounds good actually does good. In the case of Scared Straight, the program was a tale worthy of an Academy Award. Yet when tested in rigorous evaluations, Scared Straight turned out to increase crime rates. A short experience of jail put youngsters at risk, and ended up making them more likely to commit offences in the future.

  When it comes to punishing crimes, most people’s minds turn to prison. But prison is only one way in which societies punish their wrongdoers. Over the centuries, punishments have included everything from the guillotine to the pillory, chain gangs to house arrest.

  But does the punishment fit the crime? When it comes to dealing with addicts, there’s a temptation to take a zero-tolerance approach. Psychological research suggests that drug users tend to evoke a visceral response of disgust, which can make it harder for people to support rational policies that reduce rates of drug abuse and drug-related crime.36 From Richard Nixon’s War on Drugs to Chinese labour camps for drug users, societies across the globe ha
ve often opted for a ‘tough on drugs’ policy.

  In our prisons today, many prisoners are locked up for crimes committed while using narcotics. New drugs – such as crack cocaine or crystal methamphetamine – often cause a spike in crime. As anyone who has tried to quit smoking knows, it isn’t easy to stop using an addictive drug. Yet the traditional court system is set up to punish crime, not help people kick the habit.

  In 1999 Australia was in the midst of a heroin epidemic. Use of the opioid had risen fourfold over the decade, and almost 150,000 people were shooting up regularly.37 New South Wales premier Bob Carr, who had lost his younger brother to a heroin overdose, announced that his state would trial a controversial alternative: the Drug Court. Drug Courts aim to treat addiction outside the jail system. Offenders who successfully complete the one-year program are typically put on a good behaviour bond.38 Failure generally means going to jail.

  The Drug court wasn’t the first in the world, but it was still radical for Australia.39 To address the critics, the state government decided they had to have strong evidence of its effectiveness. So they set up a trial. A group of eligible offenders – who had committed non-violent crimes and were willing to plead guilty – were randomly assigned either to the Drug Court or to the traditional judicial process. Those people were then matched to court records in order to compare reoffending rates over the next year or more.

  The study found that for every 100 offenders who went through the traditional court system, there were sixty-two drug crimes committed in the year after release.40 For those who went through the Drug Court, there were eight drug crimes committed in the subsequent year. It showed that even a citizen who didn’t care about the wellbeing of drug users should support the Drug Court, since it reduced crime at a cost about the same as that of the traditional system. Randomised evaluations of drug courts elsewhere have reached similar conclusions.41 Director of Public Prosecutions Nicholas Cowdery admitted that he had evolved from being a sceptic to a strong supporter: ‘the Drug Court of NSW is a success’.42

  Today, the NSW Drug Court remains an unconventional part of the judicial system. Its style is informal, and the focus is on whether offenders are managing to break their drug habit. Testing can occur multiple times a week. Admitting to drug use attracts one sanction. Getting caught lying about drug use attracts three sanctions. Fourteen sanctions and you go to jail.

  When participants move forward – by finding a job, staying off drugs or graduating from the program – the judge leads the courtroom in a round of applause. As senior judge Roger Dive notes, ‘You can see how much encouragement the participant gets from that applause. You can see how crestfallen are the ones who have fallen back and aren’t applauded. It’s all-important to them.’43 When one man, a former heavy user and dealer, graduated from the program to the applause of the courtroom, he wept. ‘You were the first people who gave me a chance.’44

  The Drug Court has also maintained its evidence-based philosophy. In 2010 the court conducted another randomised trial this time to see whether there was value in bringing Drug Court offenders before the judge twice a week rather than once a week. The evaluation found that more intense judicial scrutiny halved the chances of a positive drug test.45

  On the other side of the Pacific Ocean, another judge had been thinking about similar challenges. Appointed to the bench in 2001 after a stint as Hawaii’s US attorney, Steven Alm was immediately struck by the way in which his state punished probation violators. Probation officers would let slide the first ten to fifteen violations – such as failing a drug test – before recommending that a person be sent to prison. Those running the system were frustrated that most breaches went unpunished. Reasonably enough, offenders saw the system as arbitrary. When a breach landed them behind bars, they blamed the probation officer, the judge and an unfair system.

  Looking at the ad hoc enforcement of probation violations, Alm thought about how we treat children. When the rules are clear and promptly enforced, kids are less likely to be naughty than when the rules are arbitrary and unpredictable. ‘I thought about how I was raised and how I raise my kids. Tell ’em what the rules are and then if there’s misbehavior you give them a consequence immediately. That’s what good parenting is all about.’46

  In the case of parolees, Alm devised a system known as Hawaii’s Opportunity Probation with Enforcement, or HOPE. When offenders fail a drug test or miss a probation appointment, a judicial hearing is held as soon as possible – typically within seventy-two hours. If the judge is satisfied that there has been a breach, the parolee is immediately jailed for a few days. The approach aims to be swift, certain and proportionate. As Alm puts it, it’s ‘Parenting 101’.

  Buoyed by promising results, an external team devised a randomised evaluation. The study found that those who were assigned to the HOPE program were less likely to miss probation appointments, spent half as much time in jail, and were half as likely to commit a crime in the following year.47

  Conversations with the parolees also revealed an intriguing aspect of the program: offenders preferred it. An overwhelming majority felt that it helped them stay off drugs and improved their family relationships. Many saw the court system as an ally in rehabilitation, rather than the enemy. In the early days of the program, a researcher spoke to a man who was serving a short jail spell for failing a drug test. The probationer told the interviewer: ‘Judge Alm, he’s tough, but he’s fair. You know where you stand.’48 Programs based on the HOPE model are now operating in more than thirty US states.49

  Fairness is at the heart of the HOPE program, but the irony is that its randomised evaluation was arbitrary. Like every randomised experiment, assignment to the treatment or control groups was purely based on chance. Like the Australian Drug Court trial, participants were given no choice about whether they would be part of the study. The principle of equal justice requires that the same offence receives the same punishment. Yet once they were in the experiment, otherwise identical offenders were randomly assigned to different punishments.

  Is this ethical? I believe so. While those designing new criminal justice programs anticipate they will produce better outcomes, they do not know for sure that being in the treatment group is better than being in the control group. In fact, while the treatment group did better in the HOPE and Drug Court experiments, the control group did better in the Scared Straight and Neighbourhood Watch experiments.

  One way of thinking about the ethical issue in randomisation is that it turns on what we know about a program’s effectiveness.50 Adam Gamoran, a sociologist at the University of Wisconsin–Madison, agrees that if you are confident that a program works, then it is unethical to conduct a randomised trial. But if you are ignorant about whether the program works, and a randomised trial is feasible, Gamoran argues that it is unethical not to conduct one.

  When it comes to randomised experiments on punishment, the results are frequently unpredictable. As we have already seen, the results of the Scared Straight experiments demonstrated that a short stint in jail makes a young person more likely to commit crimes, not less likely. Likewise, studies of militaristic ‘boot camps’ find that they reduce reoffending rates only when they are designed to teach inmates new skills, not simply make life as tough as possible.51

  *

  Finally, let’s look at prisons, the institutions that currently house over 2 million Americans. The rapid growth of the jail population has been most dramatic in the United States, but similar trends can be seen in other countries. Over the past few decades, the United Kingdom has seen a significant fall in violent crime rates and a marked rise in incarceration. The same is true in Australia, where the imprisonment rate is now the highest in the nation’s history.52

  While some have claimed that bigger jails have made our streets safer, most experts disagree. That’s because the rise in imprisonment has been driven largely by tougher sentencing practices – such as California’s ‘three strikes’ laws – which have been shown not to have muc
h of an impact on criminal behaviour.53 Mark Kleiman, a leading researcher on the HOPE evaluation, argues that most people about to commit a violent crime are living day by day, and that raising the maximum sentence from ten to twenty years isn’t likely to have much of a deterrent effect. In Hawaii, HOPE succeeded not because it was punitive, but because it was predictable. If you want to deter crime, focus on certainty, not severity.54

  Prison can potentially also reduce crime through the ‘incapacitation effect’ – a fancy way of saying that you can’t steal my television while you’re doing time. But because crime tends to be a young man’s game, very long sentences don’t do much to incapacitate people who would otherwise be menacing the population. Grey-haired inmates are becoming increasingly common. Reviewing the evidence, an expert US National Academy panel concluded: ‘Most studies estimate the crime-reducing effect of incarceration to be small.’55 Rather, mass imprisonment risks creating further disadvantage, as released offenders struggle to find legal work, and to integrate back into their families and communities.

  If only we had randomised evidence on the impact of prisons. Then again, it’s hard to imagine that any prison authority would agree to run an experiment to answer this question. Courts and parole boards aim to dispense equal justice, not rely on luck. To have enough statistical power would require thousands of prisoners. There would need to be big differences in the sentences of the two groups, based on nothing more than chance. The cries of unfairness would be deafening.

  Or so you might think. In 1970 the California parole board agreed to run just such an experiment. That year, 3000 prisoners who were coming up for release were divided into two groups. Using a random table of numbers, half of the prisoners had their sentence shortened by six months, while the rest served their regular term. After release, the authorities looked to see who reoffended. They found no difference between the two groups, suggesting that another six months behind bars didn’t reduce recidivism.56