The Silo Effect

by Gillian Tett

  The factor that was most important in determining the murder rate, by far, was the movement of gangs. Across the urban landscape of Chicago, an estimated seventy-five gangs operated in different neighborhoods, who were estimated to have almost 70,000 members.16 That made them very powerful. However, the territory controlled by particular gangs often shifted. One group might control a particular block for a few months, but then the boundaries would shift as the drug trade or other criminal activities developed. Traditionally, the police had never tried to monitor these geographical patterns in a systematic way. Insofar as the police working in different districts communicated with each other directly (which they usually did not), they just swapped stories over the patrol car radios about which gangs were operating where. If they ever needed to collaborate with colleagues in any formal way, this communication could take a long time. “Historically, the Chicago police was so siloed,” Weis observed, “you had towers of information going up and down [the bureaucracy]. If I needed help from [someone] I needed to go up my full chain of command with paper and then work down [another] chain of command with paper.”

  But Goldstein started to collect all the reports that he could find about gang movements and put them on a centralized database, using a technique known as geospatial and temporal reporting. “What this showed was that we were suffering a lot of gang conflicts moving from the 7th to the 8th District—you could see the migration of the gangs and this conflict,” Weis observed. “Brett color-coded it on the chart and it almost looked like amoeba interacting. You had gangs fighting for a particular area to sell narcotics. We could get the information that day, and then see the interaction zone getting bigger and bigger.” Then Goldstein plotted his amoeba chart of gang movements against the reports of murders. Unsurprisingly, this showed a high level of correlation. Indeed, the rhythm was so closely connected that if you factored in other items such as temperature swings, the chart seemed to have predictive powers. Or to put it another way, even if you knew nothing else about what was happening on the actual streets, just watching the pattern of gang movements on a computer screen and other on-the-ground reports about violence could give you a good clue about which streets and city blocks were likely to see the biggest wave of deaths in the coming days, or even hours. The maps, in other words, did not just predict in a general sense what might occur, but gave real-time, immediate signals. “While it’s fine to say ‘this is a bad block’ we wanted to start being able to say ‘this is going to be a bad block tonight,’ ” Goldstein said. Or as Chief of Staff Masters explained: “Rather than look at data which was seven days old and base deployments of police off that pattern, we wanted to look for patterns in the data itself [and make predictions]. If you look at weather, the weatherman doesn’t just look at what happened last week and say ‘Okay we will have you wear a raincoat on Tuesday because it rained last Tuesday.’ ”

  By the start of 2010, Weis was ready to launch an experiment. He told Goldstein to start running his “weather maps”—or murder maps—and, based on these forecasts, issue warnings to the police officers about where crime was likely to explode next. The idea was that if Goldstein knew where violence could erupt, the Mobile Strike Force units could scramble to respond with their teams, in tandem with the regular police. To make this system work, Goldstein had to be in touch with the ordinary police patrols, like his own teammates from the 11th district. So they introduced a system of twice-daily calls. However, Goldstein insisted that this communication was a two-way interaction. He needed to collect all the live data that he could about conditions on the ground. So each day he issued his forecasts and called the patrol officers to pepper them with questions. Were particular gangs fighting over drugs? Was there an incident over a girlfriend? He pored over arrest logs. Then he plugged that data into his algorithms to predict where violence could erupt next. Until then this information had been scattered between different districts. But Goldstein was keen to centralize the information flows and remove the bureaucratic splits that had plagued the force in the past. So he passed his information out to the regular patrol officers and the Mobile Strike Units. “Brett would roll the numbers,” Weis explained. “Then he would [contact the police officers and] say things like: ‘Hey, be very careful, because in this area [the gangs] are fighting and shooting, they are trying to reestablish their turf.’ ”

  When the police in the patrol cars heard about the new system, many were suspicious. Within a couple of months, Weis had promoted Goldstein to the high rank of commander, an honor usually only given after decades of service on the streets. Goldstein tried to curb the jealousy by dressing in civilian clothes and using the title “director” not “commander.” But the resentment festered. Angry officers started to refer to his experiment in scathing terms as the “crystal ball unit.” “The jealousies were amazing,” Weis recounted. “You have got folks who have been three, four, five generations of policing and their grandfathers never had computers. So they don’t like the idea of change at all. Some guys embraced it. But a lot of them looked at it and thought this is total BS.” Weis and Goldstein tried to shrug the criticism off. After all, as Masters kept pointing out to them, when the idea of officers carrying police radios first cropped up in places such as Chicago in the 1960s, many policemen had loathed it, fearing it would be used against them as a surveillance tool. Within a couple of years, that resistance had melted away to the point where radios were considered so normal they were rarely discussed at all. “Whenever you are going to do something new in a large organization there is invariably going to be institutional resistance,” Weis explained. “This was the same [as with police radios]—it called for a change in the way that people were thinking.”

  By the end of 2010, Goldstein, Masters, and Weis were elated. Not only did their data maps give good signals about where murders were likely to occur in a general sense in the medium term—but they could even, on occasion, predict short-term developments. “There was one day we ran the target [areas for likely murders], and a minute after I sent the target list, I got a shooting notification. It was the weirdest thing, since it happened exactly in one of our targets,” Goldstein recalled. “Our machine had written up the targets [of likely places for a shooting] and someone was shot at 60 seconds later!”17 Better still, it seemed as if the murder rate was falling. At the start of 2011 the city announced the latest murder statistics and these showed that in 2010 the murder rate for the year was 5 percent below 2009, its lowest level since the 1960s. “Chicago’s murder rate hasn’t been that low since Lyndon B. Johnson was president,” two local community workers declared with delight.18 In early 2011 it fell even faster. Traditionally in Chicago, the most intense bout of killing occurs in the summer months, when gangs are on the streets. But in the summer of 2011, the murder rate tumbled to its lowest level since the 1960s. Indeed, when Goldstein extrapolated the statistical trends forward, it looked as if Chicago would record fewer than 400 homicides in the entire year. “It’s sad that we would view that [400 figure] as a sign of being successful, but it was a milestone,” Weis observed.

  Weis and Goldstein did not know how much of that shift was down to the use of their murder maps. But the anecdotal evidence seemed to be overwhelming. By sending the special units to the places identified as locations where homicides were about to occur, it seemed that the police were actually preventing some of the killing. “The point where I knew that we had been successful was that I got a call from the SAC [Special Agent in Charge] of the FBI,” Weis recalled, speaking with the type of police jargon that is incomprehensible to anyone outside the police tribe. “This said: ‘We are picking up on our Title-3s [the listening wires the FBI uses] stuff where the bad guys are saying ‘there’s this whole new unit in town. They don’t play, they are serious, they are real police, stay away from the 44 hundred units [the Mobile Strike Force]’! I took that as a real compliment. It told me that [our program] was successful. We had the right officers and Brett had them in the right locatio
n.”

  ON JUNE 21, 2014, Brett Goldstein turned forty. It was a symbolic moment for him. Back on the day when the World Trade Center had collapsed, and Goldstein had decided to break out from his cozy world as a corporate employee, he had tried to imagine what his life would be like when he turned forty. At the time, it seemed almost unimaginably distant. “I was twenty-something at the time and I just thought I don’t want to turn forty and just be a person who chased money,” he explained. But now that the moment was upon him, he wondered, had he achieved what he had hoped? The answer was mixed. By 2014, Goldstein had long since left his police project.

  In 2011, Rahm Emanuel became mayor of Chicago and soon after Jody Weis announced he would step down. The move left Goldstein and other reformers dismayed.19 By then Weis had become deeply unpopular in the Chicago police and morale was low. Many police had never forgiven Weis for being an outsider who pioneered radical ideas. The final straw came when Weis became caught up in a long-running scandal over a former Chicago police commander known as Jon Burge, who had been accused of letting his police officers torture suspects to get criminal confessions back in the 1970s and 1980s. Burge left the force in the 1990s, amid controversy, but was subsequently sentenced to four years of prison. Normally that would imply he would lose his pension. But in early 2011 the police pension board voted, after an acrimonious debate, to let Burge keep his pension. Weis publicly criticized the decision, and police loyalists were incensed. In the eyes of many officers, Weis had betrayed the tribal allegiances of the Chicago police.20

  Shortly after Weis left, Emanuel asked Goldstein to move across from the police to City Hall, with a view to replicating what he had done with crime in the heart of the Chicago government as chief data officer, or CDO. Goldstein was wary. He had become worn down by the in-fighting and was thinking of joining another start-up in the private sector. Part of him hankered to return to his “T-shirt and jeans” life, as he liked to say. But he was flattered by Emanuel’s offer and intrigued, since no city had ever before created the position of CDO. So he jumped another boundary to City Hall, hoping to start more experiments. Like most American cities, Chicago’s government was sitting on a vast ream of data about its citizens. But—again, like most government entities—the data was held in numerous different silos. The same problems that plagued Michael Bloomberg’s City Hall were replicated in Chicago. So Goldstein started trying to combine all this information into a single database, by pulling in local volunteers from the Chicago start-up scene, where he had once worked during his days at OpenTable. He christened them the “nerd herd.” Their style of working was radically different from what City Hall was used to, or indeed any government bureaucracy. The nerd herd wore T-shirts. They ate donuts around their laptops. They drew diagrams on the office whiteboards. And when they ran out of space on the board, they scribbled pieces of computer code or mathematical equations with markers on the windows of City Hall. “The mayor would walk by and give me a look to say: ‘What the hell is going on here?’ ” Goldstein recalled. But slowly some projects emerged. After one “hackathon” (all-night brainstorming session) organized in conjunction with Google, a local web developer named Scott Robbin created an interactive map that could show the public what had happened to any car that had been towed by the traffic police. “What people want to know in that situation is ‘Where is it [my car]?’ ‘Where can I find it,’ right?” explained Danielle DuMerer, the City Hall official who oversaw the project. “When we first released it we were updating the dataset every twenty-four hours, but later we were able to start updating that dataset every fifteen minutes. Obviously if your car has been towed you want to know right now where it is, not the next day.” Then the nerd herd created an interactive map to show Chicago residents what was happening with street sweeping. Eventually, Goldstein decided to pull together all the different data series into one giant interactive map, to show residents what was happening in the city. It also showed government officials potential security threats.

  The new interactive platform, known as WindyGrid, went live during the NATO summit of 2012, which was held in Chicago. It was a nerve-racking moment, particularly since the city’s networks were repeatedly attacked by the hacker group Anonymous. But the system survived and took root. Then Goldstein and DuMerer looked for ways to use the interactive platform to create collaboration with other branches of government. Back in 2011, just before Goldstein left the Chicago Police, his former colleague Michael Masters moved to a new post running the Department of Homeland Security and Emergency Management in Cook Country, Illinois, a large region that includes Chicago city. In previous decades, coordination between the mayor’s office and country level government officials had been patchy, at best: the different bureaucracies often competed as much as they collaborated. However, Goldstein and Masters set about trying to break down these barriers by using the data maps to swap information. In the summer of 2013, for example, Chicago hosted a food vendor fair, with 45,000 people in attendance. Just before the fair started, Masters’s Homeland Security team learned that a very violent storm was about to hit. In previous decades, it would have been hard to organize a quick response, since the county government did not usually communicate well with city officials, let alone the federal weather services. But in this case, the data systems had become so coordinated that the evacuation took place unusually smoothly. “We’re breaking down these silos and recognizing that weather crosses our boundaries so information has to as well,” Masters explained. “Problems don’t stop at political borders. Nor do floodwaters or pandemic flu.”

  But though Goldstein was excited by what his group had done with WindyGrid, it was his police murder map that made him particularly proud. In truth, the revolution he had tried to start at the Chicago Police Department did not evolve as he had hoped. After Goldstein and Weis left the department in 2011, the predictive analytics program was partly wound down. To a certain extent, the project was a casualty of political infighting and budget cuts. But what made the experiment doubly vulnerable was an issue that has long haunted Chicago’s political scene: race. The people running the CPD were overwhelmingly white. But Goldstein’s charts tended to predict that murders would happen in African American or Latino districts. Goldstein and Weis vehemently denied that this pattern was driven by any racial agenda whatsoever; their map simply reflected the homicides that were actually occurring and used this data to issue statistical predictions of where murders might occur next. But in a city such as Chicago, race was a very sensitive issue, even—or especially—when viewed through the prism of numbers.

  Tragically, almost as soon as the program was dismantled, the murder rate rose. Weis was deeply disappointed. “At the end of August [just before the units were closed], there were forty-one homicides below the previous year for 2011. The end of the summer is normally the worst time of the year for homicide counts, but that year they were just crazy low. But in the last four months of the year, when they eliminated those specialized units and Brett’s program, they lost all that traction,” Weis later fumed. “It’s kind of confusing to me that the mayor would ask Brett to basically do the role that he did for the police department for the city of Chicago and yet somehow allow the police department to ignore it. It’s very sad.”

  But Goldstein himself tried to be philosophical. After all, he reasoned, what he had essentially done with his murder maps was plant a seed. It had not sprouted in the way he had hoped in Chicago. But in the years after he left the CPD he could see signs that this seed of an idea was growing in more fertile soil elsewhere. Soon after they left their posts, Goldstein and Weis began to receive requests from other police forces to explain how their experiment had worked. Then, ironically, just as the CPD was shuttering the program, other forces started to pursue similar experiments. Over on the West Coast, the Los Angeles Police Department created a predictive analytics capability similar to the one that Goldstein had built. The police in Memphis, Tennessee, did the same, and quickly became a lea
der in the field. Then when riots broke out in London in 2012, the police there started using the same types of techniques that Goldstein had pioneered in Chicago to respond to British gangs. By 2014, Goldstein and Weis were being asked to share their ideas with police departments all over the world.

  When Goldstein had decided back in 2001 to change his career he had dreamed of changing the world. But now, at the age of forty, he had come to realize that you do not need to start a revolution to make a difference. Just shifting the dial a few inches matters too. The experiment in Chicago might not have transformed the police, but it had showed what could be achieved if somebody was willing to take a gamble and jump out of their cozy mental box. “I am not looking to solve the big problems in life. I’m completely fine with solving lots of small problems. It’s small things that can make places better,” Goldstein said.