Inside the Department of Agriculture, Friedberg’s math team found data on the size and shape of every one of America’s twenty-six million fields. Inside the Department of the Interior, they found data on the soil composition of those fields. “They said,‘No one has ever asked us for this,’” said Friedberg. That one database was so big that it couldn’t be transmitted over the internet. He’d had to pay the government agency to send it on hard drives, which he then sent to engineers at Amazon, who moved it all to the cloud. In each of the six years from 2007 to 2013, Friedberg’s company used forty times more data than the year before. “All this data, it would never have existed if not for the government infrastructure that collected it,” said Friedberg. “There’s no private institution that on their own would have collected it. And without it we couldn’t have made predictions. We would never have had a business without that data. But by the time we were done, we could really quantify the effects of weather on farming.”
In 2011 Friedberg decided to sell exclusively to farmers, and WeatherBill changed its name to The Climate Corporation. “We needed to feel a little less Silicon Valley and less whimsical,” said Friedberg. For the next few years he would spend half his time on the road, explaining himself to people whose first step was toward mistrust. “Farmers don’t believe anything,” he said. “There’s always been some bullshit product for farmers. And the people selling it are usually from out of town.”
He’d sit down in some barn or wood shop, pull out his iPad, and open up a map of whatever Corn Belt state he happened to be in. He’d let the farmer click on his field. Up popped the odds of various unpleasant weather events—a freeze, a drought, a hailstorm—and his crops’ sensitivity to them. He’d show the farmer how much money he would have made in each of the previous thirty years if he had bought weather insurance. Then David Friedberg, Silicon Valley kid, would teach the farmer about his own fields. He’d show the farmer exactly how much moisture the field contained at any given moment—above a certain level, the field would be damaged if worked on. He’d show him the rainfall and temperature every day—which you might think the farmer would know, but then the farmer might be managing twenty or thirty different fields, spread over several counties. He’d show the farmer the precise stage of growth of his crop, the best moments to fertilize, the optimum eight-day window to plant his seeds, and the ideal harvest date.
The fertilizer was a big deal to them. “The biggest expense farmers have is fertilizer,” said Friedberg. “They’ll spend a hundred bucks an acre on corn seed and two hundred bucks on fertilizer. And their net profit might be a hundred bucks an acre. If it rains right after you fertilize, the fertilizer washes away. So how do you decide when to plant and when to fertilize? I had guys come up to me after and say,‘You saved me four hundred grand last year.’”
Farming had always involved judgment calls that turned on the instincts of the farmer. The Climate Corporation had turned farming into decision science, and a matter of probabilities. The farmer was no longer playing roulette but blackjack. And David Friedberg was helping him to count the cards. “For a lot of these guys it was like,‘My mind is blown,’” Friedberg recalled. “They didn’t believe that the knowledge could be created. All the new technology they had ever seen in their lives was physical. New machines, new seeds, new kinds of fertilizer. All these had just been tools for the farmer to use. None of them had replaced the farmer.” No one ever asked Friedberg the question: If my knowledge is no longer useful, who needs me? But it was a good question. “There is stuff the farmer picks up on that we haven’t got data on yet,” he said. “For example, are there bugs in the field? But over time that’ll go to zero. Everything will be observed. Everything will be predicted.”
About a year after they started selling insurance to farmers, the people at the Climate Corporation noticed something funny was going on. The farmers buying their weather insurance were spending a lot of time playing with the software to which the insurance gave them access. “We found the farmers logging in just to see the data on their fields,” said Friedberg. To insure American farmland, he’d needed to understand the fields better than the farmers did themselves: now they knew it. “We thought we were in the insurance business, but we were actually in the knowledge business,” said Friedberg. “It went from being insurance to being recommendations for farmers.” That first year, in 2011, the Climate Corporation generated $60 million in sales, just from selling weather insurance to farmers. Three years later they were insuring 150 million acres of American farmland—the bulk of the Corn Belt—and teaching the farmers how to farm them more efficiently. Six years after venture capitalists valued David Friedberg’s new company at $6 million, Monsanto bought it for $1.1 billion.
And yet through the entire experience, David Friedberg had this growing sense of unease. “When you come from San Francisco and grew up in Silicon Valley, every measure is about progress,” he said. “The progress in society. The progress in the economy. The progress of technology. And you kind of get used to that. And you think that’s the norm in the way the world operates, because you see everything getting better. Then you get on a plane and if you land anywhere but a big city, it feels the same. It’s total stagnation. It’s ‘we’ve been farming the same six fields for the last seventy years.’ It’s getting married at nineteen or twenty. It’s the opposite of progression. Life is about keeping up. Life is about keeping everything the same.”
People in the places he’d traveled lived from paycheck to paycheck. They were exposed to risks in ways that he was not: the weather was just one of those risks. He began to notice other kinds of data—for instance, that 40 percent of Americans can’t cover an unexpected expense of a thousand bucks. The farmers usually weren’t so bad off, but their situation was inherently precarious and threatened by modernity. Farmers didn’t work on desktop computers, and so they’d largely skipped the initial internet revolution. But they had mobile phones, and in 2008, when the 3G networks went up in rural America, farmers finally got online. “The problem with the internet is that it shows everyone on earth what they’re missing,” said Friedberg. “And if you can’t get to it, you feel you are getting fucked. That there is this very visceral and obvious shift that is happening in the world that you’re missing out on.”
At the same time David Friedberg was helping farmers to secure their immediate economic future, he was threatening their identity. Your family has been tilling this same soil for a century, and yet this data-crunching machine I’ve built in just a few years can do it better. The phrase was a whisper underlying every conversation he’d had with a farmer.
Friedberg played in a high-stakes poker game with some friends in the tech world. In their last game before the 2016 presidential election, he offered to bet anyone who would take the other side that Donald Trump would win.
After Trump took office, DJ Patil watched with wonder as the data disappeared across the federal government. Both the Environmental Protection Agency and the Department of the Interior removed from their websites the links to climate change data. The USDA removed the inspection reports of businesses accused of animal abuse by the government. The new acting head of the Consumer Financial Protection Bureau, Mick Mulvaney, said he wanted to end public access to records of consumer complaints against financial institutions. Two weeks after Hurricane Maria, statistics that detailed access to drinking water and electricity in Puerto Rico were deleted from the FEMA website. In a piece for FiveThirtyEight, Clare Malone and Jeff Asher pointed out that the first annual crime report released by the FBI under Trump was missing nearly three-quarters of the data tables from the previous year. “Among the data missing from the 2016 report is information on arrests, the circumstances of homicides (such as the relationships between victims and perpetrators), and the only national estimate of annual gang murders,” they wrote. Trump said he wanted to focus on violent crime, and yet was removing the most powerful tool for understanding it.
And as for the country’s
first chief data scientist—well, the Trump administration did not show the slightest interest in him. “I basically knew that these guys weren’t going to listen to us,” said DJ, “so we created these exit memos. The memos showed that this stuff pays for itself a thousand times over.” He hoped the memos might give the incoming administration a sense of just how much was left to be discovered in the information the government had collected. There were questions crying out for answers: for instance, what was causing the boom in traffic fatalities? The Department of Transportation had giant pools of data waiting to be searched. One hundred Americans were dying every day in car crashes. The thirty-year trend of declining traffic deaths has reversed itself dramatically. “We don’t really know what’s going on,” said DJ. “Distracted driving? Heavier cars? Faster driving? More driving? Bike lanes?”
The knowledge to be discovered in government data might shift the odds in much of American life. You could study the vaccination data, for instance, and create heat maps for disease. “If you could randomly drop someone with measles somewhere in the United States, where would you have the biggest risk of an epidemic?” said DJ. “Where are epidemics waiting to happen? These questions, when you have access to data, you can do things. Everyone is focused on how data is a weapon. Actually, if we don’t have data, we’re screwed.”
His memos were never read, DJ suspects. At any rate, he’s never heard a peep about them. And he came to see there was nothing arbitrary or capricious about the Trump administration’s attitude toward public data. Under each act of data suppression usually lay a narrow commercial motive: a gun lobbyist, a coal company, a poultry company. “The NOAA webpage used to have a link to weather forecasts,” he said. “It was highly, highly popular. I saw it had been buried. And I asked: Now, why would they bury that?” Then he realized: the man Trump nominated to run NOAA thought that people who wanted a weather forecast should have to pay him for it. There was a rift in American life that was now coursing through American government. It wasn’t between Democrats and Republicans. It was between the people who were in it for the mission, and the people who were in it for the money.
The first time DJ Patil met Kathy Sullivan, he’d gone to talk to her about how she might better use data. He wound up learning from her how he might better approach his new mission. “She said something very insightful. She said working for the government, you need to imagine you are tied down, Gulliver-style. And if you want to even wiggle your big toe, first you need to ask permission. And that if you can imagine that and still imagine getting things done, you’ll get things done.”
The single most important source of data for the weather models are the satellites. The geostationary satellites hover over the equator, taking pictures of whatever is happening beneath them. The polar satellites circle the globe from North Pole to South Pole and gather data from the entire planet. They take soundings of the temperature and moisture in the atmosphere; measure vegetation coverage; monitor ozone levels; detect hot spots and so are able to report fires before people on the ground even know they have been lit; and feed weather forecasting models not just in the United States but in Europe and Asia. Without the information supplied by the polar satellites, weather forecasts everywhere would be worse. You’d be more likely to turn up at the airport and find that your flight had been canceled, or to be surprised by a wildfire, or to be hit without warning by a storm. “We ran the no-satellite experiment in Galveston in 1900,” says Tim Schmit, a career NOAA researcher who has spent the last twenty-two years creating new and better satellite images of Earth. “Ten thousand people died.”
Kathy Sullivan’s life after her astronaut career had been one ambitious science project after another. She’d spent the first three years as NOAA’s chief scientist. From there she’d gone on to run the Center of Science and Industry, a 320,000-square-foot museum and research center in Columbus, Ohio. After a decade of running that, she was hired in 2006 by Ohio State University to be the first director of their new science and math education center. When she returned to NOAA, in 2011, a polar satellite launched in the 1990s was approaching the end of its useful life. Its replacement was late, mired in political controversy, and facing cuts to a budget it had already exceeded. “She walks in the door and finds that the decisions made by a lot of other people are about to screw us all,” said DJ Patil. “Now it’s a question of national security. Because you won’t be able to see the storms.” A storm that went unseen, to DJ’s way of thinking, belonged in the same category as a terrorist who went undetected.
The Clinton administration had asked three different agencies—the Department of Defense, NASA, and NOAA—to manage the polar satellites. The collaboration hadn’t gone well. “The dynamic was a typical Washington sociopathic thing mixed up with a lack of leadership,” said a former NOAA official. “Three agencies is hard. Because when you’re busy or something annoys you, you can just assume or pretend that someone else will handle it. It’s also hard because nobody wants to be responsible when things go badly. It’s hard to control headlines and explain complicated things. Congress sends agencies very mixed signals, changes budgets, moves on to new things, speaks with many voices. Administrations and Congress don’t often agree or even know about all the things the agencies are working on. Everybody blames someone else, and whoever is better at the blame game usually comes out on top. And the Department of Defense always comes out on top because it has the most resources and protective reflexes and friends.”
The Obama administration had broken up the marriage between NASA and the Department of Defense and handed the entire mess to NOAA. But the NOAA to which Kathy Sullivan returned had drifted further in the direction it had been heading when she’d left. While the weather forecasts from inside it had gotten better and better, the political climate outside it had gotten worse and worse. Working at NOAA—or anyplace else in the federal government—could not be more different from working at NASA. When you were an astronaut, everyone loved you. When you told people that you worked for NASA they were usually curious, and even a bit informed. There was a reason for this, over and above the drama of the work: NASA had been encouraged, right from the start, to promote itself. “NASA was allowed to tell its story to the world,” Kathy said. “There was a conscious need to publicize, because it was meant to restore confidence. NASA had heroes.” NOAA didn’t have heroes or drama. Or, rather, it had drama, and people who had done genuinely heroic things, but the American public never heard about any of it. It had people like Tim Schmit, the satellite guy, whose work had saved thousands of American lives. “NOAA has a hidden utility problem,” said Kathy. “You cannot market NOAA. You really cannot market NOAA. Over the last several decades they not only don’t get marketed. They are routinely slandered.”
The relationship between the people and their government troubled her. The government was the mission of an entire society: why was the society undermining it? “I’m routinely appalled by how profoundly ignorant even highly educated people are when it comes to the structure and function of our government,” she said. “The sense of identity as Citizen has been replaced by Consumer. The idea that government should serve the citizens like a waiter or concierge, rather than in a‘collective good’ sense.”
Her first big task upon returning to NOAA was to fix the polar satellite, and she did it. “She’s unflappable with whiny politicians and lawyers,” says a former NOAA official, who watched Sullivan attack the problem. “She was good at saying,‘Stop bothering my people and let them do their job.’” She got a new polar satellite, launched in November 2017, back on schedule, but with a twist: she arranged it so that the problems that had bedeviled her predecessors would not trouble her successors. “Of the many incredibly stupid things that a person can do on this planet, one is to build and buy a single satellite, when you know you’ll need more of them,” she said. There was no reason that NOAA could not budget for, and begin to plan, the next two, three, or four satellites; there were even economies of scale for
some of the complicated parts. The problem was that no one in government liked to pay now if they could pay later. Nevertheless, she somehow persuaded the relevant parties in Congress and NOAA to make a deal for multiple satellites.
The ins and outs of how she’d done all this would have made for an excellent Harvard Business School case study—or a briefing memo for the new Trump administration. But that memo would never be read. The first Trump budget proposed removing the money in NOAA’s budget that she’d secured for future satellites. The Trump people would never call her, but if they had she would have offered them one simple piece of advice. “You need to figure out what you want your leadership team to be intentional about—because if they aren’t intentional about it, it won’t happen. There’s hundreds of things that will naturally happen. And then there are the things that won’t.” One of the things that wouldn’t happen is satellites getting built on time, within budget. Another was that Americans would die, if you didn’t work hard to figure out what was going on inside their heads.
That had been her next big project. A Weather-Ready Nation, she called it. The Joplin tornado had been the catalyst. It had various ambitions—making communities more responsive to the weather, making fishing stocks more resilient to the climate—but at its heart was the desire to better prepare Americans to face threats. Kathy had helped to install Louis Uccellini as head of the National Weather Service; he shared her passion for the problem. The meteorologists inside the Weather Service were bothered that people didn’t respond as expected to their warnings. But then they were weather geeks. Scientists. “I can’t trace exactly where or when or how the realization dawned [on us] that the jargon-laden bulletins were not comprehensible to users,” said Kathy. “Or that people didn’t respond to raw data; they respond to other human beings, trusted voices. Or that the punch line—what this storm may do to you—was often buried after many paragraphs of geeky weather details. Or that normal humans don’t understand probabilities and cannot translate a wind speed or rain rate into tangible worries about the roof coming off or being knee-deep in water. You don’t particularly care what the wind speed at five hundred millibars is. You want to know: What’s it going to do to my house?”
The Fifth Risk Page 14