Everyday Chaos

by David Weinberger

  Coda: The Kingdom of Accidents

  You’re driving on a residential street in Boston. It’s winter. The streets have been plowed, but there are occasional slippery patches where the scraped snow has melted and refrozen. So you’re driving as if you’re not from Boston: slowly and considerately.

  As you glide to a stop at an intersection, working the steering wheel to compensate for the car’s sideways slippage, you feel a bump on the back of your car: a canonical fender bender. You can’t get too angry. The bump was more like a gentle tap, so it’s likely that the car behind you was also going slowly and carefully. The roads are just impossible. What are you going to do? It was an accident.

  Classifying an event as an accident puts it into a kingdom with its own special rules. “It happened by accident” is meant to imply not that there wasn’t a cause but rather that we were unable to exert control over that cause. It thus wasn’t our fault. While any particular accident perhaps could have been prevented, not all can be; the persistence of the Kingdom of Accidents is a given.

  Accidents are the exception. They are unexpected, unasked for, and usually unwelcome, visitors to the Kingdom of the Normal. In the Normal, we make our plans and succeed at them with some well-calibrated degree of confidence. We normally get to work within ten minutes of the mandated start of our workday, but if our car gets bumped or a subway track loses power, we’ll feel we’ve been paid an unwanted visit from that other kingdom.

  These two worlds have fluid borders, unlike the line the ancient Greeks drew between the perfectly orderly heavens, where the sun rises and falls without mishap, and Earth, where chaos outlives every mortal. These days we assume we control what happens in this human world to a degree that is new in history—if we could sue tornadoes, we would—but we still need the Kingdom of Accidents to explain why things don’t always go as planned.

  What makes the Normal normal is not simply that it’s what is usually the case. The Normal feels as if it’s the real world, the way things are, even the way things should be. When we say about outrageous behavior, “That’s not normal,” we’re not just making a statistical observation. The Normal is our home. Accidents are home invaders.

  But if the Normal is our home, it’s like the perfect suburban neighborhood, with perfectly manicured lawns and perfectly coiffed wives who take care of the kids all day and have martinis waiting when their menfolk return from work. Later in the movie, we find out it’s too perfect to be real and too gendered to be desirable. The Normal is a fiction that simply ignores the galaxies of accidents that enable it.

  The Normal arises from a trick of focus. The Accidental looks like the exception to the Normal because we define the Normal around our plans … and we make plans only about that which we can control to some degree. Even gaining that imperfect degree of control has required us to build massive, interconnected systems to do something as simple as take a long drive. We have cars with controls and instrumentation to enable us to keep them apart from other cars. We have laws governing how we drive them and penalties for those who fail. We have motels spaced out along the highways and Airbnbs filling the gaps. We have GPS systems, credit card systems, and car-resale systems all designed to keep automobile trips predictable. The rare visitors from the Kingdom of Accidents look like strangers because we’ve worked so hard on keeping them out.

  Meanwhile, our cross-country trip runs on and through accidents beyond count: The opacity of the lane markings varies according to which cars wobbled over them and which trucks braked so hard they wore away the edge of the stripe your car just drove past. The grass untrimmed around the edge of a sign grew from seeds whose stories can’t be found. Each of the cars you pass is going somewhere for some reason that arose through the billions of accidents of biology, emotion, politics, and the pure luck of living in this time and place. The asphalt was poured by workers whose boots were splashed black in patterns no one could have outlined beforehand. A worker in those boots was surprised on some Tuesday by the extra bag of chips he found in his lunch bucket.

  The Normal is a poorly paved road running through the endless territory belonging to the Kingdom of Accidents. Our plans are low beams that point wherever we look and leave the rest in the dark.

  Chapter Two

  Inexplicable Models

  Until a chick is five or six weeks old, it’s surprisingly difficult to tell the pullets from cockerels—or the boys from the girls, if you’re not in the trade. This is a problem if you’re a commercial egg producer who sees males as nothing but a waste of chicken feed.

  In the 1920s, the Japanese hit on a technique to address this. There are two ways to determine the sex of a chick: check their wingtips or their cloacae—better known as their buttholes.1 Japanese chicken sexers settled on this second approach because it is more reliable for more breeds of chicken. (This decision explains the title of an NBC report in 2015: “Chick Sexer: The $60K a Year Job Nobody Wants.”) But the odd thing is there’s nothing about a male chick’s cloaca that obviously differentiates it from a female’s. At least there’s nothing that anyone can point to or describe. So a student learns how to tell the males from the females by guessing which is which, while an expert confirms or rejects the decision. After classifying thousands of chicks this way, eventually, somehow, the student starts getting it consistently right, but still can’t tell you how they know.

  Training people how to sex chickens therefore is not like training soldiers to identify enemy aircraft or teaching people how to become proficient bird watchers, as Richard Horsey points out in a philosophical article about chicken sexing.2 Planes and birds have distinguishing features you can learn to notice: a Japanese Zero has rounded wingtips; a robin has an orange chest. Chicken sexers have nothing they can point to. If you ask them how they knew a particular chick was male or female, they cannot tell you. Yet a trained chicken sexer is able to classify up to 1,200 chicks an hour—one every three seconds—with 98 percent accuracy.

  Philosophers who study knowledge find this fascinating. Because the accuracy of the chicken sexers’ predictions is so high—the world record is 1,682 chicks sexed in an hour with 100 percent accuracy—their decisions demonstrate they know the genders of the chicks.3 But since the ancient Greeks, we’ve considered knowledge to consist not just of true beliefs but of true beliefs that are justifiable.4 Saying, “I know it’s a pullet, but I can’t tell you how I know or why you should believe me,” is like saying, “I fully know that the next card is going to be an ace but I have no reason for thinking so”—a textbook example of guessing versus knowing, even if you turn out to be right.5

  But commercial egg operations don’t care about the philosophical conundrum chicken sexing poses. They care about identifying the male chicks so they can grind them up alive to avoid wasting food on them. Since people are reluctant to make a career out of staring up the butts of hundreds of chicks an hour, we might figure that this is a process ripe for traditional computer-based automation.

  It’s not. Traditionally—before machine learning—we’d teach a computer to recognize birds by programming in a model of birds. A model of a thing or system lists its salient features and how they’re arranged relative to one another. For example, for a computer program designed to identify birds, a model might include the sorts of things a field guide points out in its illustrations: the beak, the shape of the head, the body shape and color, the length and shape of the wings relative to the body, the length of the legs, and the shape of the feet attached to those legs. The model also specifies where each of those parts is located in relation to the others. If the computer is intended to identify still photos of perched birds, its model won’t bother to include the way the bird glides or what it sounds like. The model is also likely to ignore whether a bird has sand on its claws, although that might in fact be a clue to its habitat. The model works if it enables the machine to sort birds into the right bins, using the same sort of criteria that human birders would use when arguing about whether they ju
st saw a downy or a hairy woodpecker. “Its beak was as long as its head” is likely to be a convincing argument that it was the latter, for that’s what the models of those two birds tell the quarreling birders.

  That’s exactly the sort of conceptual model that chicken sexers cannot provide for themselves—or to computer programmers trying to build a working model that will let technology replace the sexers. A conceptual model is the idea of a system’s parts and relationships. A working model is a representation of the conceptual model, built out of atoms or bits, that you can manipulate to see how the system behaves. Your old classroom’s solar system made out of small balls, representing the sun and the planets, that you can crank to see their relative positions is a working model. It crudely represents the astronomer’s conceptual model that includes the mathematical relationships that express each planet’s position, mass, and velocity.

  Traditional computers may be stopped by the chicken sexers’ lack of a conceptual model, but now there’s a new type of computer program in town. With machine learning, we could in theory build a chicken-sexing computer without programming a model into it at all. We would train it the same way that chicken sexers are trained: feed it lots of examples of chicks with their sex noted and let the computer figure out for itself what the salient differences are. The result would be a working model, quite possibly without a conceptual model.6 And, as we’ll see in this chapter, that threatens to upend one of our core assumptions about our position as human beings: we are the special creatures who can understand how the world works.

  Working Models, Conceptual Models

  Good news: we can now stop talking about chicken butts, for handwriting recognition has become a standard example of machine learning’s way of acquiring a new skill.7

  Traditionally, in the old world of working models based on conceptual models, we would train a computer to recognize handwritten numbers by telling it what the salient characteristics of the numerals are. For example, to recognize an 8, look for a stack of two circles, with the top one the same size or smaller than the bottom one. With machine learning systems, instead of telling the computer about the geometry of 8s, we would give it thousands of examples of handwritten 8s. Many of those will violate the geometry of penmanship we were taught in grade school: the circles might lean, they will rarely be perfect circles, and many of the circles won’t be fully closed because people write hastily. The machine learning algorithms would analyze those scanned-in samples as grids of various shades of gray (because our writing implements don’t just lay down perfectly black lines) and come up not with a rule about circles but rather with a way to compare a new sample against the distribution of gray dots in all the samples. If the new sample is in fact an 8, the system will—if it’s working—assign it a high probability that it’s an 8, perhaps a lower probability that it is a 3 or a B, and probably a very low probability that it is a 1 or 7.

  This is how the National Archives of the United Kingdom has been teaching its machines to read ancient documents written with quill pens in characters that are hard for us to recognize and that have changed over time. Volunteers transcribed sixty thousand words—the length of a short book—from old manuscripts to create what in machine learning language is called ground truth: because humans identified the letters the pen strokes stand for, we can be highly confident that the identifications are correct. A machine learning system called Transkribus, funded by the European Union, analyzed the scanned-in manuscripts, figured out the range of patterns for the letters, and then applied what it learned to new manuscripts. The pilot project resulted in machines getting more than 86 percent of the characters right, which means humans are still more reliable at the task, but we are far slower. Even with Transkribus’s current level of accuracy, the National Archives thinks it will make its manuscript collection searchable for the first time, which will fling the doors open to researchers.8

  Machine learning systems with varying degrees of accuracy now recognize faces and objects in photographs, translate over one hundred languages, identify teens who may be contemplating suicide, and—highly controversially—are used to identify defendants that are likely to skip out on bail.9 They do this with varying degrees of accuracy, sometimes without having been given a conceptual model, and sometimes using models that they’ve built for themselves that are too complex for humans to understand.

  The inexplicability of some machine learning models may not matter when a machine sexes chickens, or is used to make movie recommendations, but it does when it’s diagnosing a woman’s likelihood of developing breast cancer or recommending a prison term at the end of a trial. The patient may well want to know why the machine has concluded that she needs preemptive surgery, and the defendant may well want to know whether his race had anything to do with the sentence the system recommended. There are tremendous controversies now about whether and how we want to limit the use of this technology in such cases, and perhaps in all cases.

  No matter how that political, cultural, and intellectual struggle resolves itself, it is making us ever more aware that what happens can sometimes be best predicted by models that we may not be able to understand. As we saw in the previous chapter, prediction discloses the source of the constant change that surrounds us: supernatural creatures that can be swayed by sacrificing animals to them, hidden relationships that let us read the future in a swirl of tea leaves, or immutable laws of physics than can be expressed in mathematical equations. In this chapter we’ll see that the models we rely on when making predictions have assumed not only how things happen but also that our human essence is to be the knowers of the world. That’s why we like it when our working models not only work but also reflect our conceptual models.

  Our success with machine learning is teaching us to question those assumptions by showing us a new way to see how things happen—a way that changes our idea about where we as humans stand.

  Models We Can Understand

  We have not always insisted on understanding our predictions. For example, some of the Founding Fathers of the United States made daily records of the weather and the factors they thought were related to it: when plants start blooming, the first frost, and so forth. They hoped this aggregated data would reveal reliable correlations, such as the daffodils’ blooming early signifying that there’s a good chance it will be a wet summer. Until the early 1900s, that sort of weather forecasting worked better than not predicting at all.

  As Nate Silver explains in The Signal and the Noise, this is statistical forecasting: we gather data and use it to make an informed guess about what will happen, based on the assumption that the data is expressing a regularity.10 Silver says that is how hurricanes were predicted until about thirty years ago. It works pretty well, at least as long as the natural system is fairly consistent.

  Statistical forecasting doesn’t need a model of the sort proposed in 1900 by Vilhelm Bjerknes, which we looked at in chapter 1. Bjerknes’s model explained the dynamics of global weather using seven variables and Newtonian physics: relevant factors connected by rules governing their interactions.11 But there was a problem: even using only seven variables, the computations were so complex that in 1922 a mathematician named Lewis Fry Richardson spent six full weeks doing the work required to predict the weather on a day years earlier, based on data gathered on the days before it. He wasn’t even close. After all that grueling work, Richardson’s calculated air pressure was 150 times too high.12

  These days we track hundreds of variables to forecast the weather, as well as to predict the longer-term changes in our climate. We do so with computers that chortle at the 1940s computer—the ENIAC (Electronic Numerical Integrator and Computer)—that took twenty-four hours to predict the next day’s weather.13 Nevertheless, until machine learning, we relied on the model-based technique that harks back to Pierre-Simon Laplace’s demon: if we know the rules governing the behavior of the seven factors that determine the weather, and if we have the data about them for any one moment in the life o
f the Earth, we should be able to predict what the next moment’s weather will be.

  The problem is that so very many factors can affect the weather. In fact, Silver says “the entire discipline of chaos theory developed out of what were essentially frustrated attempts to make weather forecasts.”14 Literally everything on the surface of the planet affects the weather to one degree or another. It is not a coincidence that the example forever associated with Chaos Theory involves a butterfly that creates a catastrophic weather event thousands of miles away.

  So if we want to make a prediction about a system like the weather—a third level of predictive complexity, in the terms discussed in the previous chapter—we seem to be left with bad choices. We can rely on statistics and hope that we’ve been gathering the relevant ones, and that the future will repeat the patterns of the past as surely as the Nile overflows after the Dog Star returns. Or we can figure out the laws governing change and hope that the system is as simple as the model we’re using … and that it is not disrupted by, say, the Krakatoa volcano that erupted in 1883, spewing forth enough ash to cool the seasons for a year and to chill the oceans for a full century afterward.15

  Bjerknes’s seven-factor weather model has the advantage of providing a working model that at least crudely reflects its conceptual model. But we don’t always insist on that. The following four examples show different ways successful working models may or may not coincide with our conceptual models. They’ll also let us see how deeply machine learning models break with our traditional practices and age-old assumptions about how things happen … and our assumptions about how suited we humans are to understanding what happens.

  Spreadsheets

  Although computerized spreadsheets date back to the early 1960s,16 they only became widely used after 1978 when Dan Bricklin, a student working on his MBA at Harvard Business School, was annoyed by a class assignment that required calculating the financial implications of a merger. Having to recalculate all the dependent numbers when any single variable changed was more than irksome.17