The Future of Everything: The Science of Prediction

by David Orrell

  PRESENT

  4 RED SKY AT NIGHT

  PREDICTING THE WEATHER

  Big whirls have little whirls that feed on their velocity, and little whirls have lesser whirls and so on to viscosity—in the molecular sense.

  —Lewis Fry Richardson, British physicist and psychologist

  There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say, we know there are some things we do not know. But there are also unknown unknowns—the ones we don’t know we don’t know.

  —Donald Rumsfeld, secretary of defense of the United States

  TAKING THE TEMPERATURE

  If the world has a favourite opening topic for conversation—an icebreaker—it is probably the weather. If you are ever lost for anything more meaningful to say, you can always try “Chilly out, isn’t it?” (or the equivalent in the local dialect). It usually isn’t very controversial, unless it’s noon in the Sahara, and it might lead to something more interesting.

  One of the first to write down his thoughts on the atmosphere and the oceans was, again, Aristotle. Like a good scientist, he focused on hammering the theories of his competitors and promoting his own. “The belief held by Democritus that the sea is decreasing in volume and that it will in the end disappear is like something out of Aesop’s fables,” he thundered in Meteorologica (meteorology was defined more broadly then, and encompassed such phenomena as comets, earthquakes, and oceans). “Plato’s description of rivers and seas in the Phaedo is impossible,” he declared. “It is equally absurd for anyone to think, like Empedocles, that he has made an intelligible statement when he says that the sea is the sweat of the Earth. Such a statement is perhaps satisfactory in poetry, for metaphor is a poetic device, but it does not advance our knowledge of nature.” His own view was that thunder, lightning, and hurricanes were all caused by “windy exhalations,” perhaps from quarrelling philosophers.

  Aristotle’s student Theophrastus of Eresos compiled the first attempt to predict weather using empirical observations. It included such chestnuts as “Red sky at night, sailor’s delight / Red sky at morning, sailor take warning,” and created a basis for weather forecasting that, while short on theoretical backing, persisted for two millennia.

  Like scientific prediction in general, weather forecasting didn’t take a major step forward until the Renaissance, when Galileo built the first thermometer and his student Torricelli followed up with the barometer. Galileo’s early version of a thermometer was a glass bulb attached to a tube the size of a straw. In lecture demonstrations, he would warm the bulb and insert the end of the straw into a basin filled with water. As the bulb cooled, the air inside would contract, drawing liquid up into the straw. The height it reached reflected the room’s temperature, so Galileo could make it go up or down by moving the device from a warm place to a cold one.

  A variety of improved versions soon appeared. One, by Robert Hooke, was filled with “best rectified spirit of wine highly ting’d with the lovely colour of cochineal,” perhaps so that you could drink the contents if it got too cold out.1 In 1714, Gabriel Fahrenheit constructed sealed mercury thermometers with reliable scales, which turned the device into a serious scientific instrument.

  Temperature, we now know, is a measurement of the kinetic energy of air molecules—that is, how fast they are moving on average— as opposed to wind, which measures the net overall motion of a parcel of air. The barometer measures the atmospheric air pressure, which is essentially the air’s weight, or number of molecules per unit volume. Pressure and temperature therefore represent statistical averages over a large number of particles. It is meaningful to take such averages because one air molecule is assumed to be physically the same as any other.

  Aristotle tried to weigh air by first weighing a leather bag when it was pressed flat, then weighing it again when it was full of air. He found there was no difference, so concluded that air was without weight. Galileo, always the skeptic, tried to devise better experiments, but it was his student Torricelli who managed to accurately weigh air with the barometer. This was a metre-high tube partially filled with mercury, its open end inverted into a reservoir so that the weight of the air pressing down on the reservoir was balanced by the mercury in the tube. Fluctuations in pressure caused the mercury level to go up or down.2 Blaise Pascal proved that pressure decreases with height by convincing his brother-in-law to lug a barometer up a mountain and compare the pressure at the top with that at the base. (The air at the bottom has all the weight of the air above pressing down on it, which increases the pressure.)

  The fact that air has weight will seem reasonable to anyone who has seen tornadoes tossing around telephone poles like they were matchsticks. But of course it’s not just the weight, or the heat, that matters: it’s the humidity. Molecules of water can be mixed in with the air. An early hygrometer, for measuring humidity, was supplied by the ever-inventive Hooke, who employed the “beard of a wild goat.” This protruded from a hole in the top of a well-ventilated box, and its end was attached to a pointer. As the hair curled or uncurled depending on the moisture in the air, the pointer indicated the humidity on a dial. Similar hair hygrometers are still in use today, though another method is to compare the temperature of the humid air with that of dry air.3

  By the mid-1600s, temperature, pressure, and humidity were being measured on a regular basis. In 1724, the Royal Society managed to organize a truly international data-collection arrangement, complete with standardized forms and instruments. Synchronized observations had to await the invention of the telegraph in 1844— until then, the weather tended to outrun the messenger—and led to weather predictions based on a synopsis of the available data. The synoptic method exploited the fact that the weather has identifiable features, such as masses of warm/cold or dry/humid air, or storms. If, say, you learn over the telegram that frigid polar air was last seen heading your way, then it is reasonable to suppose that it will soon get cold.

  FORECAST

  The first meteorological office was set up in Britain in 1854. The Met. Office, as it became known, was headed by Admiral Robert FitzRoy—he who had captained the Beagle and taken Darwin around the world. The ex-navy man saw that weather forecasting had the potential to save lives by warning mariners of storms, like the one that destroyed nearly thirty French and British vessels in the Crimean a month before his appointment. A network of forty weather stations was set up around the United Kingdom, and weather reports were published in London newspapers. In France, the chemist and accountant Antoine Lavoisier funded a chain of observation stations before being sent to the guillotine for his unpopular taxation activities.

  FitzRoy’s efforts were also not well received, by the public or the scientific establishment. At the time, weather prediction was something practised by astrologers, and it was not seen as a fit subject for scientific pursuit. The popular press enjoyed comparing the Met. Office’s inaccurate predictions to those from astrological sources, such as Zadkiel’s Almanac. The mainstream scientists saw all this as a threat to their reputation.

  FitzRoy tried to blunt the comparison to astrology by avoiding loaded words like “prediction”; instead, he invented a new word of his own: forecast. “Prophecies or predictions they are not; the term forecast is strictly applicable to such an opinion as is the result of a scientific combination and calculation.”4 In 1863, he published The Weather Book, which tried to make the weather comprehensible to people of average education. But his attempt to popularize the subject further annoyed elitist scientific institutions like the Royal Society. It didn’t help that he spoke about the weather in intuitive rather than mathematical terms, and claimed that the telegraphic network had provided him with a “means of feeling . . . successive states of the atmosphere over the greater extent of our islands,”5 which sometimes sounded more mystical than objective.

  On April 30, 1865, at the age of fifty-nine, Robert FitzRoy took his own life by slitting his throat with his razor. He might
have had an inherited tendency to depression. His uncle Lord Castlereagh had similarly killed himself, and Darwin, who dined with him on the Beagle every day for five years, had noted his occasionally stormy temper. He might also have been affected by his association with Darwin’s theory of evolution, which, as a creationist, he considered blasphemous. However, it appears that the primary cause of his depression was being caught between the so-called astro-meteorologists on the one side and the scientific establishment on the other. After his death, a committee formed by the Royal Society and the Board of Trade, and chaired by Darwin’s cousin Sir Francis Galton, released a report that tore apart every aspect of FitzRoy’s work. It claimed that his forecasts were not deduced “by means of accurate induction from known facts,” and were “wanting in all elements necessary to inspire confidence.”6 As a result, storm warnings were suspended.

  FitzRoy, however, was not without supporters. Fishermen, maritime insurers, and the navy had actually found the storm warnings useful, and they were reinstated in 1867. Public interest in weather forecasts was also aided by the publication, in the London Times, of weather maps, which like their modern versions showed winds, precipitation, temperature, and air pressure.

  OBSERVATION AND THEORY

  Galton had become interested in maps and meteorology while exploring southwestern Africa for the Royal Geographical Society, and he played a large part in the development of weather maps. He was nothing if not an enthusiastic measurer. On his many journeys to Africa and elsewhere, he measured the longitude and latitude of towns and the altitude of mountains (by noting the barometric pressure at the summit). On one occasion, he measured an African woman whose voluptuous physique he found particularly striking. He recorded “a series of observations upon her figure in every direction, up and down, crossways, diagonally, and so forth, and I registered them carefully upon an outline for fear of any mistake; this being done I boldly pulled out my measuring tape, and measured the distance from where I was to the place where she stood, and having thus obtained both base and angles, I worked out the results by trigonometry and logarithms.”7 As we’ll see in the next chapter, his interest in measuring human bodies continued when he later shifted from meteorology to his theory of inheritance.

  The Norwegian scientist Vilhelm Bjerknes also believed that meteorology needed to be shed of its astrological associations and turned into a hard science like physics. The key would be to show that it could provide accurate predictions. Echoing Laplace’s allseeing demon, he wrote in a 1904 paper that this would require two things: “1. A sufficiently accurate knowledge of the state of the atmosphere at the initial time. 2. A sufficiently accurate knowledge of the laws according to which one state of the atmosphere develops from another.”8 In other words, forecasting requires the initial condition (today’s weather) and a model of the atmospheric dynamics. Observation and theory; a Tycho and a Kepler.

  As it happened, the quality of both these factors was improving because of the impending war in Europe. Regular flights by airships and airplanes meant improved observational data. The new military technologies of aviation, long-range artillery, and gas warfare were all affected by the weather. Meteorologists suddenly found themselves in demand by military planners. Indeed, the term “front” for the boundary between different air masses was taken from First World War military terminology.

  Bjerknes devised a set of seven differential “primitive equations,” based on the Navier-Stokes fluid-flow equations from the 1840s, to model the atmosphere. They were essentially an application of Newton’s laws of motion to fluid flow, and they described how the atmospheric variables of temperature, wind, air pressure, and humidity change with time. Some of the relations can be understood intuitively. For example, differences in pressure between regions create a force that drives winds from high pressure to low. Converging winds increase the pressure, while diverging winds decrease it. Temperature also varies with pressure, since air gets warmer as it is compressed. The equations could only coarsely approximate the effects of turbulence, or the behaviour of water vapour, but they captured the overall atmospheric flow.

  Solving the equations numerically was difficult, so Bjerknes had to rely on graphical techniques, based on weather maps, to find approximate solutions. However, this was so slow that it was hard to make useful forecasts before the weather arrived. As the Englishman Lewis Fry Richardson realized, true numerical weather prediction would require computers—though he used the word differently than we do today.

  THE STORMS OF WAR

  Richardson was a Quaker and a pacifist. When the First World War started, he had a comfortable position working for the Met. Office at Eskdalemuir Observatory in Scotland. In 1916, at the age of thirty-five, he quit his job to serve as an ambulance driver in the war. While helping ferry the wounded away from the fronts, he developed the ideas behind his book Weather Prediction by Numerical Process. His aim was to show how the weather could be predicted using the method of finite differences.

  Richardson had experience with the finite difference technique from a previous position with the National Peat Industries, where he had modelled the flow of water in peat. The method works by dividing both space and time into a Cartesian grid. At the initial time t = 0, the values of atmospheric variables such as temperature, wind, and pressure are specified at each grid point. The finer the grid, the greater number of variables. At time ∆t in the future, the differential equations are used to anticipate how the variables will change, based on the physical laws. This step is repeated each ∆t time units. The procedure is similar to the numerical techniqueused for solving Newton’s laws of motion, described in Chapter 2, except that instead of a single falling stone, the system was a swirling mass of air and vapour.

  For a square area measuring about 1,000 kilometres a side, roughly centred on Germany, Richardson chose a grid of 5 divisions in each horizontal direction and 5 vertically, for a total of 125 cells. Each was about 200 kilometres square and 2 to 5 kilometres high (since pressure decreases with height, the upper layers were made taller so they contained a similar mass of air). For his initial condition, he used observations provided by Bjerknes for May 20, 1910, when observatories throughout Western Europe had released a large number of weather balloons.

  Because the balloon launches did not coincide with the grid, Richardson’s first step was to estimate, or interpolate, the values for each cell based on the nearest measurements. He then attempted to calculate changes in the atmospheric variables six hours into the future, using a time step ∆t of three hours. Since there were hundreds of variables, and even more equations, the computation was a massive task. It was amazing enough that Richardson completed it, even more so that he did it while in the middle of a war. Scribbling calculations into his notebooks, his office a “heap of hay in a cold rest billet,” he almost lost everything during the Battle of Champagne, in April 1917, when the manuscript was sent to the rear and apparently lost, before being found months later under a heap of coal. In the end, he succeeded in producing the world’s first numerical forecast (or hindcast, since the date was in the past). Unfortunately, his predicted changes in air pressure were out by an ear-blowing factor of about a hundred. He blamed poor observational data, though instabilities in his numerical method also played a role.

  The calculations for a six-hour prediction took him about the same number of weeks to carry out. “Perhaps some day in the dim future it will be possible to advance the calculations faster than the weather advances,” he concluded. “But that is a dream.”9 He imagined a large hall, a kind of “forecast factory,” full of “computers,” by which he meant people armed with slide rules, each of whom would be responsible for calculating the changes in one small cell of the atmosphere. The walls would be painted with a map of the globe, and at a pulpit in the centre would be a single person, “like a conductor of an orchestra,” who kept everybody in time. Richardson estimated that his scheme would require 64,000 “computers” just to keep pace with the weather. Per
haps as a result, other scientists didn’t try to pursue numerical weather forecasting.

  Another approach, pioneered by the climatologist Sir Gilbert Walker in his attempts to predict the Indian monsoon rains, was to look for statistical patterns in past data.10 That didn’t catch on either—in part because of the lack of high-quality, global observational data; in part because such statistical methods do not provide a testable, cause-and-effect explanation for phenomena; but mostly because weather patterns are rather unreliable.11 However, the techniques Walker developed to find patterns and correlations in graphs of variables such as rainfall versus time, known as time series analysis, helped to detect weather patterns associated with El Niño and are still used in many areas of prediction, including finance.

  LONG-DISTANCE CONNECTIONS

  El Niño is now usually associated with extreme weather, but the name was first used to describe a benign event off the coast of Peru—a warming of the ocean waters. This, it was said, would lead to an año de abundancia, when “the sea is full of wonders, the land even more so. First of all the desert becomes a garden. . . . The soil is soaked by the heavy downpour, and within a few weeks the whole country is covered by abundant pasture. The natural increase of flocks practically doubles and cotton can be grown in places where in other years vegetation seems impossible.” (Murphy 1926). The inhabitants of the region called the warm current El Niño, after the Christ child, since it usually set in shortly after Christmas. (The opposite state is now called La Niña, “the little girl.”)