The bigger difficulty remains: even though the scale of observation becomes ever smaller, the scale on which weather phenomena begin is always smaller still. Even if the world were surrounded by a lattice of sensors only one meter apart, things would happen unobserved between them, spawning unexpected major weather systems within days. Part of the answer, therefore, is to introduce a little extra randomness on this smallest scale. Take, for instance, inertia-gravity waves: these are tiny shudders in the atmosphere or ocean that arise when, say, air flows over rough ground or tide runs against pressure; you can sometimes see their trace in rows of small, high, chevron-shaped clouds, like nested eyebrows. These waves are usually too small to resolve in existing weather models, but there are circumstances when they can shape larger movements in the atmosphere. How can this rare but significant causation be accommodated in the predictive model? By introducing a random term; in effect, making the picture jiggle slightly at its smallest scale. Most of the time, this jiggling remains too small to affect the larger system; but sometimes it sets off changes that would not appear in a simplified, deterministic model that filtered out small-scale variation. This technique is called stochastic resonance and it has applications as widely separated as improving human visual perception and giving robots better balance. Adding randomness can actually aid precision.
Meteorologists are interested in the weather itself as a fascinating, complex system—but they also know the rest of us are primarily interested in its effects. We only want to know about the weather so we can exclude it from our portfolio of uncertainty. Tim Palmer is therefore working on ways to connect the probability gradients that come out of ensemble forecasting to the resource-allocation decisions that people and institutions need to make. “Let’s take malaria in Africa. Its appearance and spread is very weather dependent: there are good models that connect the weather to the disease, but it’s hard to use these to predict outbreaks because the way in which the weather is integrated into the model is quite complicated. Instead, we can attach the disease model directly to our ensemble prediction and generate, not a weather probability curve, but a malaria probability curve. The weather becomes just an intermediate variable. Health organizations can then make the decision to take preventative action wherever the malaria probability exceeds a particular threshold.”
This is the great difference between probabilistic forecasting and traditional forecasting: the decision is up to us. And as with all problems of free will, getting rid of Fate is psychologically difficult; the world becomes less easy the more we know about it. In the old days, when the smiling face on the evening news simply told us that overnight temperatures were going to be above freezing—and then a frost cracked our newly poured concrete or killed our newly sown wheat—we could blame it on the weatherman. Probabilistic forecasting offers a percentage chance (with, if you want to explore the data further, a confidence spread and a volatility); this will mean little until we can match it with our own personal percentages: our ratio of potential gain to loss, our willingness to assume risk. Once the weather ceases to be fate or fault, it becomes another term in our own constant calculation of uncertainty.
One day in 1946 Stanislaw Ulam lay convalescing in bed, playing solitaire. While others might lose themselves in the delicious indolence of illness and the turning cards, Ulam kept wondering exactly what the chances were that a random standard solitaire, laid out with 52 cards, would come out successfully. Perhaps we can count ourselves fortunate that he was not feeling well, since, after trying to solve the problem by purely combinatorial means, Ulam gave up—and discovered a less mathematically elegant but more generally fruitful approach: “I wondered whether a more practical method than ‘abstract thinking’ might not be to lay it out, say, one hundred times and simply observe and count the number of successful plays.”
He described his idea to John von Neumann, who, with his dandyish streak, called it the “Monte Carlo method”—because it resembles building a roomful of roulette wheels and setting them all spinning to see how your bet on the whole system fares over the long term. This technique has spread through every discipline that requires an assessment of the sum of many individually unpredictable events, from colliding neutrons in atomic bombs, through financial market trades, to cyclones. When analysts say, “We’ll run it through the computer,” they are usually talking about these probabilistic simulations.
Monte Carlo (or, for the less dandyish, stochastic simulation) means inviting the random into the heart of your calculations. It takes the results of observation—statistics—and feeds them back as prior probabilities. Let’s say you know statistically how often a neutron is absorbed in a given interaction. You set up in your computer simulation a program that assigns the probability of absorption for that part of the system randomly, but with the randomness weighted according to your observation, like a roulette wheel laid out with red and black distributed according to your statistics. The randomly generated results are then fed into the next stage of the simulation, which is programmed in the same way. The whole linked simulation then becomes like an ensemble prediction: if you run it over and over again, you get a distribution that you can analyze, just as if it came from a real collection of experiments or observations. This may seem conceptually crude, but it can generate results as precise as you can afford with the time you have available and the power of your computer.
In the world of weather, Monte Carlo simulation gives insurers a handle on catastrophe. Until about 1980, any assessment of exposure to, say, hurricane loss was based on little more than corporate optimism or pessimism. Although cloaked in equations, the reasoning of many insurers ran along these simple lines: “Imagine the worst storm possible in this area. Assume our loss from it will be equal to the worst loss we’ve ever had, plus a percentage reflecting how much bigger we’ve grown since then. Then decide how often that worst storm is likely to happen and spread the potential loss over the years it won’t.” Straightforward reasoning, but dangerous—not just because it involves guesswork and approximation, but because the wind doesn’t work that way. It bloweth where it listeth: its power is released, not smoothly over wide areas, but savagely in narrow confines. It does not wait a decent interval before striking again; the “hundred-year storm” is a deceptively linear form of words disguising a non-linear reality.
Now, therefore, the computers of large insurers are given over to ensemble forecasting—taking, for instance, the data for known U.S. hurricanes and generating from them a simulated 50,000 years’ worth of storms yet to come. Each of these thousands of stochastically generated tracks takes its eraser through a different range of insured property, and every major building is modeled separately for its vulnerability to wind from different directions. The result is a distribution not of weather, but of loss, allowing premiums to reflect the full range of potential claims. The apparent randomness of disaster (my house reduced to kindling; yours, next door, with its patio umbrella still in place) is not ignored but taken as the basis for the overall assessment of risk.
Probabilistic reasoning about weather means a shift from asking “What will happen?” to considering “What difference does what could happen make to me?” This is an obvious calculation for catastrophe, but it also extends into the normal variety of days: cool or balmy, damp or dry. When you are luxuriating in an unseasonably warm fall, think for a moment of the bad news it represents for the woolen industry. Well, if this is the weather the shepherd shuns, maybe he should do something about it.
He could, for instance, call Barney Schauble, whose company insures against things that normally happen. Schauble’s background is financial, so he knows how prosaic, day-to-day risks can cause problems for many businesses: “Energy companies created this market, because weather makes such a difference to their demand; a warm winter in Denver or a cool summer in Houston can really throw off your income projections if you sell gas for heating or electricity for air conditioners. If you own a theme park, the weather can make a big
difference: rain is bad, but rain on Saturday is worse and rain on the Saturday of Memorial Day weekend is worst of all.” The sums involved in even small variations can add up. The summer of 1995 in England and Wales was, in its modest way, unusually hot: temperatures were between 1°C and 3°C above average. The extra payout for the insurance industry that year, in claims for lost crops, lost energy consumption, lost clothing sales, and building damage through soil shrinkage and subsidence totaled well over $2 billion.
Barney Schauble’s company focuses on the variable but generally predictable: those elements of the weather that have normal distributions around definable means. “Heating days, cooling days, precipitation above a threshold during a defined period: things that have many years of accurate data behind them. People come to us because they’re exposed to a risk they simply can’t avoid, but one that we can diversify. If it were a financial risk, most companies would know right away that they needed to hedge it. Weather is still a new market but it works much the same way; companies come to us to even out their expectations over time and make their balance sheets more predictable.
“People aren’t used to thinking about a range of possible events that could happen normally—they tend to divide things between the unimportant and the catastrophic. They have to think more probabilistically about risk management. In finance, you don’t make a point forecast for exchange rates six months ahead: instead, you talk about a range of movement with a percentage of likelihood. If you can think this way about all kinds of risk, it makes the world as a whole a less risky place, because you accept that the unusual is within the range of probability. Otherwise you’re committing to a guess—and the one sure thing about a guess is that it will be erroneous.”
Schauble has no use for forecasting (“We had a forecaster for a while: he was neither consistently right nor consistently wrong—either of which would have been preferable”) because he deals in smoothing out variation. Long series of recorded data provide all he needs to know about volatility, and his clients are prepared to pay a little from the surplus of good years to reduce the loss in bad. For some enterprises, though, accurate prediction of this year’s weather could be essential to business decisions.
California’s southern San Joaquin valley is a one-crop district—something you might guess if you drove through the town of Raisin on a September afternoon. Out in the vineyards, paper trays laid out between the rows hold the year’s harvest: more than $400 million worth—99 percent of the state’s crop—by far the largest single collection of raisins in the world. It takes three weeks of sunshine to dry out a Thompson’s Seedless; if you fear that rain is on the way during that crucial period, you could decide to sell your grapes off the vine for juice. The profit would be smaller, but certain. What should you do?
This decision links to the broader question of how to calculate risk and advantage from random events. Its treatment dates back to the solution proposed in 1738 by Daniel Bernoulli to the Saint Petersburg Paradox, a problem first described by his cousin Nicholas in 1718. The paradox is easily stated: A lunatic billionaire proposes a coin-flipping game in which tails pays you nothing, but the first head to come up will pay you 2n ducats—where n is the number of the flip on which heads first appeared. How much should you pay to join the game? Your expectation should be the chance of the payout times the amount of the payoff. In this case, the first flip has probability 1/2 and a payment of 2 ducats, so your expectation for it is 1 ducat; the second has a probability of 1/4, a payout of 4, an expectation of 1 ducat; and so on. Your expectation of the game as a whole, therefore, is 1 + 1 + 1 + 1 + ... in fact, it’s infinite. Should you therefore pay an infinite number of ducats to make the bet even? You would have to be even more deranged than your opponent.
Daniel Bernoulli’s solution was to propose that money, although it adds up like the counting numbers, does not grow linearly in meaning as its sum increases. If the billionaire proposed either to pay you one dollar or to flip a coin double or quits, you (and most people) would take your chance on winning two dollars. If instead the billionaire proposed to pay you one million or give you a 1-in-2 chance at two million, you, like most people, would swallow hard, probably thinking of what they would say at home if you came back empty-handed. Money’s value in calculations of probability is based on what you could do with it—its utility, to use the term favored by economists. When the financier John Jacob Astor comforted a friend after the Panic of 1837 by saying, “A man who has a million dollars is as well off as if he were rich,” he was making a subtle point: the marginal utility of wealth over that sum was small. So if you assume that there is some term that expresses the diminishing marginal utility of money as it increases, then you should multiply your expectation from each of the games in the Saint Petersburg series by that term: the result is a finite stake for the game.
Information also has marginal utility for all transactions that depend on uncertainty. If you are a raisin farmer, the accuracy of the weather forecast can have enormous marginal utility, because it moves your decision—whether to sell now for grape juice or go for the full shrivel—out of the realm of coin flipping and into sensible risk management. You could even take the probability figure for rain and use it to calculate which proportion of the harvest to bank as juice and thus cover the extra costs of protecting the remainder from rain on its journey to raisinhood. The problem is that the marginal utility of information is also a function of how many people have it. Fresno County, home to some 70 percent of the crop, is not a big place: every vineyard is subject to the same weather. Demand, whether for raisins or grape juice, is not fathomless. So in fact, the market price of juice or raisins depends in part on at least a few growers’ having bet the wrong way on the weather. If all raisin growers had access to a perfect three-week forecast and thus made the “right” decision every time, supplies would rise, prices would fall, and the industry as a whole be out of pocket.
Fate, it seems, justifies a premium for business dependent on the weather—and for some this premium, in turn, justifies a stubborn insistence on letting fate do its worst. No French appellation contrôlée allows the farmer to irrigate in times of drought. Most forbid using black plastic mulches to keep down weeds and warm the soil. Rules closely circumscribe rootstocks, fungicides, and winemaking techniques—all in the name of terroir, the mystical union of microclimate and geology that makes a Chablis grand cru distinguishable from a Catawba super-Chard. Such a self-denying regime may indeed be essential to assure the subtle evocation of time and place that French wine achieves at its best; but there is also the argument that only by allowing the weather to wash out the Cabernet in ’97 and frizzle the Merlot in ’03 can the industry justify the prices it asks when everything miraculously comes out right. Nothing in this business—not pricing, not information, not the weather itself—has a normal distribution. Things do not settle to the average; the system remains resolutely non-linear.
Ed Lorenz said that the upper bound for dependable forecasting is ten days. Of course, our biggest worry now is about dependable forecasting for the next hundred years. The old meteorological saw says, “Climate is what you expect; weather is what you get”—but even climatic expectation is shot through with uncertainty.
Back in 1939, Time magazine breezily mentioned that “gaffers who claim that winters were harder when they were boys are quite right . . . Weathermen have no doubt that the world at least for the time being is growing warmer.” This improving climate seemed another of fate’s gifts to America, favorite among nations. Life was becoming daily richer and more convenient—why not the weather? Who wouldn’t welcome the idea of palm trees in Maine? So why are we worried now? Because we begin to have an inkling of the dynamics of climatic change—just enough to see how little positive control we have over it.
Climate is a human invention: an “average” of a system that does not tend to averages. Like rippling water in a stream, climate moves in reaction to many superimposed forces. Recurring ice ages seem
related to wobbles in our orbit. The Sun itself varies in its radiant energy. Populations of living things, like the phytoplankton at the base of the ocean’s food chain, go through their own chaotic trajectories of growth and collapse. The Southern Oscillator, source of El Niño events, knocks the regularity of tropical weather off center every four or five years. Volcanoes periodically call off every meteorological bet: one in New Guinea in the sixth century blotted out the sun for months, drawing a final line under the Roman Empire; Tambora in Indonesia canceled summer in 1816; and the great magma chamber under Yellowstone will one day blow its lid and make all human worries seem comparatively trivial. Each of these independent forces pulses at a different rate, enforcing or retarding the effects of the others on our climate. And then there’s us, burning our coal and oil, reversing, in a couple of centuries, millions of years of carbon absorption by ancient plants.
Time was right: things are getting warmer. Global average temperatures have gone up about 0.6°C over the twentieth century; mean sea level has risen around seven inches as the warmer water expands and as melting glaciers and ice caps add their long-hoarded substance to the liquid ocean. The mean, though, is a fiction; things are warming up much faster in the high subarctic latitudes while areas of the North Pacific are actually cooling. We also know that change, when it happens, happens fast: ice cores, tree rings, lake sediments all reveal past major climate shifts occurring within a geological blink.
Chances Are Page 26