23
theories of chaos lorenz equations
John von Neumann had a dream. In 1954, speaking at the dedication of the world’s new largest computer, Neumann (one of America’s first great computer scientists) predicted that computers would one day make it possible to forecast the weather for 30 to 60 days.
More than half a century later, computer power has grown beyond Neumann’s wildest dreams. Yet computer weather forecasting has made only incremental improvements. Two-day forecasts are now pretty reliable. Five-day forecasts, considerably less so. And not even the most optimistic meteorologist dreams any more of predicting a storm 60 days in advance.
What went wrong? The answer, in a word, is chaos.
A chaotic system is one that obeys deterministic rules—such as the equations that describe how air circulates in the atmosphere—yet behaves, after a certain amount of time, as if it were random. In theory, if you have perfect information about a chaotic system, you can make perfect forecasts. But the slightest inaccuracy or incompleteness in your data will grow exponentially over time, and eventually render your forecast useless.
Ironically, the one tool that most enabled scientists to grasp the implications of chaos, and thereby to place a permanent limit on the power of computation … was the computer. The computer did something that von Neumann never expected. Instead of merely crunching numbers, it provided humans a new way of looking at the world.
x, y, and z are meteorological variables in an abstract and highly simplified model of the atmosphere. These equations were historically the first dynamical system in which scientists recognized the possibility of chaos.
A second irony is that the person who first understood the mathematical concept of chaos was not a mathematician. Edward Lorenz, a meteorologist, discovered the equations above that effectively ended von Neumann’s dream.
Lorenz’s three simple equations, which describe a highly idealized atmosphere, are shown above. It is worth taking a little bit of time to understand what these equations mean. First, note that they are differential equations: they express the rate of change of three variables (x, y, and z) in terms of their current values. Calculus was built to solve such equations. They are the type of equations Newton and Laplace used to describe the motion of planets, and that Apollo engineers used to send rockets to the Moon. For centuries, scientists assumed that the solutions to such equations were manageable and predictable.
Second, note that the equations are nonlinear. They include two terms (xz in the second equation, and xy in the third one) that are not first powers of the variables but products of two variables (which makes them count as second powers). This tiny detail makes all the difference in the world.
Linear equations describe a textbook world, where effects are always proportional to their causes. The whole is always exactly equal to the sum of its parts. A tiny error in measuring one of the variables will remain a tiny error—or at worst, will grow in nice linear fashion—for all time. It is a world where the weather is predictable for weeks, or months, or even forever.
THE REAL WORLD, however, is nonlinear. Feedback loops amplify small causes into big effects. In biology, nonlinearity arises whenever one cell signals another cell to stop working, or start working harder. In chemistry, nonlinearity occurs when one chemical catalyzes a reaction involving another. In aerodynamics, the equations become nonlinear when the air turns into a moving object rather than a passive medium. Most of the interesting phenomena in science, including anything that involves the mediation of one part of a system by another part, are nonlinear.
Below The Lorenz Attractor, a three-dimensional graph which became emblematic of chaos.
In the Lorenz equations, thanks to those two terms xy and xz, the variable x mediates the way that z responds to y, and also mediates the way that y responds to z. Nevertheless, this nonlinearity seems so mild that nearly every mathematician who has seen these equations has probably thought, “Why, I could solve them!” But they can’t.
Now I’ll explain what the variables x, y, and z meant to Lorenz when he wrote these equations down in 1963. Lorenz’s model describes the convection of air in a long rectangular tube when the bottom is heated. Hot air tends to rise, so eventually a rolling current forms, with hot air rising on one side of the tube and cool air descending on the other. But in Lorenz’s model, the convection current eventually starts going too fast. The hot air doesn’t have a chance to cool down completely before it gets swept down the other side of the tube. Hot air doesn’t like to descend, so this slows down the rolling motion—which eventually stops, and then switches direction. These reversals of direction constitute the unpredictable feature of the system.
Lorenz describes the meaning of the variables x, y, and z as follows: x represents the strength of the convective current; y represents the size of the temperature gradient between the ascending and descending streams. The meaning of z is a bit elusive, but it is crucial: “The variable z is proportional to the distortion of the vertical temperature profiles from linearity, a positive value indicating that the strongest gradients occur near the boundaries,” Lorenz writes.
When the variables x, y, and z are plotted over time, from almost any starting point, they will eventually coalesce around an intricate, butterfly-shaped structure, shown opposite. The two “wings” of the butterfly correspond to the two directions of rotation of the convection currents. A typical trajectory starts toward the center of one wing and gradually spirals outward. Eventually it goes “too far” out (the convection currents get out of control). At that point the trajectory plunges through the complicated mess between the two wings and emerges on the other side, ready to start its spiraling motion again.
If you start a second trajectory at a slightly different point, it will behave the same way for a little while. Both trajectories may, for instance, make two loops around the left wing, then three loops around the right. But the distance between the trajectories will grow, and then there will come a time where trajectory 1 veers left while trajectory 2 heads right. From then on, the two trajectories will be uncorrelated. It is easy to make the metaphorical leap to weather forecasts. Think of “left” as “sunny” and “right” as “rainy.” If trajectory 1 represents the real weather, while trajectory 2 represents a forecast based on slightly different initial conditions, the two may agree for a few days, but eventually the forecast is guaranteed to bear no resemblance to reality.
Before we leave the details of the Lorenz model behind, let me point out one more curious feature: the seemingly arbitrary constants, 10, 28, and 8/3. These are called “parameters,” and they have a strong effect on the shape of the solutions. If you replace the number 28 by 24 (or any number below about 24.8), the chaos disappears; the convection currents are not sufficiently excitable. Starting from any initial state, the convection currents will eventually settle down into a stable state, either rotating left or rotating right. After that, the system becomes 100 percent predictable. Thus, nonlinearity is not a guarantee of chaos; it merely opens up the possibility. The tipping point where a regular system becomes chaotic often depends on parameters that we cannot observe.
In one short paper, Lorenz had identified most of the main ingredients of chaos, although he had not named them yet: Sensitive dependence on initial conditions (the “butterfly effect”)‡; long-term behavior that is controlled by an infinitely complicated (and beautiful!) geometric structure (later called a “strange attractor”); a parameter (or several) that can switch on or switch off the chaos; and nonlinear but completely deterministic dynamics.
The importance of Lorenz’s paper was not immediately apparent; it was buried in a specialist journal, read only by meteorologists. However, the same process repeated in other disciplines. Michel Hénon, an astronomer, discovered chaos in the equations governing stellar orbits around a galaxy’s center. David Ruelle, a physicist, along with mathematician Floris Takens, discovered strange attractors in turbulent fluid flow. Robert May, a biolo
gist, discovered chaos in the simplest system of all: a one-variable equation that had been used for years to model populations competing for scarce resources.
Each of these pioneers was isolated at first, and they all faced disbelief from other scientists. A colleague of Lorenz, Willem Markus, recalled in James Gleick’s bestselling book Chaos: Making a New Science what he told Lorenz about his equations: “Ed, we know—we know very well—that fluid convection doesn’t do that at all.”
This incredulity is perhaps a typical reaction to any paradigm-altering discovery. In the case of chaos there were specific reasons why mathematicians and other scientists had been so blind for so long. When mathematicians teach their students differential equations, they concentrate on the simplest, most understandable cases first. First, they teach them to solve linear equations. Next, they might teach them about some simple two-variable systems, and show how the behavior of the solutions near a fixed point can be described by linearizing. No matter what the number of variables, they will always concentrate on equations that can be solved explicitly: x(t) is given by an exact formula involving the time t.
ALL OF THESE simplifying assumptions are perfectly understandable, especially the last one. Solving equations is what mathematicians do … or did, in the years BC (before chaos). And yet these assumptions are collectively a recipe for blindness. Chaos does not occur in linear systems; it does not occur in a continuous-time system with less than three variables;§ and it does not occur in any system where you can write a formula for the solution.
It is as if mathematicians erected a “Danger! Keep out!” sign at all of the gates leading to chaos. Scientists from other disciplines—biologists, physicists, meteorologists—never went past the “Keep out!” signs, and so when they encountered chaos it was something utterly unfamiliar.
A very small number of mathematicians did venture past the warning signs. The first one, universally acknowledged by all chaos theorists, was Henri Poincaré, France’s greatest mathematician at the turn of the century. In 1887, he entered an international competition, sponsored by the King of Sweden, to find a solution to the three-body problem, in other words to find explicit formulas for the trajectories of three or more mutually gravitating planets. Isaac Newton had, of course, solved the problem for two bodies, and it had been a bone in the throat of mathematicians ever since that they could not do the same for even the simplest system of three bodies.
Poincaré won the prize even though he did not solve the problem. In fact, he thought he had solved it, but as he was preparing his manuscript for publication (after he had been awarded the prize!) he discovered a mistake. He had assumed that small perturbations in a planet’s motion would produce small effects. Analyzing a planet’s “return map” more carefully, he realized that was not the case. Thus, he clearly discovered the first hallmark of chaos, the sensitive dependence on initial conditions. Much more obscurely, he sensed the second feature as well, the strange attractor. In the following passage he is describing the trajectories of planets in phase space, but I encourage you to think about the Lorenz attractor as you read it:
“These intersections form a kind of lattice-work, a weave, a chain-link network of infinitely fine mesh; each of the two curves can never cross itself, but it must fold back on itself in a very complicated way so as to recross all the chain-links an infinite number of times … One will be struck by the complexity of this figure, which I am not even attempting to draw.”
He is not describing the Lorenz attractor per se, but he might as well be. There, too, we see a phenomenally complex interweaving of curves, a sort of freeway where infinitely many lanes merge and then go off in different directions without colliding. Thus mathematicians had the opportunity to discover chaos in 1893, when Poincaré’s book appeared. But they didn’t. They were not prepared to look for chaos; the whole point of the prize competition was to look for stable solutions.
Opposite Fractal image of part of the Mandelbrot Set. Fractal geometry is part of the mathematics of chaos, the study of unpredictable dynamical systems.
The other reason mathematicians were blind to chaos was that they had no computers, and were left with the kind of vague description that Poincaré gave, which other mathematicians failed to understand. With a computer, you can’t help but see the attractor in all its glory. For scientists like Lorenz and Hénon who were not professional mathematicians, Poincaré’s work was inaccessible but the strange attractors were there on their computer printouts, begging to be explained.
BETWEEN 1893 AND 1970, mathematicians assembled some of the ingredients of chaos theory, without managing to bring them all together. Around the same time as Poincaré, Aleksandr Lyapunov in Russia defined the Lyapunov exponent, a measure of the tendency of nearby trajectories to diverge. In the 1930s and 1940s, Mary Cartwright and John Littlewood in England studied the van der Pol equation, an early nonlinear equation used in radio and radar. Littlewood commented on the “whole vista of the dramatic fine structure of solutions.” In the 1960s, Steven Smale, an American mathematician and fervent anti-Vietnam War activist, described very general topological conditions which guaranteed that a dynamical system would approach a complicated limit set. This established that chaos was a generic phenomenon that exists over a wide range of parameter values. Finally, Benoit Mandelbrot in France, for completely different reasons, opened up the world’s eyes to the ubiquity of “fractals” in nature. The Lorenz attractor, with a fractional dimension of about 2.07, is a prime example. Mandelbrot put his finger exactly on what was strange about strange attractors: they have fine structure at every scale, so that a magnified version looks just as finely filigreed as an unmagnified version.
IN THE 1970S the grand synthesis occurred. The disparate scientists who had encountered chaos began to find each other and connect with the mathematicians who could explain their discoveries. In 1975 the field acquired its seductive nickname, thanks to a paper by Tien-Yien Li and James Yorke, called “Period Three Implies Chaos.” Li and Yorke showed that a one-variable discrete dynamical system, like the one studied by May, must be chaotic if there is even one point that comes back to itself after three time steps. A Ukrainian mathematician, Olexandr Sharkovsky, had proved the same fact eleven years earlier, but no one in the West knew about it because of the lack of communication across the Iron Curtain. Sharkovsky’s paper was finally translated in 1995 and the theorem is now known after him, but Li and Yorke had the honor of giving “chaos theory” its name.
In the 1980s and ’90s, the trope of “chaos” leaped out of the realm of science and into popular culture. James Gleick brought attention to the field with his best-selling book. In the blockbuster movie Jurassic Park, one of the leading characters is a doom-predicting chaos theorist, and chaos is the central metaphor for the disastrous failure of scientists to anticipate the consequences of their actions.
The same period was a high-water mark for the subject, with three major chaos journals launching in 1990 and 1991. Two decades later, the excitement has died down a bit. In fact, I was surprised to read in a recent historical survey that “as a unified site of social convergence, the ‘science of chaos’ does not exist any more.”
I think the obituary is premature. Certainly a number of interesting discoveries have been made in chaos since 1990. Perhaps the most surprising is synchronized chaos. You might expect that two oscillators that both behave in an apparently random way would be impossible to synchronize. Yet it turns out that if you feed just one of the three output variables of one Lorenz oscillator (or any other chaotic system of your choice) into another Lorenz oscillator, both of the other variables in the second oscillator will also lock onto the first oscillator. It is an ingenious way to exploit the deterministic laws that lie hidden underneath the apparent randomness. This may explain how living organisms, such as nerve cells, synchronize their behavior.
Other recent developments include the discovery of limited forms of “quantum chaos.” For a long time the equations of qu
antum mechanics resisted the incursion of chaos because they are linear. It’s a good thing they are; we would not want the electrons, protons, and neutrons that make up matter to be unstable. But somewhere in the “quasi-classical limit,” the gray zone between the macroscopic world and the quantum world, chaos has to make its appearance, and both mathematicians and physicists have been probing how.
Finally, much work remains to be done in understanding turbulent fluids. Chaos is not the end of the story, but only the beginning. Using Lyapunov exponents, scientists can find the invisible attracting structures and repelling structures that orchestrate fluid motion. They can identify where the flow is chaotic and where it is not. They can map out, for example, the invisible dividing line between water in the Gulf of Mexico that will circulate back into the gulf and water that will escape into the Atlantic. Methods like this could have been used to predict the motion of the BP oil spill in 2010.
So I think it is fair to say that the concept of chaos is alive and well, and always will be, now that we have finally learned to see it. The discipline of chaos is also alive and well, although it may have outlived the fad stage and will probably end up being seen as an organic part of a more traditionally named discipline, such as “dynamical systems” or “nonlinear differential equations.”
* * *
‡ This name came from the title of a paper that Lorenz himself presented in 1972, called “Predictability: Does the Flap of a Butterfly’s Wings in Brazil Set Off a Tornado in Texas?”
§ In discrete-time systems, such as May’s equation that describes the change in population of a species from one year to the next, chaos is possible even with only one variable.
The Universe in Zero Words Page 17