Dancing With Myself

by Charles Sheffield

  Well, we ought to be amazed. It is not just that we can write down an equation for the relation between the acceleration and the speed. It is that this integration process somehow corresponds exactly to something in the real world, so that when we have done our pencil-and-paper integrations, we are able to know where the object will be in reality.

  No justification is usually offered for this in courses on mechanics and the calculus, other than the fact that it works. It becomes what Douglas Hofstadter in another context terms a “default assumption.”

  This is an especially simple example, but scientists are at ease with far more complex cases. Do you want to know how a fluid will move? Write down the three-dimensional time-dependent Navier-Stokes equation for compressible, viscous flow, and solve it. That’s not a simple proposition, and you may have to resort to a computer. But when you have the results, you expect them to apply to real fluids. If they do not, it is because the equation you began with was not quite right—maybe we need to worry about electromagnetic forces, or plasma effects. Or maybe the integration method you used was numerically unstable, or the finite difference interval too crude. The idea that the mathematics cannot describe the physical world never even occurs to most scientists. They have in the back of their minds an idea first made explicit by Laplace: the whole universe is calculable, by defined mathematical laws. Laplace said that if you told him (or rather, if you told a demon, who was capable of taking in all the information) the position and speed of every particle in the Universe, at one moment, he would be able to define the Universe’s entire future, and also its whole past.

  The twentieth century, and the introduction by Heisenberg of the Uncertainty Principle, weakened that statement, because it showed that it was impossible to know precisely the position and speed of a body. Nonetheless, the principle that mathematics can exactly model reality is usually still unquestioned.

  Now, hidden away in the assumption that the world can be described by mathematics there is another one; one so subtle that most people never gave it a thought. This is the assumption that chaos theory makes explicit, and then challenges. We state it as follows:

  Simple equations must have simple solutions.

  There is no reason why this should be so, except that it seems that common sense demands it. And, of course, we have not defined “simple.”

  Let us return to our accelerating object, where we had a simple-seeming equation, and an explicit solution. One requirement of a simple solution is that it should not “jump around” when we make a very small change in the system it describes. For example, if we consider two cases of an accelerated object, and the only difference between them is a tiny change in the original position of the object, we would expect a small change in the final position. And this is the case.

  But now consider another simple physical system, a pendulum (this was one of the first cases where the ideas of chaos theory emerged). The equation that describes the motion of a simple pendulum, consisting of a bob on a string, can be written down easily enough. It is:

  d2q/dt2 + k.sin(q) = 0

  Even if you are not familiar with calculus, this should seem hardly more complicated than the first equation we considered. And for small angles, q, its solution can easily be written down, and it exhibits the quality of simplicity, namely, the solution changes little when we make a small change in the starting position or starting speed of the pendulum bob.

  However, when large angles, q, are permitted, a completely different type of solution becomes possible. If we start the bob moving fast enough, instead of swinging back and forward, like a clock, the pendulum keeps on going, right over the top and down the other side. If we write down the expression for the angle as a function of time, in one case the angle is a periodic function (back and forth) and in the other case it is constantly increasing (round and round). And the change from one to the other occurs when we make an infinitesimal change in the initial speed of the pendulum bob. This type of behavior is known as a bifurcation in the behavior of the solution, and it is a worrying thing. A simple equation begins to exhibit a complicated solution.

  The mathematician Poincaré used a powerful graphical method to display the behavior of the solutions of dynamical problems. It is called a phase space diagram, and it plots position on one axis, and speed on the other. For any assumed starting position and speed, we can then plot out where the solution goes on the phase diagram. (It works, because the equations of dynamics are what is known as second-order equations; for such equations, when the position and speed of an object are specified, that defines the nature of its whole future motion.)

  If we make the phase space diagram for the case of the uniformly accelerating object, the result is not particularly interesting. It is shown in Figure 1, and consists of a set of parabolas.

  Figure 1: Phase space diagram for uniform accelerated motion.

  Things become more interesting when we do the same thing for the pendulum (Figure 2). Phase space now has two distinct regions, corresponding to the oscillating and the rotating forms of solution, and they are separated by an infinitely thin closed boundary. An infinitely small change of speed or position at that boundary can totally change the nature of the solution.

  This kind of behavior can be thought of as a sensitive dependence on the initial conditions. In fact, at the dividing curve of the phase space diagram, it is an infinitely sensitive dependence on initial conditions.

  Figure 2: Phase space diagram for the simple pendulum.

  At this point, the reasonable reaction might well be, so what? All that we have done is show that certain simple equations don’t have really simple solutions. That does not seem like an earth-shaking discovery. For one thing, the boundary between the two types of solution for the pendulum, oscillating and rotating, is quite clear-cut. It is not as though the definition of the location of the boundary itself were a problem.

  Can situations arise where this is a problem? Where the boundary is difficult to define in an intuitive way? The answer is, yes. In the next section we will consider simple systems that give rise to highly complicated boundaries between regions of fundamentally different behavior.

  4.ITERATED FUNCTIONS

  Some people have a built-in mistrust of anything that involves the calculus. When you use it in any sort of argument, they say, logic and clarity have already departed. The examples I have given so far began with a differential equation, and needed calculus to define the behavior of the solutions. However, we don’t need calculus to demonstrate fundamentally chaotic behavior; and many of the first explorations of what we now think of as chaotic functions were done without calculus. They employed what is called iterated function theory. Despite an imposing name, the fundamentals of iterated function theory are so simple that they can be done with an absolute minimum knowledge of mathematics. They do, however, benefit from the assistance of computers, since they call for large amounts of tedious computation.

  Consider the following very simple operation. Take two numbers, x and r. Form the value y = rx(1-x).

  Now plug the value of y back in as a new value for x. Repeat this process, over and over.

  For example, suppose that we take r = 2, and start with x = 0.1. Then we find y = 0.18.

  Plug that value in as a new value for x, still using r = 2, and we find a new value, y = 0.2952.

  Keep going, to find a sequence of y’s, 0.18, 0.2952, 0.4161, 0.4859, 0.4996, 0.5000, 0.5000…

  In the language of mathematics, the sequence of y’s has converged to the value 0.5. Moreover, for any starting value of x, between 0 and 1, we will always converge to the same value, 0.5, for r = 2.

  Here is the sequence when we begin with x = 0.6:

  0.6000, 0.4800, 0.4992, 0.5000, 0.5000…

  Because the final value of y does not depend on the starting value, it is termed an attractor for this system, since it “draws in” any sequence to itsel
f.

  The value of the attractor depends on r. If we start with some other value of r, say r = 2.5, we still produce a convergent sequence. For example, if for r = 2.5 we begin with x = 0.1, we find successive values: 0.1, 0.225, 0.4359, 0.6147, 0.5921, 0.6038, 0.5981…0.6. Starting with a different x still gives the same final value, 0.6. (For anyone with a computer available to them and a knowledge of a programing language such as FORTRAN or BASIC, I recommend playing this game for yourself. The whole program is only a dozen lines long, and fooling with it is lots of fun. Suggestion: Run the program in double precision, if you have it available, so you don’t get trouble with round-off errors. Warning: larking around with this sort of thing will consume hours and hours of your time.)

  The situation does not change significantly with r = 3. We find the sequence of values: 0.2700, 0.5913, 0.7250, 0.5981, 0.7211…0.6667. This time it takes thousands of iterations to get to a final converged value, but it makes it there in the end. Even after only a dozen or two iterations we can begin to see it “settling-in” to its final value.

  There have been no surprises so far. What happens if we increase r a bit more, to 3.1? We might expect that we will converge, but even more slowly, to a single final value.

  We would be wrong. Something very odd happens. The sequence of numbers that we generate has a regular structure, but now the values alternate between two different numbers, 0.7645, and 0.5580. Both these are attractors for the sequence. It is as though the sequence cannot make up its mind. When r is increased past the value 3, the sequence “splits” to two permitted values, which we will call “states,” and these occur alternately.

  Let us increase the value of r again, to 3.4. We find the same behavior, a sequence that alternates between two values.

  But by r = 3.5, things have changed again. The sequence has four states, four values that repeat one after the other. For r = 3.5, we find the final sequence values: 0.3828, 0.5009, 0.8269, and 0.8750. Again, it does not matter what value of x we started with, we will always converge on those same four attractors.

  Let us pause for a moment and put on our mathematical hats. If a mathematician is asked the question, “Does the iteration y = rx(1-x) converge to a final value?”, he will proceed as follows:

  Suppose that there is a final converged value, V, towards which the iteration converges. Then when we reach that value, no matter how many iterations it takes, at the final step x will be equal to V, and so will y. Thus we must have V = rV(1 - V).

  Solving for V, we find V = 0, which is a legitimate but uninteresting solution, or V = (r - 1)/r. This single value will apply, no matter how big r may be. For example, if r = 2.5, then V = 1.5/2.5 = 0.6, which is what we found. Similarly, for r = 3.5, we calculate V = 2.5/3.5 = 0.7142857.

  But this is not what we found when we did the actual iteration. We did not converge to that value at all, but instead we obtained a set of four values that cycled among themselves. So let us ask the question, what would happen if we began with x = 0.7142857, as our starting guess? We certainly have the right to use any initial value that we choose. Surely, the value would simply stay there?

  No, it would not.

  What we would find is that on each iteration, the value of y changes. It remains close to 0.7142857 on the first few calculations, then it—quite quickly—diverges from that value and homes in on the four values that we just mentioned: 0.3828, 0.5009, etc. In mathematical terms, the value 0.7142857 is a solution of the iterative process for r = 3.5. But it is an unstable solution. If we start there, we will rapidly move away to other multiple values.

  Let us return to the iterative process. By now we are not sure what will happen when we increase r. But we can begin to make some guesses. Bigger values of r seem to lead to more and more different values, among which the sequence will oscillate, and it seems as though the number of these values will always be a power of two. Furthermore, the “splitting points” seem to be coming faster and faster.

  Take r = 3.52, or 3.53, or 3.54. We still have four values that alternate. But by r = 3.55, things have changed again. We now find eight different values that repeat, one after the other. By r = 3.565, we have 16 different values that occur in a fixed order, over and over, as we compute the next elements of the sequence.

  It is pretty clear that we are approaching some sort of crisis, since the increments that we can make in r, without changing the nature of the sequence, are getting smaller and smaller. In fact, the critical value of r is known to many significant figures. It is r = 3.569945668…. As we approach that value there are 2n states in the sequence, and n is growing fast.

  What happens if we take r bigger than this, say r = 3.7? We still produce a sequence—there is no difficulty at all with the computations—but it is a sequence without any sign of regularity. There are no attractors, and all values seem equally likely. It is fair to say that it is chaos, and the region beyond the critical value of r is often called the chaos regime.

  This may look like a very special case, because all the calculations were done based on one particular function, y = rx(1-x). However, it turns out that the choice of function is much less important than one would expect. If we substituted any up-and-down curve between zero and one (see Figure 3) we would get a similar result. As r increases, the curve “splits” again and again. There is a value of r for which the behavior becomes chaotic.

  Figure 3: Bifurcations do not depend qualitatively on the form of the function.

  For example, suppose that we use the form y = r sin(x)/4 (the factor of 4 is to make sure that the maximum value of y is the same as in the first case, namely, 1/4). By the time we reach r = 3.4 we have four different values repeating in the sequence. For r = 3.45 we have eight attractors. Strangest of all, the way in which we approach the critical value for this function has much in common with the way we approached it for the first function that we used. They both depend on a single convergence number that tells the rate at which new states will be introduced as r is increased. That convergence number is 4.669201609…, and is known as the Feigenbaum number, after Mitchell Feigenbaum, who first explored in detail this property of iterated sequences. This property of common convergence behavior, independent of the particular function used for the iteration, is called universality. It seems a little presumptuous as a name, but maybe it won’t, in twenty years’ time.

  This discussion of iterated functions may strike you as rather tedious, very complicated, very specialized, and a way of obtaining very little for a great deal of work. However, the right way to view what we have just done is this: we have found a critical value, less than which there is a predictable, although increasingly complicated behavior, and above which there is a completely different and chaotic behavior. Moreover, as we approach the critical value, the number of possible states of the system increases very rapidly, and tends to infinity.

  To anyone who has done work in the field of fluid dynamics, that is a very suggestive result. For fluid flow there is a critical value below which the fluid motion is totally smooth and predictable (laminar flow) and above which it is totally unpredictable and chaotic (turbulent flow). Purists will object to my characterizing turbulence as “chaotic,” since although it appears chaotic and disorganized as a whole, there is a great deal of structure on the small scale since millions of molecules must move together in an organized way. However, the number of states in turbulent flow is infinite, and there has been much discussion of the way in which the single state of laminar flow changes to the many states of turbulent flow. Landau proposed that the new states must come into being one at a time. It was also assumed that turbulent behavior arose as a consequence of the very complicated equations of fluid dynamics.

  Remember the “common sense rule”: Simple equations must have simple solutions. And therefore, complicated behavior should only arise from complicated equations. For the first time, we see that this may be wrong. A very simple system is exhibiting v
ery complicated behavior, reminiscent of what happens with fluid flow. Depending on some critical variable, it may appear totally predictable and well-behaved, or totally unpredictable and chaotic. Moreover, experiments show that in turbulence the new, disorganized states come into being not one by one, but through a doubling process as the critical parameter is approached. Maybe turbulence is a consequence of something in the fluid flow equations that is unrelated to their complexity—a hidden structure that is present even in such simple equations as we have been studying.

  This iterated function game is interesting, even suggestive; but to a physicist it was for a long time little more than that. Physics does not deal with computer games, went the argument. It deals with mathematical models that describe an idea of how Nature will behave in any given circumstance.

  The trouble is, although such an approach works wonderfully well in many cases, there are classes of problems that it doesn’t seem to touch. Turbulence is one. “Simple” systems, like the dripping of water from a faucet, can be modeled in principle, but in practice the difficulties in formulation and solution are so tremendous that no one has ever offered a working analysis of a dripping tap.

  The problems where the classical approach breaks down often have one thing in common: they involve a random, or apparently random, element. Water in a stream breaks around a stone this way, then that way. A snowflake forms from supersaturated vapor, and every one is different. A tap drips, then does not drip, in an apparently random way. All these problems are described by quite different systems of equations. What scientists wanted to see was physical problems, described by good old differential equations, that also displayed bifurcations, and universality, and chaotic behavior.