Borderlands of Science

by Charles Sheffield

  But this is not what we found when we did the actual iteration. We did not converge to that value at all, but instead we obtained a set of four values that cycled among themselves. So let us ask the question, what would happen if we began with x=0.7142857, as our starting guess? We certainly have the right to use any initial value that we choose. Surely, the value would simply stay there?

  No, it would not.

  What we would find is that on each iteration, the value of y changes. It remains close to 0.7142857 on the first few calculations, then it—quite quickly—diverges from that value and homes in on the four values that we just mentioned: 0.3828, 0.5009, etc. In mathematical terms, the value 0.7142857 is a solution of the iterative process for r=3.5. But it is an unstable solution. If we start there, we will rapidly move away to other multiple values.

  Let us return to the iterative process. By now we are not sure what will happen when we increase r. But we can begin to make some guesses. Bigger values of r seem to lead to more and more different values, among which the sequence will oscillate, and it seems as though the number of these values will always be a power of two. Furthermore, the "splitting points" seem to be coming faster and faster.

  Take r=3.52, or 3.53, or 3.54. We still have four values that alternate. But by r=3.55, things have changed again. We now find eight different values that repeat, one after the other. By r=3.565, we have 16 different values that occur in a fixed order, over and over, as we compute the next elements of the sequence.

  It is pretty clear that we are approaching some sort of crisis, since the increments that we can make in r, without changing the nature of the sequence, are getting smaller and smaller. In fact, the critical value of r is known to many significant figures. It is r=3.569945668. . . . As we approach that value there are 2n states in the sequence, and n is growing fast.

  What happens if we take r bigger than this, say r=3.7? We still produce a sequence—there is no difficulty at all with the computations—but it is a sequence without any sign of regularity. There are no attractors, and all values seem equally likely. It is fair to say that it is chaos, and the region beyond the critical value of r is often called the chaos regime.

  This may look like a very special case, because all the calculations were done based on one particular function, y=rx(1-x). However, it turns out that the choice of function is much less important than one would expect. If we substituted any up-and-down curve between zero and one we would get a similar result. As r increases, the curve "splits" again and again. There is a value of r for which the behavior becomes chaotic.

  For example, suppose that we use the form y=r.sin(x)/4 (the factor of 4 is to make sure that the maximum value of y is the same as in the first case, namely, 1/4). By the time we reach r=3.4 we have four different values repeating in the sequence. For r=3.45 we have eight attractors. Strangest of all, the way in which we approach the critical value for this function has much in common with the way we approached it for the first function that we used. They both depend on a single convergence number that tells the rate at which new states will be introduced as r is increased. That convergence number is 4.669201609 . . . , and is known as the Feigenbaum number, after Mitchell Feigenbaum, who first explored in detail this property of iterated sequences. This property of common convergence behavior, independent of the particular function used for the iteration, is called universality. It seems a little presumptuous as a name, but maybe it won't, in twenty years time.

  This discussion of iterated functions may strike you as rather tedious, very complicated, very specialized, and a way of obtaining very little for a great deal of work. However, the right way to view what we have just done is this: we have found a critical value, less than which there is a predictable, although increasingly complicated behavior, and above which there is a completely different and chaotic behavior. Moreover, as we approach the critical value, the number of possible states of the system increases very rapidly, and tends to infinity.

  To anyone who has done work in the field of fluid dynamics, that is a very suggestive result. For fluid flow there is a critical value below which the fluid motion is totally smooth and predictable (laminar flow) and above which it is totally unpredictable and chaotic (turbulent flow). Purists will object to my characterizing turbulence as "chaotic," since although it appears chaotic and disorganized as a whole, there is a great deal of structure on the small scale since millions of molecules must move together in an organized way. However, the number of states in turbulent flow is infinite, and there has been much discussion of the way in which the single state of laminar flow changes to the many states of turbulent flow. Landau proposed that the new states must come into being one at a time. It was also assumed that turbulent behavior arose as a consequence of the very complicated equations of fluid dynamics.

  Remember the "common sense rule": Simple equations must have simple solutions. And therefore, complicated behavior should only arise from complicated equations. For the first time, we see that this may be wrong. A very simple system is exhibiting very complicated behavior, reminiscent of what happens with fluid flow. Depending on some critical variable, it may appear totally predictable and well-behaved, or totally unpredictable and chaotic. Moreover, experiments show that in turbulence the new, disorganized states come into being not one by one, but through a doubling process as the critical parameter is approached. Maybe turbulence is a consequence of something in the fluid flow equations that is unrelated to their complexity—a hidden structure that is present even in such simple equations as we have been studying.

  This iterated function game is interesting, even suggestive, but to a physicist it was for a long time little more than that. Physics does not deal with computer games, went the argument. It deals with mathematical models that describe a physical system, in a majority of cases through a series of differential equations. These equations are solved, to build an idea of how Nature will behave in any given circumstance.

  The trouble is, although such an approach works wonderfully well in many cases, there are classes of problems that it doesn't seem to touch. Turbulence is one. "Simple" systems, like the dripping of water from a faucet, can be modeled in principle, but in practice the difficulties in formulation and solution are so tremendous that no one has ever offered a working analysis of a dripping tap.

  The problems where the classical approach breaks down often have one thing in common: they involve a random, or apparently random, element. Water in a stream breaks around a stone this way, then that way. A snowflake forms from supersaturated vapor, and every one is different. A tap drips, then does not drip, in an apparently random way. All these problems are described by quite different systems of equations. What scientists wanted to see was physical problems, described by good old differential equations, that also displayed bifurcations, and universality, and chaotic behavior.

  They had isolated examples already. For example, the chemical systems that rejoice in the names of the Belousov-Zhabotinsky reaction and the Brusselator exhibit a two-state cyclic behavior. So does the life cycle of the slime mold, Dictyostelium discoideum. However, such systems are very tricky to study for the occurrence of such things as bifurcations, and involve all the messiness of real-world experiments. Iterated function theory was something that could be explored in the precise and austere world of computer logic, unhindered by the intrusion of the external world.

  We must get to that external and real world eventually, but before we do so, let's take a look at another element of iterated function theory. This one has become very famous in its own right (rather more so, in my opinion, than it deserves to be for its physical significance, but perhaps justifiably most famous for its artistic significance).

  The subject is fractals, and the contribution to art is called the Mandelbrot Set.

  11.5 Sick curves and fractals. Compare the system we have just been studying with the case of the pendulum. There we had a critical curve, rather than a critical value. On the other hand, the be
havior on both sides of the critical curve was not chaotic. Also, the curve itself was well-behaved, meaning that it was "smooth" and predictable in its shape.

  Is there a simple system that on the one hand exhibits a critical curve, and on the other hand shows chaotic behavior?

  There is. It is one studied in detail by Benoit Mandelbrot, and it gives rise to a series of amazing objects (one hesitates to call them curves, or areas).

  We just looked at a case of an iterated function where only one variable was involved. We used x to compute y, then replaced x with y, and calculated a new y, and so on. It is no more difficult to do this, at least in principle, if there are two starting values, used to compute two new values. For example, we could have:

  y=(w2-x2)+a

  z=2wx+b

  and when we had computed a pair (y,z) we could use them to replace the pair (w,x). (Readers familiar with complex variable theory will see that I am simply writing the relation z=z2+c, where z and c are complex numbers, in a less elegant form.)

  What happens if we take a pair of constants, (a,b), plug in zero starting values for w and x, and let our computers run out lots of pairs, (y,z)? This is a kind of two-dimensional equivalent to what we did with the function y=rx(1-x), and we might think that we will find similar behavior, with a critical curve replacing the critical value.

  What happens is much more surprising. We can plot our (y,z) values in two dimensions, just as we could plot speeds and positions for the case of the pendulum to make a phase space diagram. And, just as was the case with the pendulum, we will find that the whole plane divides into separate regions, with boundaries between them. The boundaries are the boundary curves of the "Mandelbrot set," as it is called. If, when we start with an (a,b) pair and iterate for (y,z) values, one or both of y and z run off towards infinity, then the point (a,b) is not a member of the Mandelbrot set. If the (y,z) pairs settle down to some value, or if they cycle around a series of values without ever diverging off towards infinity, then the point (a,b) is a member of the Mandelbrot set. The tricky case is for points on the boundary, since convergence is slowest there for the (y,z) sequence. However, those boundaries can be mapped. And they are as far as can be imagined from the simple, well-behaved curve that divided the two types of behavior of the pendulum. Instead of being smooth, they are intensely spiky; instead of just one curve, there is an infinite number.

  The results of plotting the Mandelbrot set can be found in many articles, because they have a strange beauty unlike anything else in mathematics. Rather than drawing them here, I will refer you to James Gleick's book, Chaos: Making a New Science (Gleick, 1987), which shows some beautiful color examples of parts of the set. All this, remember, comes from the simple function we defined, iterated over and over to produce pairs of (y,z) values corresponding to a particular choice of a and b. The colors seen in so many art shows, by the way, while not exactly a cheat, are not fundamental to the Mandelbrot set itself. They are assigned depending on how many iterations it takes to bring the (y,z) values to convergence, or to a stable repeating pattern.

  The Mandelbrot set also exhibits a feature known as scaling, which is very important in many areas of physics. It says, in its simplest terms, that you cannot tell the absolute scale of the phenomenon you are examining from the structure of the phenomenon itself.

  That needs some explanation. Suppose that you want to know the size of a given object—say, a snowflake. One absolute measure, although a rather difficult one to put into practice, would be to count the number of atoms in that snowflake. Atoms are fundamental units, and they do not change in their size.

  But suppose that instead of the number of atoms, you tried to use a different measure, say, the total area of the snowflake. That sounds much easier than looking at the individual atoms. But you would run into a problem, because as you look at the surface of the snowflake more and more closely, it becomes more and more detailed. A little piece of a snowflake has a surface that looks very much like a little piece of a little piece of a snowflake; a little piece of a little piece resembles a little piece of a little piece of a little piece, and so on. It stays that way until you are actually seeing the atoms. Then you at last have the basis for an absolute scale.

  Mathematical entities, unlike snowflakes, are not made up of atoms. There are many mathematical objects that "scale forever," meaning that each level of more detailed structure resembles the one before it. The observer has no way of assigning any absolute scale to the structure. The sequence-doubling phenomenon that we looked at earlier is rather like that. There is a constant ratio between the distances at which the doublings take place, and that information alone is not enough to tell you how close you are to the critical value in absolute terms.

  Similarly, by examining a single piece of the Mandelbrot set it is impossible to tell at what level of detail the set is being examined. The set can be examined more and more closely, forever, and simply continues to exhibit more and more detail. There is never a place where we arrive at the individual "atoms" that make up the set. In this respect, the set differs from anything encountered in nature, where the fundamental particles provide a final absolute scaling. Even so, there are in nature things that exhibit scaling over many orders of magnitude. One of the most famous examples is a coastline. If you ask "How long is the coastline of the United States?" a first thought is that you can go to a map and measure it. Then it's obvious that the map has smoothed the real coastline. You need to go to larger scale maps, and larger scale maps. A coastline "scales," like the surface of a snowflake, all the way down to the individual rocks and grains of sand. You find larger and larger numbers for the length of the coast. Another natural phenomenon that exhibits scaling is—significantly—turbulent flow. Ripples ride on whirls that ride on vortices that sit on swirls that are made up of eddies, on and on.

  There are classes of mathematical curves that, like coastlines, do not have a length that one can measure in the usual way. A famous one is called the "Koch curve" and although it has weird properties it is easy to describe how to make it.

  Take an equilateral triangle. At the middle of each side, facing outward, place equilateral triangles one third the size. Now on each side of the resulting figure, place more outward-facing equilateral triangles one third the size of the previous ones. Repeat this process indefinitely, adding smaller and smaller triangles to extend the outer boundary of the figure. The end result is a strange figure indeed, rather like a snowflake in overall appearance. The area it encloses is finite, but the length of its boundary turns out to be 3x4/3x4/3x4/3 . . . , which diverges to infinity. Curves like this are known as pathological curves. The word "pathological" means diseased, or sick. It is a good name for them.

  There is a special term reserved for the boundary dimension of such finite/infinite objects, and it is called the Hausdorff-Besicovitch measure. That's a bit of a mouthful. The boundaries of the Mandelbrot set have a fractional Hausdorff-Besicovitch measure, rather than the usual dimension (1) of the boundary of a plane curve, and most people now prefer to use the term coined by Mandelbrot, and speak of fractal dimension rather than Hausdorff-Besicovitch dimension. Objects that exhibit such properties, and other such features as scaling, were named fractals by Mandelbrot.

  Any discussion of chaos has to include the Mandelbrot set, scaling, and fractals, because it offers by far the most visually attractive part of the theory. I am not convinced that it is as important as Feigenbaum's universality. However, it is certainly beautiful to look at, highly suggestive of shapes found in Nature and—most important of all—it tends to show up in the study of systems that physicists are happy with and impressed by, since they represent the result of solving systems of differential equations.

  11.6 Strange attractors. This is all very interesting, but in our discussion so far there is a big missing piece. We have talked of iterated functions, and seen that even very simple cases can exhibit "chaotic" behavior. And we have also remarked that physical systems often exhibit chao
tic behavior. However, such systems are usually described in science by differential equations, not by iterated functions. We need to show that the iterated functions and the differential equations are close relatives, at some fundamental level, before we can be persuaded that the results we have obtained so far in iterated functions can be used to describe events in the real world.

  Let us return to one simple system, the pendulum, and examine it in a little more detail. First let's recognize the difference between an idealized pendulum and one in the real world. In the real world, every pendulum is gradually slowed by friction, until it sits at the bottom of the swing, unmoving. This is a single point, termed an attractor for pendulum motion, and it is a stable attractor. All pendulums, unless given a periodic kick by a clockwork or electric motor, will settle down to the zero angle/zero speed point. No matter with what value of angle or speed a pendulum is started swinging, it will finish up at the stable attractor. In mathematical terminology, all points of phase space, neighbors or not, will approach each other as time goes on.

  A friction-free pendulum, or one that is given a small constant boost each swing, will behave like the idealized one, swinging and swinging, steadily and forever. Points in phase space neither tend to be drawn towards each other, nor repelled from each other.

  But suppose that we had a physical system in which points that began close together tended to diverge from each other. That is the very opposite of the real-world pendulum, and we must first ask if such a system could exist.

  It can, as we shall shortly see. It is a case of something that we have already encountered, a strong dependence on initial conditions, since later states of the system differ from each other a great deal, though they began infinitesimally separated. In such a case, the attractor is not a stable attractor, or even a periodic attractor. Instead it is called a strange attractor.