by Lee Smolin
But even though Weyl’s first try at unification failed, he had invented the modern concept of unification that would lead eventually to string theory. He was the first, but far from the last, to proclaim, “I am bold enough to believe that the whole of physical phenomena may be derived from one single universal world-law of the greatest mathematical simplicity.”8
A year after Weyl’s theory, a German physicist named Theodor Kaluza found a different way to unify gravity and electromagnetism, by reviving Nordström’s idea of the hidden dimension. But he did it with a twist. Nordström had found gravity by applying Maxwell’s theory of electromagnetism to a five-dimensional world (of which four dimensions are spatial and one is time). Kaluza did this in reverse: He applied Einstein’s general theory of relativity to a five-dimensional world and found electromagnetism.
You can visualize this new space by attaching a little circle to each point of ordinary three-dimensional space (see Fig. 4). This new geometry can curve in new ways, because the little circles can be attached differently at different points. There is then something new to measure at each point of the original three-dimensional space. This information turns out to look just like the electric and magnetic fields.
Another wonderful spin-off is that it turns out that the charge of the electron is related to the radius of the little circle. This should not be surprising: If the electric field is just a manifestation of geometry, the electric charge should be, too.
Fig. 4. Curled-up extra dimensions used in Kaluza-Klein theory. Left: A sphere is placed over every point of ordinary three-dimensional space, making a five-dimensional space. Right: A small circle is placed over a one-dimensional space. From far away, the space looks one-dimensional, but examined closely it is seen to be two-dimensional.
And not only this. General relativity describes the dynamics of the geometry of spacetime in terms of certain equations, called the Einstein equations. I don’t need to write them down to describe the key fact: Those same equations can be applied to the five-dimensional world we just described. As long as we impose one simple condition, they turn out to be the right equations to describe the electric and magnetic fields and gravity, unified together. Thus, if this theory is right, the electromagnetic field is just another name for the geometry of the fifth dimension.
Kaluza’s idea was rediscovered and further developed in the 1920s by the Swedish physicist Oskar Klein. Their theory was beautiful and compelling indeed. Gravity and electromagnetism are unified in one blow, and Maxwell’s equations are explained as coming out of Einstein’s equations, all by the simple act of adding a single dimension to space.
This time Einstein was enthralled. In April 1919 he wrote to Kaluza, “The idea of achieving [a unified theory] by a five-dimensional cylinder had never occurred to me. . . . At first glance I like your idea enormously.”9 In a letter some years later to Dutch physicist Hendrik Lorentz, he exulted, “It appears that the union of gravitation and Maxwell’s theory is achieved in a completely satisfactory way by the five-dimensional theory.”10 George Uhlenbeck, a prominent physicist, remembered first hearing Klein’s idea in 1926: “I felt a kind of ecstasy! Now one understands the world.”11
Unfortunately, Einstein and the other enthusiasts were wrong. As with Nordström’s theory, the idea of unification by adding a hidden dimension failed. It is important to understand why.
I said earlier that for a proposed unification to succeed, it has to win its place by making new predictions that are confirmed by experiment. Successful unifications also generate a plethora of new insights that lead to further discoveries. Compelling as it was to some, neither of these things happened in the case of Kaluza-Klein theory. The reason is simple: The theory imposed an extra condition, referred to earlier, which is that the extra dimension is curled up into a circle whose radius is too small to see. Not only that: To get electromagnetism out of the theory, the radius of the circle must be frozen, changing in neither space nor time.
This is the Achilles’ heel of the whole enterprise and led directly to its failure. The reason is that freezing the radius of the extra dimension undermines the very essence of Einstein’s theory of general relativity, which is that geometry is dynamical. If we add another dimension to spacetime as described by general relativity, the geometry of that extra dimension should also be dynamical. And indeed, it would be, were the radius of the little circle allowed to move freely. The theory of Kaluza and Klein would then have infinitely many solutions in which the radius of the circle varies over space and changes in time. This would have wonderful implications, because it would lead to processes in which gravitational and electrical effects convert into each other. It would also lead to processes in which electrical charges vary over time.
But if the Kaluza-Klein theory is a true unification, the fifth dimension cannot be treated differently from the others: The little circle must be allowed to change. The resulting processes are hence the necessary consequences of unifying electricity and geometry. If they were ever observed, they would confirm directly that geometry, gravity, electricity, and magnetism are all aspects of one phenomenon. Unfortunately, such effects have never been observed.
This is not one of those cases in which theorists can quickly celebrate the consequences of the unification; instead, they must hide it, by insisting on studying only an infinitesimal fraction of solutions where the radius of the fifth dimension is frozen in space and time.
It gets worse, because such solutions, it turns out, are unstable. Tickle the geometry just a bit, and the small circle collapses quickly to a singularity marking the end of time. Tickle it a different way and the circle grows, so that soon the extra dimension becomes visible, discrediting the theory entirely. As a result, the theory’s predictions must be hidden to cover the fact that it gets so much wrong.
At this point, even Einstein lost his enthusiasm. He wrote to his friend Paul Ehrenfest, “It is anomalous to replace the four-dimensional continuum by a five-dimensional one and then subsequently to tie up artificially one of those five dimensions in order to account for the fact that it does not manifest itself.”12
As if this were not enough, physicists had other reasons to reject the theory. By the 1930s, people knew that there were more forces in the world than gravity and electromagnetism. They knew about the strong and the weak nuclear forces, so it made no sense to leave these out of the unification. But no one knew how to include them in these unified theories. Still, for a while the search for a unified-field theory continued, led by Einstein. Some of the great mathematicians and physicists of the time contributed to this effort, including Wolfgang Pauli, Erwin Schrödinger, and Weyl. They found other ways to modify the geometry of spacetime so as to unify gravity with electromagnetism. These relied on deep mathematical insights, but they, too, led nowhere; they either made no new predictions or they predicted phenomena that were not seen. By the 1940s, Einstein and the few others who still pursued a unified-field theory were mostly laughed at.
My first job after getting my PhD was in 1979 at the Institute for Advanced Study, in Princeton. One of my main reasons for taking it was the hope of making contact with some living legacy of Einstein, who had died twenty-four years earlier. In this I was disappointed. There was no trace of his time there, apart from a bust of him in the library. No student or follower of Einstein could be found. Only a few people who had known him, like the theoretical physicist Freeman Dyson, were still there.
My first week there, Dyson, very much the gentleman, came by and invited me to lunch. After inquiring about my work, he asked if there was anything he could do to make me more at home in Princeton. I had but one request. “Could you tell me what Einstein was really like?” I asked. Dyson replied, “I’m very sorry, but that’s one thing I can’t help you with.” Surprised, I insisted, “But you came here in 1947 and you were a colleague of his until he died in 1955.”
Dyson explained that he too had come to the institute hoping to get to know Einstein. So he went to Ein
stein’s secretary, Helen Dukas, to make an appointment. The day before the appointment, he began to worry about not having anything specific to discuss with the great man, so he got from Ms. Dukas copies of Einstein’s recent scientific papers. They were all about Einstein’s efforts to construct a unified-field theory. Reading them that evening, Dyson decided they were junk.
The next morning, he realized that although he couldn’t face Einstein and tell him his work was junk, he couldn’t not tell him either. So he skipped the appointment and, he told me, spent the ensuing eight years before Einstein’s death avoiding him.
I could only say the obvious: “Don’t you think Einstein could have defended himself and explained his motivation to you?”
“Certainly,” Dyson replied. “But I was much older before that thought occurred to me.”
One problem that Einstein and the other few unified-field theorists faced (besides the derision of the particle physicists) was that this kind of unification turned out to be too easy. Rather than being hard to find, unified-field theories were a dime a dozen. There were many different ways to achieve them and no reason to choose one over another. In decades of work, there was only one real advance: The problem of incorporating the two nuclear forces was solved. It turned out that all that was required was to add still more extra dimensions. The fields needed to describe the weak and strong nuclear forces pop out when several more new dimensions are added to general relativity. The story is much the same as Kaluza’s attempt with electromagnetism: One has to freeze the geometry of the extra dimensions, making sure that their geometry never changes in time or varies in space, and one must make them too small to see. When all this is done correctly, the necessary equations (known as Yang-Mills equations) result from applying the equations of general relativity to the higher dimensions.
The fact that the Yang-Mills equations were hidden in higher-dimensional extensions of general relativity was not discovered until the 1950s, but their significance was not grasped until the 1970s, when we finally understood that these equations described the weak and strong nuclear forces. When people did finally make that connection, there were a few attempts at reviving the Kaluza-Klein idea, but they didn’t go very far. By then, we had learned that nature lacked a certain symmetry—that of parity between left and right. Specifically, all neutrinos are what is referred to as left-handed (that is, the direction of their spin is always opposite to that of their linear momentum). This means that if you look at the world in a mirror, you will see a false world—one in which neutrinos are right-handed. So the world seen in a mirror is not a possible world. But this asymmetry turned out to be hard to explain in a world described by Kaluza-Klein theory.
Beyond that, the higher-dimensional theories continued to make no new predictions. The conditions we had to impose on the extra dimensions in order to get the physics we wanted were the theory’s seeds of destruction. Indeed, the more dimensions you include, the higher the price you pay for freezing their geometry. The more dimensions, the more degrees of freedom—and the more freedom is accorded to the geometry of the extra dimensions to wander away from the rigid geometry needed to reproduce the forces known in our three-dimensional world. The problem of instability gets worse and worse.
Moreover, whenever there is more than one hidden dimension, there are many different ways to curl them up. Rather than there being just one—a circle—there are an infinite number of ways the higher dimensions can be curled up, so there are an infinite number of possible versions of the theory. How is nature to choose among them?
Over and over again in the early attempts at unifying physics through extra dimensions, we encounter the same story. There are a few solutions that lead to the world we observe, but these are unstable islands in a vast landscape of possible solutions, the rest of which are very unlike our world. And once conditions are imposed to eliminate those, there is no smoking gun—no consequence of the unification that has not yet been seen but might be, if experimentalists were to look for it. So there is nothing to celebrate and a lot to hide.
But there was an even more fundamental problem, which had to do with the relationship of the unified theories to the quantum theory. The early attempts at unified-field theories took place before quantum mechanics was completely formulated, in 1926. Indeed, a few of the quantum theory’s proponents had interesting speculations about the relationship between the extra dimensions and quantum theory. But after 1930 or so, there was a split. Most physicists ignored the problem of unification and concentrated instead on applying quantum theory to a vast array of phenomena, from the properties of materials to the processes by which stars produce energy. At the same time, those few who persisted in working on the unified theories increasingly ignored quantum theory. These people (Einstein included) worked as if Planck, Bohr, Heisenberg, and Schrödinger had never existed. They were living after the quantum-mechanical revolution but pretending to work in an intellectual universe in which that revolution had never occurred. They seemed to their contemporaries like the quaint communities of Russian émigré aristocrats who, in the 1920s and 1930s, carried on their elaborate social rituals in Paris and New York as if they were back in tsarist Saint Petersburg.
Of course, Einstein was not just some washed-up émigré intellectual from a lost world (even if he was an émigré intellectual from a lost world). He knew full well that he was ignoring quantum theory, but he had a reason: He didn’t believe in it. Even though he had sparked the quantum revolution with his realization that the photon was real, he rejected the outcome. He was hoping to discover a deeper theory of quantum phenomena that would be acceptable to him. That is exactly where he hoped his unified-field theory would lead him.
But it didn’t. Einstein’s dream of an end run around quantum theory failed, and it more or less died with him. By that time, few respected him and fewer followed him. The physicists at the time thought they had better things to do than play with fanciful ideas about unification. They were hard at work cataloging the many new particles that were being discovered and honing the theories of the two newly discovered fundamental forces. That someone might speculate that the world had more than three dimensions of space, curled up too small too see, seemed to them as crazy and unproductive as studying UFOs. There were no implications for experiment, no new predictions, so, in a period when theory developed hand in hand with experiment, no reason to pay attention.
But suppose for a moment that despite all the obstacles, we still wanted to take ideas of a unified field seriously. Could these theories be formulated in the language of quantum theory? The answer was resoundingly no. No one knew at that time how to make even general relativity consistent with quantum theory. All early attempts to do this failed. When you added more dimensions, or more twists to the geometry, things always got worse, not better. The larger the number of dimensions, the faster the equations spun out of control, spiraling into infinite quantities and inconsistencies.
So, while the idea of unification by invoking higher dimensions was very attractive, it was abandoned, and for good reason. It made no testable predictions. Even if such a theory produced special solutions that did describe our world, there were, as noted, many more that did not. And the few that did were unstable and could easily evolve into singularities, or into worlds quite unlike our own. And finally, they could not be made consistent with quantum theory. Keep these reasons in mind—because, again, the success or failure of the newer proposals for unification, such as string theory, depend on whether they can solve these very problems.
By the time I began my study of physics in the early 1970s, the idea of unifying gravity with the other forces was as dead as the idea of continuous matter. It was a lesson in the foolishness of once great thinkers. Ernst Mach didn’t believe in atoms, James Clerk Maxwell believed in the aether, and Albert Einstein searched for a unified-field theory. Life is tough.
4
Unification Becomes a Science
AFTER THE IDEA of unifying all four fundam
ental forces by inventing new dimensions failed, most theoretical physicists gave up on the idea of relating gravity to the other forces, a decision that made sense because gravity is so much weaker than the other three. Their attention was drawn instead to the zoo of elementary particles that the experimentalists were discovering in their particle accelerators. They searched the data for new principles that could at least unify all the different kinds of particles.
Ignoring gravity meant taking a step backward, to the understanding of space and time before Einstein’s general theory of relativity. This was a dangerous thing to do in the long run, as it meant working with ideas that had already been superseded. But there was also an advantage, in that this approach led to a great simplification of the problem. The chief lesson of general relativity was that there is no fixed-background geometry for space and time; ignoring this meant that you could simply choose the background. This sent us back toward a Newtonian point of view, in which particles and fields inhabit a fixed background of space and time—a background whose properties are fixed eternally. Thus, the theories that developed from ignoring gravity are background-dependent.
However, it was not necessary to go all the way back to Newton. One could work within the description of space and time given by Einstein’s 1905 special theory of relativity. According to it, the geometry of space is that given by Euclid, which many of us study in junior high school; however, space is mixed with time, in order to accommodate Einstein’s two postulates, the relativity of observers and the constancy of the speed of light. The theory cannot accommodate gravity, but it’s the right setting for Maxwell’s theory of the electric and magnetic fields.
Once quantum mechanics was fully formulated, the quantum theorists turned their attention to unifying electromagnetism with quantum theory. As the basic phenomena of electromagnetism are fields, the unification that would eventually result is called a quantum field theory. And because Einstein’s special theory of relativity is the right setting for electromagnetism, these theories can also be seen as unifications of quantum theory with special relativity.