The Trouble With Physics: The Rise of String Theory, The Fall of a Science, and What Comes Next

by Lee Smolin

  String theorists are aware of their dominant position in the physics world, and most seem to feel that it’s deserved—if the theory itself doesn’t justify it, the fact that so many talented people work on it certainly should. If you raise detailed questions about one of string theory’s claims with an expert, you risk being regarded, with faint puzzlement, as someone who has inexplicably chosen a path that precludes membership in the club. Of course, this isn’t true of the more open-minded string theorists—but there is a peculiar tightening of the face muscles that I’ve seen too often to ignore, and it usually happens when a young string theorist suddenly realizes that he or she is talking to someone who does not share all the assumptions of the clan.4

  Another hallmark of string theory is that, unlike other fields of physics, there is a clear distinction between string theorists and non–string theorists. You may have written several papers on string theory, but that does not necessarily mean you will be considered by string theorists to be one of them. At first I found this puzzling. I was pursuing an old strategy of working on different approaches to try to learn from each what I could. I also initially saw much of what I did, even the work on quantum gravity, as addressing an important open issue in string theory, which was how to make it a background-independent formulation. Eventually some friends explained to me that to be considered part of the string theory community—and hence to have any hope of making a mark there—you had to work not just on string theory but on the particular problems that string theorists were then preoccupied with. I don’t think it occurred to my friends that doing so might compromise my judgment or impinge on my academic freedom.

  I have a broad range of interests, and I’ve always gone to conferences in fields outside my own. But only at string theory conferences have people come up to me and asked, “What are you doing here?” If I explained that I was working on string theory and wanted to see what other people were doing, they would say, brows quizzically furrowed, “But aren’t you that loop guy?” No one at a conference on astrophysics, cosmology, biophysics, or postmodernism has ever asked me what I was doing there. At one string theory conference, a leading string theorist sat down, offered his hand, and said, “Welcome home!” Another said, “It’s so nice to see you here! We’ve been worried about you.”

  In any given year, there are no more than two or three areas in string theory being intensively investigated. These change from year to year, and the fashions can be tracked by looking at the titles of talks at annual major string theory conferences. Often at least two-thirds of the talks concern one or two directions that were not strongly represented two years previously and will be almost absent from the conference two years later. Young people are very aware that a successful career requires following two or more of these fads in quick succession, just long enough to get a good postdoc and then a good assistant professorship. If you talk about this with the leaders of string theory, as I have from time to time, you discover that they genuinely believe that concentrating the efforts of a large community of very bright people will lead to faster progress than encouraging colleagues to think independently and pursue a variety of directions.

  This monolithic and (as it was called by one senior string theorist) “disciplined” approach has had three unfortunate consequences. First, problems that cannot be solved in two or three years are dropped and often never revisited. The reason is simple and brutal: Young string theorists who do not quickly give up their hard-won specialization in a no-longer-trendy area and switch to the new direction sometimes find themselves without academic positions. Second, the field continues to be driven by the ideas and research agendas of a few people who are now quite senior. In the past decade, only two young string theorists, Juan Maldacena and Raphael Bousso, have made discoveries that changed the direction of the field. This stands in contrast to many other fields in physics, where the majority of new ideas and directions come from people in their twenties and thirties. Third, string theory uses the talents and labor of the large number of people in its community inefficiently. There is much duplication of effort, while many potentially important ideas go unexplored. This narrowing of avenues is evident to anyone who sits on university committees that choose postdocs. In areas such as cosmology, quantum information theory, or quantum gravity, there are as many research proposals as candidates, and often ideas that no one has heard before. In the string theory pool, you tend to encounter the same two or three research proposals over and over again.

  Of course, the young people know what they’re doing. I’ve had many years of experience on such committees, and I’ve found that with a few exceptions, the standards used by string theorists in evaluating their applicants are different from those in other fields. The ability to do mathematically clever work on problems of current interest is, as far as I can tell, valued over the invention of original ideas. Someone who had published papers only with the leading senior scientists and whose research proposal showed little evidence of independent judgment or originality would probably not be offered a position in a leading center of quantum gravity, but it seems the surest route to a postdoc in the leading string theory centers. The kind of applicant that excites me—someone with single-authored papers describing surprising insights and risky new ideas—leaves my string theory friends cold.

  In other communities I spend time in, such as quantum gravity and cosmology, there is a diversity of views toward the open problems. If you talk to five different experts, young and old, you will get five different takes on where a subject is headed. Except for recent arguments over the landscape and the anthropic principle, string theorists have maintained a remarkable uniformity of views. One hears the same thing, sometimes in the same words, from different people.

  I know some young string theorists who will object to this characterization. They insist that there is a wide range of views within the community—a range that outsiders just aren’t privy to. This is good to know, but what people say privately to their friends is not the point. Indeed, if that wider range of views is expressed privately rather than publicly, it suggests that there is a hierarchy controlling the conversation—and the research agenda.

  The deliberate narrowing of the research agenda by the leaders of string theory is deplorable not just in principle but also because it has almost certainly led to slower progress. We know this because of the number of ideas that became important for the field many years after they were first proposed. For example, the discovery that string theory is composed of a vast collection of theories was first published by Andrew Strominger in 1986, but it was widely discussed by string theorists only after 2003, following the work of Renata Kallosh and her colleagues at Stanford.5 Here is a recent quote from Wolfgang Lerche, a well-known string theorist at CERN.

  Well, what I find irritating is that these ideas are out since the mid-80s; in one paper on 4d string constructions a crude estimate of the minimal number of string vacua was made, to the order 101500; this work had been ignored (because it didn’t fit into the philosophy at the time) by the same people who now re-“invent” the landscape, appear in journals in this context and even seem to write books about it. . . . [T]he whole discussion could (and in fact should) have been [sic] taken place in 1986/87. The main thing what [sic] has changed since then is the mind of certain people, and what you now see is the Stanford propaganda machine working at its fullest.6

  My own proposal that string theory had to be regarded as a landscape of theories was first published in 1992 and was also ignored.7 This is not an isolated example. Two eleven-dimensional supersymmetric theories were invented before the first superstring revolution in 1984 but were ignored until they were revived in the second one, more than a decade later. These were eleven-dimensional supergravity and the eleven-dimensional supermembrane. Between 1984 and 1995, a small number of theorists worked on these theories, but they were pushed to the margins of the string theory community. I can recall several derisive references by American string theorists to thos
e “European supergravity fanatics.” After 1995, these theories were conjectured to be unified with string theory in M-theory, and those who had labored on them were welcomed back into the string community. Obviously, progress would have been faster had these ideas not been excluded from consideration for so long.

  There are several ideas that might help string theory solve its key problems, but they have not been widely studied. One is an old idea that the number system called octonions is the key to a deep understanding of the relationship between supersymmetry and higher dimensions. Another is the requirement, which I have already emphasized, that the fundamental formulation of string theory or M-theory, so far unexplored, must be background-independent. During a panel discussion on “The Next Superstring Revolution” at the 2005 Strings conference, Stephen Shenker, the director of the Stanford Institute for Theoretical Physics, observed that it was likely to come from a topic outside string theory. If this is recognized by the field’s leaders, why do they not encourage young people to explore a wider range of subjects?

  The narrowness of the research agenda seems to be a result of the string community’s huge regard for the views of a few individuals. String theorists are the only scientists I’ve ever met who typically want to know what the senior people in the field, such as Edward Witten, think before expressing their own views. Of course Witten thinks clearly and deeply, but the point is that it is not good for any field if any one person’s views are taken too authoritatively. There is no scientist, not even Newton or Einstein, who was not wrong on a substantial number of issues they had strong views about. Many times, in discussion after a conference talk or in conversation, if a controversial issue comes up, someone invariably asks, “Well, what does Ed think?” This used to drive me to distraction, and occasionally I would let it show: “Look, when I want to know what Ed thinks, I ask him. I’m asking you what you think, because I’m interested in your opinion.”

  Noncommutative geometry is an example of a field that was ignored by string theorists until it was embraced by Witten. Alain Connes, its inventor, tells the following story:

  I went to Chicago in 1996 and gave a talk in the Physics Department. A well-known physicist was there, and he left the room before the talk was over. I didn’t meet this physicist again until two years later, when I gave the same talk in the Dirac Forum in Rutherford Laboratory, near Oxford. The same physicist was attending, this time looking very open and convinced. When he gave his talk later, he mentioned my talk quite positively. This was amazing to me, because it was the same talk, and I had not forgotten his previous reaction. So on the way back to Oxford, I was sitting next to him in the bus, and I asked him, “How can it be that you attended the same talk in Chicago and you left before the end, and now you really liked it?” The guy was not a beginner—he was in his forties. His answer was “Witten was seen reading your book in the library in Princeton!”8

  It should be said that this attitude is fading, probably in response to the current uproar surrounding the landscape. Until last year, I had hardly ever encountered an expression of doubt from a string theorist. Now I sometimes hear from young people that there is a “crisis” in string theory. “We have lost our leaders,” some of them will say. “Before this, it was always clear what the hot direction was, what people should be working on. Now there’s no real guidance,” or (to each other, nervously) “Is it true that Witten is no longer doing string theory?”

  Another facet of string theory that many find disturbing is what can only be described as the messianic tendency of some of its practitioners, especially some younger ones. For them, string theory has become a religion. Those of us who publish papers questioning results or claims of string theorists regularly receive e-mails whose mildest form of abuse is “Are you kidding?” or “Is this a joke?” Discussions of string theory’s “opponents” abound on Web sites and chat boards, where, even given the unbridled nature of such venues, the intelligence and professional competence of non–string theorists is questioned in remarkably unpleasant terms. It’s hard not to conclude that at least some string theorists have begun to see themselves as crusaders rather than scientists.

  Related to this swaggering is a tendency to read the evidence in the most optimistic way possible. My colleagues in quantum gravity commonly take a hard-nosed, often pessimistic, view of the prospects for solving open problems. Among loop quantum gravity theorists, I seem to be the great optimist. But my optimism pales compared with that of most string theorists. This is especially true when it comes to the big unanswered questions. As discussed, the “stringy” view of things is based on long-standing conjectures that are widely believed by string theorists but which have never been proved. Some string theorists believe them anyway. Optimism is good to a degree, but not when it results in outright misrepresentation. Unfortunately, the picture commonly offered to the general public in books and articles and TV shows—as well as to audiences of scientists—differs substantially from what a straightforward reading of the published results suggests. For example, in a review of Leonard Susskind’s 2005 book, The Cosmic Landscape, in a trade magazine for physicists, the reviewer, reflecting on the existence of many string theories, states:

  This problem is cured by M-theory, a unique, all-embracing theory that subsumes the five superstring theories by requiring 11 space-time dimensions and incorporating higher-dimensional extended objects called branes. Among the achievements of M-theory is the first microscopic explanation for the entropy of a black hole, first predicted in the 1970s by Hawking using macroscopic arguments. . . . The problem with M-theory is that although its equations may be unique, it has billions and billions of different solutions.9

  The most striking exaggeration here is the implication that M-theory exists as a precise theory rather than as a proposal and that it has definite equations, neither of which is true. The claim to explain black-hole entropy is (as noted in chapter 9) exaggerated, because the string theory results work only for special and atypical black holes.

  You can also find such distortions in Web pages aiming to introduce string theory to the public, such as the following:

  There is even a mode describing the graviton, the particle carrying the force of gravity, which is an important reason why String Theory has received so much attention. The point is that we can make sense of the interaction of two gravitons in String theory in a way we could not in QFT. There are no infinities! And gravity is not something we put in by hand. It has to be there in a theory of strings. So, the first great achievement of String Theory was to give a consistent theory of quantum gravity.10

  Those responsible for this particular Web page know that no one has proved there are “no infinities.” But they seem to be confident enough of the truth of the conjecture to present it as fact. Further on, they, too, bring up the issue of the five different superstring theories:

  And only then it was realized that those five string theories are actually islands on the same planet, not different ones! Thus there is an underlying theory of which all string theories are only different aspects. This was called M-theory. The M might stand for Mother of all theories or Mystery, because the planet we call M-theory is still largely unexplored.

  This clearly states that “there is an underlying theory,” even if the last line concedes that M-theory is “still largely unexplored.” A member of the public would conclude from this that there is a theory called M-theory, with the usual attributes of a theory, which is a formulation in terms of precise principles and a representation by precise equations.11

  Many review papers and talks make equally vague and imprecise statements about results. There is unfortunately a great deal of confusion about what has actually been achieved by string theory, along with a tendency to exaggerate results and minimize difficulties. When I’ve queried experts, I’ve been shocked to find that many string theorists are unable to give correct and detailed answers to questions about the status of key conjectures, such as perturbative finiteness, S-duality, t
he Maldacena conjecture, or M-theory.

  I understand that this a strong charge to bring, so let me illustrate it with an example. One of the basic claims made for string theory is that it is a finite theory. This means that the answers it gives to all physically sensible questions involve finite numbers. Clearly, any viable theory must provide finite answers to questions about probabilities, or finite predictions for the mass or energy of some particle or for the strength of some force. However, proposed quantum theories of fundamental forces frequently fail to do so. Indeed, of the huge number of different theories of forces consistent with the principles of relativity, all but a small number produce infinite answers to these kinds of questions. This is especially true of quantum theories of gravity. Many once promising approaches were abandoned because they failed to give finite answers. The few exceptions include string theory and loop quantum gravity.

  As I discussed in chapter 12, the claim that string theory gives finite answers is couched in a certain approximation scheme called string perturbation theory. This technique gives an infinite set of approximations to the motions and interactions of strings in a given setting. We speak of the first approximation, the second approximation, the seventeenth approximation, the 100 millionth approximation, all the way up to infinity. To prove a theory finite in such a scheme, one must prove that each one of the infinite number of terms is finite. This is hard to do, but it’s not impossible. It was done, for example, for the quantum theory of electromagnetism, or QED, in the late 1940s and 1950s. This was the triumph of Richard Feynman, Freeman Dyson, and their generation. The standard model of particle physics was proved finite in 1971 by Gerard ’t Hooft.