The Trouble With Physics: The Rise of String Theory, The Fall of a Science, and What Comes Next

by Lee Smolin

  Good scientists expect that their students will exceed them. Although the academic system gives a successful scientist many reasons to believe in his or her own authority, any good scientist knows that the minute you succumb to believing that you know more than your best students, you cease to be a scientist.

  The scientific community is thus both an ethical and an imaginative community.

  What should be abundantly clear from this description is that controversy is essential for the progress of science. My first principle says that when we are forced to reach a consensus by the evidence, we should do so. But my second principle says that until the evidence forces consensus, we should encourage a wide diversity of viewpoints. This is good for science—a point that Feyerabend made often, and I believe correctly. Science proceeds fastest when there are competing theories. The older, naïve view is that theories are put forward one at a time and tested against the data. This fails to take into account the extent to which the theoretical ideas we have influence which experiments we do and how we interpret them. If only one theory is contemplated at a time, we are likely to get stuck in intellectual traps created by that theory. The only way out is if different theories compete to explain the same evidence.

  Feyerabend argued that even in cases where there is a widely accepted theory that agrees with all the evidence, it is still necessary to invent competing theories in order for science to progress. This is because experiments that contradict the established view are most likely to be suggested by a competing theory and perhaps would not even have been conceived were there not a competing theory. So competing theories give rise to experimental anomalies as often as the reverse.

  Therefore Feyerabend insisted that scientists should never agree, unless they are forced to. When scientists come to agreement too soon, before they are compelled to by the evidence, science is in danger. We then have to ask what influenced them to come to the premature conclusion. As they are only human, the answer to this will likely be the same factors that cause people to agree about all sorts of things that don’t rely on evidence, from religious beliefs to fashion to trends in popular culture.

  So the question is this: Do we want scientists to come to agreement because they want to be liked or seen as brilliant by other scientists, or because everyone they know thinks the same thing, or because they like to be on the winning team? Most people are tempted to agree with other people for motives such as these. There is no reason that scientists should be immune from them, being human, after all.

  Still, we must fight those urges if we want to keep science alive. We must encourage the opposite, which is to disagree as much as the evidence permits. Given how much humans need to be liked, to fit in, to be part of the winning team, we must make it clear that when we succumb to these needs, we are letting science down.

  There are other reasons that a healthy scientific community should encourage disagreement. Science moves forward when we are forced to agree with something unexpected. If we think we know the answer, we will try to make every result fit that preconceived idea. It is controversy that keeps science alive, keeps it moving. In an atmosphere filled with controversy over rival views, sociological forces are not enough to bring people into agreement. So on those rare occasions when we do come to consensus on something, it is because we have no choice. The evidence forces us to do so, even if we don’t like it. That is why progress in science is real.

  There are several obvious objections to this characterization of science. First, there are clearly violations by members of the community of the ethic I have just described. Scientists often do exaggerate and distort the evidence. Age, status, fashion, peer pressure, all do play a role in the workings of the scientific community. Some research programs do succeed in gathering adherents and resources beyond what the evidence supports, while other research programs that ultimately bear fruit are suppressed by sociological forces.

  But I would suggest that enough scientists adhere to enough of the ethic that in the long run progress continues to be made, despite the fact that time and resources are wasted in the promotion and defense of orthodox and fashionable ideas that later turn out to be wrong. The role of time must be emphasized. Whatever may happen in the short run, over decades evidence almost always accumulates that settles contrary claims by consensus, without regard to fashion.

  Another possible objection is that the characterization I’ve given is logically incomplete. I do not offer any criteria for determining which crafts are necessary to master. But I think this is best done by the communities themselves, over many generations. There is no way that Newton or Darwin could have predicted the range of tools and procedures used now.

  Adherence to the shared ethic is never perfect, so there is always room for improvement in the practice of science. This seems especially true today, when fashion appears to be playing too large a role, at least in physics. You know this is happening whenever there are bright young recent PhDs who tell you privately that they would rather be doing X but are doing Y because that is the direction or technique championed by powerful older people, and they thus feel the need to do Y to get funding or a job. Of course, in science as in other areas, there are always a few who choose to do X in spite of the clear evidence that the doers of Y are better rewarded in the short term. Among them are the people who will most likely lead the next generation. Thus the progress of science may be slowed by orthodoxy and fashion, but as long as there is room for those who do X instead of Y, it cannot be stopped completely.

  All this is to say that like everything else that human beings do, success in science is to a large extent driven by courage and character. While the progress of science relies on the possibility of achieving consensus in the long term, the decisions an individual scientist makes as to what to do, and how to evaluate the evidence, are always based on incomplete information. Science progresses because it is built on an ethic recognizing that in the face of incomplete information we are all equal. No one can predict with certainty whether an approach will lead to definite progress or years of wasted work. All we can do is train students in the crafts that experience has shown to lead most often to reliable conclusions. After that, we must leave them free to follow their own hunches and we must make time to listen to them when they report back. As long as the community continually opens up opportunities for new ideas and points of view and adheres to the ethic that in the end we require consensus based on rational argument from evidence available to all, science will eventually succeed.

  The task of forming the community of science will never be finished. It will always be necessary to fight off the dominance of orthodoxy, fashion, age, and status. There will always be temptations to take the easy way, to sign up with the team that seems to be winning rather than try to understand a problem afresh. At its finest, the scientific community takes advantage of our best impulses and desires while protecting us from our worst. The community works in part by harnessing the arrogance and ambition we each in some degree bring to the search. Richard Feynman may have said it best: Science is the organized skepticism in the reliability of expert opinion.8

  18

  Seers and Craftspeople

  PERHAPS THERE IS something wrong with the way we are going about trying to make a revolution in physics. I argued in chapter 17 that science is a human institution, subject to human foibles—and fragile, because it depends as much on group ethics as on individual ethics. It can break down, and I believe it is doing so now.

  A community often finds it is constrained to think in a particular way because of how it is organized. An important organizational issue is: Are we recognizing and rewarding the right kind of physics, and the right kind of physicist, in order to solve the problem at hand? Its cognitive counterpart is: Are we asking the right questions?

  The one thing everyone who cares about fundamental physics seems to agree on is that new ideas are needed. From the most skeptical critics to the most strenuous advocates of string theory, you hear the sam
e thing: We are missing something big. It was the perception of the need for something new that led the organizers of the 2005 annual Strings conference to offer a session on “The Next Superstring Revolution.” And although there is currently more confidence among practitioners in other fields, every physicist I know will agree that probably at least one big idea is missing.

  How do we find that missing idea? Clearly, someone has to either recognize a wrong assumption we have all been making or ask a new question, so that’s the sort of person we need in order to ensure the future of fundamental physics. The organizational issue is then clear: Do we have a system that allows someone capable of ferreting out that wrong assumption or asking that right question into the community of people we support and (equally important) listen to? Do we embrace the creative rebels with this rare talent, or do we exclude them?

  It goes without saying that people who are good at asking genuinely novel but relevant questions are rare, and that the ability to look at the state of a technical field and see a hidden assumption or a new avenue of research is a skill quite distinct from the workaday skills that are a prerequisite for joining the physics community. It is one thing to be a craftsperson, highly skilled in the practice of one’s craft. It is quite another thing to be a seer.

  This distinction does not mean that the seer is not a highly trained scientist. The seer must know the subject thoroughly, be able to work with the tools of the trade, and communicate convincingly in its language. Yet the seer need not be the most technically proficient of physicists. History demonstrates that the kind of person who becomes a seer is sometimes mediocre when compared with the mathematically clever scientists who excel at problem solving. The prime example is Einstein, who apparently couldn’t get a decent job as a scientist when he was young. He was slow in argument, easily confused; others were much better at mathematics. Einstein himself is said to have remarked, “It’s not that I’m so smart. It’s just that I stay with problems longer.”1 Niels Bohr was an even more extreme case. Mara Beller, a historian who has studied his work in detail, points out that there was not a single calculation in his research notebooks, which were all verbal argument and pictures.2 Louis de Broglie made the astounding suggestion that if light is a particle as well as a wave, perhaps electrons and other particles also behave as waves. He proposed this in a 1924 PhD thesis that did not impress his examiners and would have failed without the endorsement of Einstein. As far as I know, he never did anything nearly as influential in physics again. There is only one person I can think of who was both a visionary and the best mathematician of his day: Isaac Newton; indeed, almost everything about Newton is singular and inexplicable.

  As noted in the preceding chapter, Thomas Kuhn made a distinction between “normal science” and scientific revolutions. Normal science is based on a paradigm, which is a well-defined practice with a fixed theory and a fixed body of questions, experimental methods, and calculational techniques. A scientific revolution happens when the paradigm breaks down, which is to say, when the theory it is based on fails to predict or explain the results of the experiments.3 I don’t think science always works this way, but there are certainly normal and revolutionary periods, and science is done differently during them. The point is that different kinds of people are important in normal and revolutionary science. In the normal periods, you need only people who, regardless of their degree of imagination (which may well be high), are really good at working with the technical tools—let us call them master craftspeople. During revolutionary periods, you need seers, who can peer ahead into the darkness.

  Master craftspeople and seers come to science for different reasons. Master craftspeople go into science because, for the most part, they have discovered in school that they’re good at it. They are usually the best students in their math and physics classes from junior high school all the way up to graduate school, where they finally meet their peers. They have always been able to solve math problems faster and more accurately than their classmates, so problem solving is what they tend to value in other scientists.

  Seers are very different. They are dreamers. They go into science because they have questions about the nature of existence that their schoolbooks don’t answer. If they weren’t scientists, they might be artists or writers or they might end up in divinity school. It is only to be expected that members of these two groups misunderstand and mistrust each other.

  A common complaint of the seers is that the standard education in physics ignores the historical and philosophical context in which science develops. As Einstein said in a letter to a young physicist who had been thwarted in his attempts to add philosophy to his physics courses:

  I fully agree with you about the significance and educational value of methodology as well as history and philosophy of science. So many people today—and even professional scientists—seem to me like someone who has seen thousands of trees but has never seen a forest. A knowledge of the historical and philosophical background gives that kind of independence from prejudices of his generation from which most scientists are suffering. This independence created by philosophical insight is—in my opinion—the mark of distinction between a mere artisan or specialist and a real seeker after truth.4

  Of course, some people are mixtures of both. No one makes it through graduate school who is not highly competent on the technical side. But the majority of theoretical physicists I know fall into one or the other group. What about me? I think of myself as a would-be seer who fortunately was good enough at my craft to contribute occasionally to the problem solving.

  When I first encountered Kuhn’s categories of revolutionary and normal science as an undergraduate, I was confused, because I couldn’t tell which period we were in. If I looked at the kinds of questions that remained open, we were clearly partway through a revolution. But if I looked at how the people around me worked, we were just as obviously doing normal science. There was a paradigm, which was the standard model of particle physics and the experimental practices that had confirmed it, and it was normally progressing.

  Now I understand that the confusion was a clue to the crisis I have been exploring in this book. We are indeed in a revolutionary period, but we are trying to get out of it using the inadequate tools and organization of normal science.

  This, then, is my basic hypothesis about the last twenty-five years of physics. There can be no doubt that we are in a revolutionary period. We are horribly stuck, and we need real seers, and badly. But it has been a long time since seers were needed. We had a few monumental visionaries at the beginning of the twentieth century: Einstein above all, but also Bohr, Schrödinger, Heisenberg, and a few others. They failed to complete the revolution they started, but they created partially successful theories—quantum mechanics and general relativity—for us to build on. The development of these theories required a lot of hard technical work, and so for several generations physics was “normal science” and was dominated by master craftspeople. Indeed, the transition from dominance by Europeans to dominance by Americans, which took place in the 1940s, was very much the triumph of master craftspeople over seers. As noted, it brought about a reversal in the style of theoretical physics, from the reflective foundational mode of Einstein and his peers to the pragmatic, aggressive mode that gave us the standard model.

  When I learned physics in the 1970s, it was almost as if we were being taught to look down on people who thought about foundational problems. When we asked about the foundational issues in quantum theory, we were told that no one fully understood them but that concern with them was no longer part of science. The job was to take quantum mechanics as given and apply it to new problems. The spirit was pragmatic; “Shut up and calculate” was the mantra. People who couldn’t let go of their misgivings over the meaning of quantum theory were regarded as losers who couldn’t do the work.

  As someone who came into physics from reading Einstein’s philosophical musings, I couldn’t accept that reasoning, but the message was clear, and I
followed it as best as I could. You could make a career only by working within quantum theory as given, not by questioning it. A fortunate circumstance won me some time at the Institute for Advanced Study in Princeton, but there was no memory there of Einstein’s way of doing science—just an empty bronze likeness gazing silently out over the library.

  But the revolution was not finished. The standard model of particle physics was certainly the triumph of this pragmatic style of doing physics, but its triumph seems now to have also marked its limit. The standard model, and just possibly inflation, is about as far as we could go with normal science. Since then, we have been mired, because what we need is a return to a revolutionary kind of science. Once again, we need a few seers. The problem is that there are now very few around, as a result of science having been done so long in a way that rarely recognized and barely tolerated them.

  Between the early twentieth century and the last quarter century, science—and the academy in general—has become much more organized and professionalized. This means that the practice of normal science has been enshrined as the single model of good science. Even if everyone can see that a revolution is necessary, the most powerful parts of our community have forgotten how to make one.

  We have been trying to do so with structures and styles of research best suited to normal science. The paradoxical situation of string theory—so much promise, so little fulfillment—is exactly what you get when a lot of highly trained master craftspeople try to do the work of seers.

  I’m sure that some string theorists will object to this characterization. Certainly they work on fundamental problems of physics, and all their work is aimed at discovering new laws. Why are string theorists not seers? Aren’t wormholes, higher dimensions, and multiple universes imaginative ideas? Yes, of course, but this is not the point. The question is: What is the context, and what are the ideas about? Hidden dimensions and wormholes are hardly a novelty more than three quarters of a century after Kaluza and Klein. Nor does it take much foresight or courage to think about these things when hundreds of other people are thinking the same thoughts.