The Trouble With Physics: The Rise of String Theory, The Fall of a Science, and What Comes Next

by Lee Smolin

  My point here is not to criticize string theory; the string theorists are just behaving in ways that members of any dominant research program would. The problem is that we have a system of decision making in academia that is far too vulnerable to takeover by an aggressively promoted research program regardless of the results. The same system once worked against string theorists. As journalist Gary Taubes notes,

  On August 4, 1985, I sat in the cantina at CERN drinking beer with Alvaro de Rujula. . . . De Rujula predicted that 90 percent of the theorists would work on superstrings and the connection with supersymmetry, because it was fashionable. When he intimated that this was not a healthy state, I asked him what he would prefer to work on. Rather than answer directly, he digressed. “It must be remembered,” de Rujula told me, “that the two people most responsible for the development of superstrings, that is to say Green and Schwarz, have spent ten to fifteen years systematically working on something that was not fashionable. In fact they were ridiculed by people for their stubborn adherence to it. So when people come and attempt to convince you that one must work on the most fashionable subject, it is pertinent to remember that the great steps are always made by those who don’t work on the most fashionable subject.”4

  I once discussed this situation with a department chair at a major university who lamented that he had not been able to persuade his colleagues to hire John Schwarz in the early 1980s. “They agreed that he was an incredibly smart theorist,” he said, “but I couldn’t convince them, because they said he was too obsessed and he would probably never work on anything else but string theory. These days I can’t convince my colleagues to hire anyone who is not a string theorist.”

  I also recall discussing these problems with Abraham Pais, the particle physicist and biographer of Bohr and Einstein. We used to meet sometimes for lunch at Rockefeller University in New York, where he had been a professor and where I once had an office. “There is nothing you can do,” Pais told me. “In my day, too, they were all bastards!”

  I think Pais was off the point. This is not about people, it is about how we have structured academic decision making. It is about ensuring that the kinds of scientists needed to advance science have the right opportunities.

  This system has another crucial consequence for the crisis in physics: People with impressive technical skills and no ideas are chosen over people with ideas of their own partly because there is simply no way to rank young people who think for themselves. The system is set up not just to do normal science but to ensure that normal science is what is done. This was made explicit to me when I applied for my first job out of graduate school. One day, as we were waiting for the results of our applications, a friend came to me looking very worried. A senior colleague had asked him to tell me that I was unlikely to get any jobs, because it was impossible to compare me with other people. If I wanted a career, I had to stop working on my own ideas and work on what other people were doing, because only then could they rank me against my peers.

  I can’t remember what I thought about this or why it didn’t make me crazy with worry. I did have to wait two months longer than everyone else to get a job offer and this was not fun. I was already thinking about what I could do to support myself that wouldn’t take too much time from my research. But then I got lucky. The theoretical physics institute in Santa Barbara had just opened, and it had a program in quantum gravity. So my career didn’t come to an end.

  But it is only now that I understand what was really happening. No one was consciously acting unethically. My friend and his mentor had my best interests, as they understood them, in mind. But here is how a sociologist would describe what happened. The older colleague who gave my friend the message had pioneered a new research program that required challenging calculations. That program needed clever, fast young theorists. I was being told that if I worked on his program, he would give me the gift of a career. It was the simplest and oldest trade in the world: A worker is given the chance to survive in exchange for his labor.

  There are many ways the trade is offered, with the takers rewarded and the rebels—those who prefer their own ideas to the ideas of their elders—punished. My friend Carlo Rovelli wanted a job in Rome. He was told to go see a certain professor, who was very friendly and explained to Carlo all about the exciting research program he and his group were pursuing. Carlo thanked him for the insightful discussion and reciprocated by describing his own research program to the professor. The interview was terminated shortly thereafter, and Carlo never got the expected job offer. I had to explain to him what had happened. He was, as we all were once, still naïve enough to think that people were rewarded for having good ideas of their own.

  In fact, what it took for Carlo to get a job offer from the University of Rome was to become the leading European scientist in his field. Only then—once he had made an influential career elsewhere, only after hundreds of people around the world had begun to work on his ideas—would the leading professors in Rome be willing to listen to the ideas that he had tried, as a new PhD, to bring to them.

  You may wonder how it was that Carlo got a job in the first place. I’ll tell you. At that time, in the late 1980s, the field of general relativity was dominated by a few older people who were students of students of Einstein, and they held strongly to the view that the young people with the best and most independent ideas should be encouraged. They led what was called then the relativity community, which had research groups in about a dozen universities in the United States. As a field they were hardly dominant, but they did control a small number of positions, perhaps one new faculty position every two or three years. Carlo was a postdoc in Rome, but due to some bureaucratic problems, his job had never been made official and he had never been paid. Every month he was told that after one more meeting or one more bit of paperwork, he would get a check. After a year and a half of this, he called his friends in the United States and said that although he had not wanted to leave Italy, he was fed up. Were there any jobs available in the United States? As it happened, one of the centers of relativity was looking for an assistant professor, and when they heard he might apply, they flew him over and rushed the appointment through in a matter of weeks. It is worth mentioning that no one at that center worked on quantum gravity—they hired Carlo because he had proved to have original and important ideas in that field.

  Could this happen today? Not likely, because now even the field of relativity is dominated by a large research program with a precise agenda set by senior scientists. This has to do with experimental gravitational-wave astronomy and with the hope (still a hope after many years) of doing computer calculations to predict what those experiments should see. These days, a young specialist in general relativity or quantum gravity who is not primarily working on these problems is unlikely to get hired anywhere in the United States.

  No matter what the field, a taste of success is often all that is required to turn erstwhile rebels into conservative guardians of their research programs. More than once I’ve been brought up short by people in my own field of quantum gravity, for supporting the hiring of someone from outside the field who offers new ideas over the hiring of a technically impressive candidate who is working on narrow issues that advance the existing research.

  There are really two issues here, and it is important to separate them. One is the dominance of decision making by senior scientists, who frequently use their power to support research programs they invented when they were young and imaginative. The second is the kind of scientist that universities are interested in and able to hire. Do they hire people doing work that everyone in a particular field understands and can judge? Or are they willing to hire people who invent their own directions, which are likely to be difficult to grasp?

  This is related to the issue of risk. Good scientists tend to elicit two kinds of responses from referees. Normal, low-risk scientists generally elicit uniform responses; everyone feels the same about them. High-risk scientists and visionaries ten
d to provoke strongly polarized reactions. There are some people who believe in them deeply and communicate it strongly. Others are highly critical.

  The same thing happens when students evaluate their teachers. There is a certain kind of good teacher that students do not feel neutral about. Some love her, or him, and will say, “This is the best teacher I’ve ever had; this is why I went to college.” But others are angry and resentful and hold back nothing on the evaluation form. If you average the scores—reducing the data to a single number, as is often done in deciding the size of professors’ raises and chances for tenure—you miss this crucial fact.

  Over the years, I’ve noticed that a polarized distribution of responses is a strong predictor of future success and influence as a scientist. If some people think X is the future of science and others think X is a disaster, this may mean that X is the real thing, someone who aggressively pushes his or her own ideas and has the talent and perseverance to back them up. An environment that embraces risk takers will welcome such people, but a risk-averse environment will shun them.

  As far as U.S. research universities are concerned, the basic fact is that people with a polarized set of recommendations frequently don’t get hired. Although I have observed this only in my field, it may be generally true. Consider the following scientists, all of them widely admired for the boldness and originality of their contributions to our understanding of evolution: Per Bak, Stuart Kauffman, Lynn Margulis, Maya Paczuski, Robert Trivers. Two of them are physicists who have studied mathematical models of natural selection, the others are leading evolutionary theorists. None made their careers at the most elite universities. When I was younger, I used to wonder why. After a while, I realized that they were too intellectually independent. Their profiles were too bimodal: If many admired them, there were also powerfully placed academics who were skeptical of them. And indeed, it is often the case that the kinds of people who originate ideas are not without faults when measured against the criteria that normal scientists use to judge excellence. They can be too bold. They can be sloppy about the details and unimpressive technically. These criticisms often apply to original thinkers whose curiosity and independence led them into fields they were not trained in. No matter how original and useful their insights, their work will be technically unimpressive to specialists in the domain.

  It is also true that some of these creative and original scientists are not easy to get along with. They can be impatient. They express themselves too directly when they disagree with you, and they lack the good manners that come easily to those for whom fitting in is more important than being right. Having known several such “difficult” people, I suspect they are angry for the same reason that very smart women in science are sometimes angry: They have suffered a lifetime of being made to feel marginal.

  These kinds of problems certainly affected the career of Per Bak, who, tragically, died of cancer a few years ago, at age fifty-four. He had the rare asset of having written papers in several fields outside his specialty, from economics to cosmology to biology. That should have made him a hot property, in demand at the best universities, but the opposite was the case, because he was not shy to point out that his way of approaching a problem led to insights the experts had missed. He would have had a much better career had he applied his creativity in only one field, but then he wouldn’t have been Per Bak.

  You might wonder why all the smart people who have become department heads and deans haven’t figured all this out and used this knowledge for the benefit of their university. Of course, a few have, and those few will hire such people. Most of the jobs that have opened up in the last decades in non-string approaches to quantum gravity in the United States came about because a chair had a rare opportunity to hire in a field not already represented at his university. Freed from the usual department politics, he made a cost-benefit calculation that convinced him that by hiring in an undersupported field he could instantly obtain a top-ranked group that would raise the status of his department.

  Indeed, the problems we are talking about affect all science, and a few influential senior scientists in other fields have expressed their concern. Bruce Alberts is a biologist and past president of the National Academy of Sciences, the most prestigious and influential organization of scientists in the United States. In his presidential address to the National Academy in April 2003, he noted:

  We have developed an incentive system for young scientists that is much too risk-averse. In many ways, we are our own worst enemies. The study sections that we establish to review requests for grant funds are composed of peers who claim that they admire scientific risk-taking, but who generally invest in safe science when allocating resources. The damping effect on innovation is enormous, because our research universities look for assistant professors who can be assured of grant funding when they select new faculty appointments. This helps to explain why so many of our best young people are doing “me too” science.

  He went on to describe a trend in which, over the ten years beginning in 1991, the proportion of National Institutes of Health funding going to researchers under thirty-five more than halved, while that going to those over fifty-five increased by more than 50 percent. He bemoaned the result, because it was greatly decreasing the intellectual independence of the younger researchers:

  Many of my colleagues and I were awarded our first independent funding when we were under thirty years old. We did not have preliminary results, because we were trying something completely new. [Now] almost no one finds it possible to start an independent scientific career under the age of thirty-five. Moreover, whereas in 1991 one-third of the principal investigators with NIH funds were under forty, by the year 2002 this fraction had dropped to one-sixth. Even the most talented of our young people seem to be forced to endure several years of rejected grant applications before they finally acquire enough “preliminary data” to assure the reviewers that they are likely to accomplish their stated goals.

  Why, if the problem is so obvious that it worries the most prominent leaders of American science, is nothing done about it? This puzzled me for a long time. I now realize that the competition to get and keep academic positions is about more than merit. The system tries to choose the best, most productive people, and to some extent it does that. But there are other agendas, and it would be naïve to ignore them. Equally important, these decisions are about establishing and enforcing a consensus within each field.

  Hiring is not the only means of accomplishing this consensus. Everything I’ve said about hiring is true also of the panels that evaluate grant applications. It also holds for tenure evaluations. These matters are related, because you cannot get tenure in science at a U.S. research university if you haven’t been successful in getting grants, and you can’t get hired unless there is a likelihood that you will get grants.

  As it happened, a few months before I first figured that out, I had been asked to write a piece on another subject for the Chronicle of Higher Education, which is a trade journal for university administrators. I wrote to the editors and proposed instead a piece on threats to academic freedom coming from the dominance of popular research programs. They were willing to look at it, but they rejected it as soon as they read my draft. I was outraged: They were suppressing dissent! So I wrote them an unusually (for me) unpleasant e-mail questioning their decision. They responded right away, telling me that the problem was not that the piece was radical—quite the opposite. Everything in it was well known and had been thoroughly aired, within the social sciences and the humanities. They sent me a pile of articles they had published in past years about power relations in academic decision making. I read them and quickly realized that it was only scientists who seem to be ignorant of these issues.

  Obviously, there are good reasons to have tenure. To a limited extent, it protects scientists who are original and independent from being fired and replaced by young careerists following the latest intellectual fad. But we pay a severe price for the tenure system: Too
much job security, too much power, and too little accountability for older people. Too little job security, too little power, and too much accountability for younger people in the prime of their creative, risk-taking years.

  Although tenure protects people who are intellectually independent, it does not produce them. I’ve heard many colleagues say they are working on what is trendy in order to get tenure, after which they will do what they really want. But it doesn’t seem ever to turn out that way. I know of only one case where that happened. In the others, it seems that if these people did not have enough courage and independence to work on what they wanted when they were worried about tenure, they did not suddenly gain courage and independence when contemplating what the panel reviewing their grant would decide. It does not help to have a system that protects tenured professors’ intellectual independence if the same system makes it unlikely that people with that independence will ever get tenure.

  In fact, professors with tenure who lose their grant funding because of having switched to a more risky area can quickly find themselves in hot water. They cannot be fired, but they can be pressured with threats of heavy teaching and salary cuts to either go back to their low-risk, well-funded work or take early retirement.

  Here is what Isador Singer, a gifted mathematics professor at MIT, said recently about the state of his discipline:

  I observe a trend towards early specialization driven by economic considerations. You must show early promise to get good letters of recommendations to get good first jobs. You can’t afford to branch out until you have established yourself and have a secure position. The realities of life force a narrowness in perspective that is not inherent to mathematics. We can counter too much specialization with new resources that would give young people more freedom than they presently have, freedom to explore mathematics more broadly, or to explore connections with other subjects, like biology these days where there is lots to be discovered.