The Quest: Energy, Security, and the Remaking of the Modern World

by Daniel Yergin

  But so, by the way, was its electricity demand. Although the implications were little understood, the impact would soon not only shake California but would be felt throughout the United States and in the rest of the world. It would also starkly dramatize fundamental realities of electric power.

  By the 1990s the regulatory bargain that had long been the foundation of the electrical power business in the United States was more than half a century old. Electric power prices were established not in the marketplace but rather by a state’s public utility commission (PUC), in accord with the model originally promoted by Samuel Insull. They did so by allowing utilities to pass on, in their rates to consumers, the cost of service—that is, the cost of everything, including plants, fuel, and operations, plus an additional sum that was the permitted profit. The PUC would then decide how those costs were to be allocated in terms of the prices paid by the different classes of customers—residential, commercial, and industrial.

  On their side of the bargain, the utilities were required to provide reliable service, universally available, at reasonable cost. They would ensure that the lights stayed on. If the power went off because a storm had knocked down the power lines or a blizzard had disrupted the system, the linemen would be out as fast as their trucks could roll, and the utility would scramble to get the power back on. This was all based on the concept of natural monopoly. Competition was definitely not part of the bargain.

  RATE SHOCK

  But change was coming. For many years electricity prices in the United States had been declining dramatically—between 1934 and 1970, by an astonishing 86 percent. That was testament to the impact of scale, technology, and lower costs that came with higher volumes. But in the 1970s and 1980s, prices abruptly turned up: New power plants—whether nuclear or coal—were proving to be expensive, sometimes very expensive. Costs were also driven by the 1978 Public Utility Regulatory Policies Act (PURPA). That law had forced utilities to buy power at high “avoided” costs from small-size generators of renewable power—largely wind and small hydro plants.

  Avoided costs were a very interesting concept: It was an estimate of how much the same amount of power would cost were it generated from an oil- or gas-fired facility. It was not an actual price, but an expected price sometime in the future. These avoided costs were often pegged at stratospherically high anticipated oil prices. But in the 1980s, oil and gas prices had declined, meaning that PURPA avoided-cost power prices were far above actual market costs. All this meant that consumers, in many parts of the country, were hit by “rate shock”—steep rises in electricity rates, as the costs from new nuclear and coal plants, and from the PURPA machines, were passed on to them in their monthly bills.

  Residential consumers may have complained about their bills, but there was little they could really do, aside from being more careful in their use of electricity. For industries that used a good deal of electricity, rate shock hit their bottom line and made them less competitive against companies in lower-cost states. They needed to do something to bring down their power prices. Their answer was to promote what was variously called “deregulation” or “restructuring,” which would allow them to find a way to buy cheaper power from someone else rather than more expensive power from their local utility. In a historic shift, that would lead toward electric rates being determined in a marketplace, not by the PUC—that is, toward competition in what had heretofore been assumed to be a natural monopoly. Getting deregulation right, however, would not prove so easy for electric power. Even competitive markets are, after all, not exactly free. They depend, crucially, on the rules by which they operate.

  Deregulation was made even more compelling by the appearance of a shift in the fuel mix for electric power. As new nuclear plants came online they contributed a growing share of power generation—leveling out at 20 percent of supply nationally. But the big growth was in coal. In the fifteen years following the natural gas shortages of the mid-1970s, coal consumption in electric generation literally doubled and was responsible for about 55 percent of all electricity produced in the United States. Coal’s great advantage was that it was abundant and it was a domestic fuel.

  But natural gas too was now abundant, and it too was also domestic. It was a fuel well suited for the deregulated power business. The gas bubble, the long-lasting surplus of natural gas following its deregulation, made gas cheap. In the face of the changing economics, the prohibition on the use of natural gas in power generation was clearly irrational, and the ban was lifted. At the same time, a new generation of highly efficient combined-cycle gas turbines—based on engines designed for jets, combined with steam turbines that run on “rejected heat”—started to enter the market. Gas plants were much less costly to build than coal and nuclear power plants, they could be constructed more quickly, and natural gas was a cleaner fuel than coal.

  Thus electricity from a new gas-fired power plant was cheaper than that from a nuclear power plant that had been constructed in the 1970s—or, for that matter, a coal plant that was built in the 1980s. But the existing regulatory system did not easily allow buyers to get access to the lower-cost power. At least not yet.

  TOWARD MARKET

  Thinking about the role of governments and markets was, at that time, undergoing a decisive change around the world. Increased confidence in markets stimulated a movement toward deregulation and privatization. In the United States, financial services were deregulated in the 1970s, after which stockbrokers could offer lower rates to customers if they wanted to. The airline industry was also deregulated, a transformation championed by Senator Edward Kennedy, Senate staffer (and later Supreme Court justice) Stephen Breyer, and the regulatory economist Alfred Kahn. As a result, the federal government stopped regulating everything from the cost of airline tickets to the size of sandwiches that could be served on planes. And, as already observed, price controls on oil as well as natural gas were abandoned in the 1980s. This same shift was even more evident in other countries. State-owned companies in Western Europe were privatized; communism collapsed in the Soviet Union and Eastern Europe; and both China and India opened up to the world economy.2

  But what laid out the path for the United States was what happened in the United Kingdom. Of all the privatizations set in motion in Britain by Prime Minister Margaret Thatcher’s market revolution, the biggest was that of the Central Electricity Generating Board (CEGB). The British power industry had been nationalized after World War II to end wasteful fragmentation, modernize the industry, and give virtually everyone access to the benefits of electric power. All of this it had done. It was an engineering-driven organization whose mandate was “to keep the lights on no matter what the cost.” The downside was that, in the process, it was racking up big losses and was in constant turmoil with trade unions.

  Beginning in 1990, the British industry was privatized. “Again and again I insisted that whatever structure we created must provide genuine competition,” said Prime Minister Thatcher. The government broke the generating part of the CEGB into three private companies. These generation companies competed both among themselves and against new independent generating companies to sell electricity into the wholesale market. As for the retail side of the market, the government converted “area boards,” which distributed electricity to the customers in a particular part of the country, into independent companies. It then gradually introduced competition among these companies.3

  The UK’s approach became the global model of how to bring market competition into electric power. It was a forceful and compelling model—including for the United States. Members of the Federal Energy Regulatory Commission, visiting Britain on a study trip, were much impressed by how the once-monolithic state-owned monopoly had been turned into a competitive business, with prices constantly changing in response to supply and demand. The FERC decided to open up the U.S. industry to competition as fast as possible. “The Brits’ enthusiasm about the early successes of their restructuring definitely emboldened us to embark upo
n restructuring,” said Elizabeth Moler, the FERC chair at the time. “We learned from both the successes and failures of the U.S. natural gas restructuring and from what the British did.” Other visitors from the U.S. power industry made the same trek to Britain and came back with similar conclusions. This seemed to be the new future for electric power.4

  ENTER THE MERCHANT GENERATORS

  In the United States, policy at both the federal and state level now began to move toward deregulation. The biggest change was to allow new competitors to get into the generation business and sell their power either to utilities or to end users. And since electricity is an undifferentiated commodity, then new entrants would compete on price. The big idea here was to drive down costs through competition. And in the process, these new entrants were determined to disprove Insull’s dictum that competition was an “unsound economic regulator.”

  The Federal Energy Policy Act of 1992 specifically allowed these newcomers to sell electricity into interstate transmission lines regulated under federal laws. These were given the name “merchant generators” because they did not own the wires and distribution system but rather would sell to those who did. The merchants might be either independent companies or subsidiaries of utilities in some other part of the country. Whichever, they either built new power plants or bought existing ones from utilities. These merchants were selling into second-by-second electronic markets. To implement the competitive intent of the 1992 Energy Policy Act, the Federal Energy Regulatory Commission promoted “wheeling.” That allowed local utilities in one part of the country to contract with a cheaper generator in another part and wheel—that is, transport—the less expensive power over wires across the United States.

  Both merchant generators and traditional utilities realized that they could become more competitive by fueling the new power plants with cheap natural gas. That set off a mad “dash to gas” across the country. In just six years, between 1998 and 2004, the United States added an enormous amount of new generating capacity—equivalent to a quarter of all the capacity that had been built since Edison’s Prince Street station in 1882! Over 90 percent of that capacity burned natural gas. Although not recognized at the time, the dash to gas was also a very big bet on cheap natural gas prices. It led to the overbuild—which produced much more generating capacity than was necessary.

  Yet by the end of the 1990s, cheap gas was disappearing. Prices started to rise sharply once again. The wager on cheap natural gas prices proved costly. Many of the independent merchant generators that had made that bet were caught out. Some went bankrupt. Nowhere did the bet on gas go so badly, or more disastrously, than in California.

  CALIFORNIA’S STRANGE RESTRUCTURING

  A power crisis that erupted in California in 2000 threw the state into disarray, created a vast economic and political firestorm, and shook the entire nation’s electric power system. The brownouts and economic mayhem that rolled over the Golden State would have been expected in a struggling developing nation, but not in the state that was home to Disneyland, and that had given birth to Silicon Valley, the very embodiment of technology and innovation. After all, California was, if an independent country, the seventh-largest economy in the world.

  What unfolded in California graphically exposed the dangers of misdesigning a regulatory system. It was also a case study of how short-term politics can overwhelm the needs of sound policy.

  According to popular lore, the crisis was manufactured and manipulated by cynical and wily out-of-state power traders, the worst being Enron, the Houston-based natural gas and energy company. Its traders and those of other companies were accused of creating and then exploiting the crisis with a host of complex strategies. Some traders certainly did blatantly, and even illegally, exploit the system and thus accentuated its flaws. Yet that skims over the fundamental cause of the crisis. For, by then, the system was already broken.

  The California crisis resulted from three fundamental factors: The first was an unworkable form of partial deregulation that explicitly rejected the normal power-market stabilizers that could have helped avoid or at least blunt the crisis but instead built instability into the new system. The second was a sharp, adverse turn in supply and demand. The third was a political culture that wanted the benefits of increased electric power but without the costs.

  This was not the way it was supposed to be. California enacted deregulation, or restructuring, as it was more commonly called, in 1994. At the time, the state was in a bad way economically. Unemployment hit 10 percent, real estate was a bust, and more people were moving out of the state than were moving in. Spending for defense, one of the state’s main industries, had been cut back sharply with the end of the Cold War, and Sacramento was running big deficits. High electricity prices were partly blamed for the state’s economic slump. Manufacturing companies were fleeing California, in part because of high energy costs, taking jobs with them. Meanwhile people did not worry much about increases in electricity demand. After all, in 1993 demand hadn’t grown at all.

  Competition, it was thought, would bring down the price of power, helping to revive the state’s fortunes. California’s brand of deregulation was fashioned out of a complex negotiation and a great compromise, involving stakeholder democracy, although the stakeholders varied much in terms of their understanding of how power markets worked. Politically, the great compromise worked brilliantly; the deregulation bill sailed through the state legislature in 1996 with not a single dissenting vote and was signed into law by Republican Governor Pete Wilson.5

  Under California’s restructuring, consumer advocates got lower prices; big industrial customers would get access to cheaper power. But in a deregulated market traditional utilities would be stuck with legacy costs of their contracts for PURPA power and the cost overruns on building other new plants—such as the Diablo Canyon nuclear facility on the central California coast that was caught in a regulatory morass and had ended up costing about $11.5 billion. These costs would prevent them from being competitive. The legislation gave the investor-owned utilities the relief they needed—various ways to extricate themselves from the burden of what was called “stranded costs.” They too embraced restructuring. As for the new entrants, the merchant generators, there were two great prizes. One was the ability to sell power into the large California market; and the other, the opportunity to buy the power plants that the state was strongly “encouraging” the utilities to sell. “Every major group got what they wanted most,” said Mason Willrich, who later became chairman of the California grid operator. “But no one connected the dots.”

  This restructuring was an extraordinary edifice in terms of political support. The entire California congressional delegation signed a letter urging the Federal Energy Regulatory Commission not to use federal authority to interfere with the plan. The political forces were so finely balanced that any alteration could cause the whole edifice to come tumbling down.

  The objective was to dismantle the traditional natural monopoly in electric power. The new system, in the words of economist Paul Joskow, was “the most complicated set of wholesale market institutions ever created on earth and with which there was no real world experience.” It yoked together a deregulated market with a regulated market. Some compared it to having a bridge designed by consensus. The subsequent collapse of this particular bridge would demonstrate the hard-earned lessons of power markets.6

  THE IRON CURTAIN

  Wholesale markets were deregulated—along with the markets in which the generators that operate the power plants that sold power to utilities that distributed it to customers. Prices in those markets would be free to fluctuate, in response to supply and demand. But the traditional retail markets—those between the utilities and their customers (home owners, factories, offices, and others)—were not deregulated. This meant that these consumers were to be protected—insulated—from rising prices. They, after all, were the ones who cast votes for governors and state legislators.

  The result was
to build an economic iron curtain between the wholesale and retail markets. The ultimate consequences would be devastating. Changes in wholesale markets, which would reflect those changes in supply and demand, would not flow as price signals into the retail markets—that is, to consumers. Thus consumers would have no incentive, no wake-up call, to make adjustments that would normally happen in response to rising prices (buying a more efficient air conditioner, putting a little more insulation in their walls). They would not get the message because it would not be transmitted to them.

  In order to make the wholesale system function like a competitive market, the state’s utilities were ordered to shear themselves of a substantial number of their in-state power plants and sell them to other companies, which would operate them and in turn sell electricity into the open market. Here was the dissolution of the formerly vertically integrated utility—the kind of utility invented by Samuel Insull, which traditionally combined generation, transmission, and distribution within the borders of a single company. Many of these new merchant generators were out-of-state companies, a number of which had arisen during the era of deregulation.

  Other key elements in the deregulation would make matters still worse. The first is that the scheme did not worry about capacity. Electricity is different from other commodities. Oil can be stored in tanks; grain, in silos; natural gas, in underground caverns. But electricity is the instantaneous commodity; here one second, gone the next. It is a business that operates with virtually no inventory.

  Therefore, a “reserve margin” is needed. Reserves are the stabilizers, the extra production capacity—above projected peak demand—that can be called into operation in order to avoid a shortage. Maintaining such a margin is a basic rule of operations—the power system in its entirety needs to be large enough not just to cover average demand but the extremes of demand, with an additional reserve to allow for accidents or malfunctioning equipment. A state like California, which depends upon hydropower for part of its electricity, needs about a 20 percent reserve margin—20 percent extra capacity—in order to be ready for a spike in demand brought about by a heat wave or a drop in hydropower production because of drought. California’s new system, however, included no incentive or encouragement to ensure sufficient extra capacity to help deregulation work. At some points during the crisis, the reserve margin got as low as 1 percent—which was frighteningly low—essentially no reserve margin at all.