The Quest: Energy, Security, and the Remaking of the Modern World

by Daniel Yergin

  Usually the global oil industry operates with a few million barrels of shut-in capacity—that is, production capability that is not used. Between 1996 and 2003, for instance, spare capacity had averaged about 4 million barrels per day. That shut-in capacity is a security cushion, a shock absorber to manage sudden surges in demand or some kind of interruption. One supplier country has made an explicit commitment to hold significant spare capacity. Saudi Arabia’s policy is to build and maintain spare capacity of between 1.5 and 2 million barrels per day in order to promote market stability. But for other countries, spare capacity is somewhat inadvertent. In 2005, however, the surge in demand and disruptions of supply shrank spare capacity to no more than a million barrels a day. In other words, the cushion was virtually gone. In terms of absolute spare capacity, the oil market was considerably tighter than it had been on the eve of the 1973 oil crisis. In relative terms it was even tighter, as the world oil market was 50 percent bigger in 2005 than in 1973.

  In such circumstances the inevitable happens. Price has to rise to balance supply and demand by calling forth more production and investment on one side of the ledger, and on the other, by signaling the need for moderation in demand growth. By the spring of 2005, OPEC’s $22-to-$28 price band was an artifact of history. Many now may have thought that $40 to $50 was the “fair price” for oil. But that was only the beginning.

  Other factors reinforced the rising price trend. In the aftermath of the 1998 price collapse, the industry had contracted, and then had continued to do so, on the basis of expectations for low prices. It was focused on keeping spending under tight control. As late as August 2004, the message from one of the supermajors was that “our long-term price guidelines are around the low $20s.” Or, as the chief financial officer of another of the supermajors put it, “ We remain cautious.” The industry continued to fear another price collapse that would undermine the economics of new projects. Investors exerted tremendous pressure on managements to demonstrate “capital discipline” and hold back spending. The reward was a higher stock price. And if companies did not heed the admonition, they would be punished with a lower stock price. As one such investor warned in mid-2004, if companies started increasing investment because of higher oil prices, “I’d look at that skeptically.”3

  WHERE ARE THE PETROLEUM ENGINEERS?

  “Capital discipline” translated into caution. The mantras were “take out costs” and “reduce capacity.” That meant reductions in people, drilling rigs, and everything else. In the late 1990s and early years of the 2000s, not only did many skilled people leave the industry, but university enrollments in petroleum engineering and other oil-related disciplines plummeted. If there were no jobs, what was the point?

  But the sharp increase in demand in 2004 and 2005 delivered an abrupt jolt. No longer was the fear about going back to 1998 and a giant surplus that would tank prices. Now it was just the opposite—not having enough oil. Hurriedly switching gears, the industry went into overdrive to develop new supplies as fast as possible. Companies started competing much more actively for acreage and access to resources. As would be expected, the price of entry for new production opportunities went up. Nations were making more money than they had anticipated and thus were tougher in their financial demands on companies, and in this more competitive environment, they could get the terms they wanted. Competition for exploration and production opportunities was made even more intense by the arrival of new entrants in the international business—national oil companies based in emerging-market countries—which were willing to spend to gain access.

  The industry was hamstrung in its ability to respond. Contraction had taken its toll. There were not enough petroleum engineers, not enough geologists, not enough drilling rigs, not enough pipe, not enough supply ships, not enough of everything. And so the cost of everything was bid up. Shortages of people and delays in the delivery of equipment meant that new projects took longer than planned, adding to the budget overruns.

  On top of that, the cost of the inputs—such as the steel that went into platforms, and nickel and copper—was also rising dramatically as China’s appetite for commodities continued to draw in supplies from all over the world. This was the era of the great bull market for commodities.

  The economic impact of all these shortages was stunning. Total costs for doing business ended up more than doubling in less than half a decade. In other words, the budget for developing an oil field in 2008 would have been twice what the budget for the same field would have been in 2004. These rising costs also, inevitably, contributed to the rising price of oil.4

  “FINANCIALIZATION”

  Then there was the matter of currencies; in particular, the dance between oil and the dollar. In this period, commodity prices would, in the jargon of economists, “co-move negatively with the U.S. dollar exchange rate.” Put more plainly, it meant that when the dollar moved down, oil prices moved up. Petroleum is priced in dollars. For part of this period the dollar was weak, losing value against other currencies. Traditionally during times of political turmoil and uncertainty, there is a “flight to the dollar” in the search for safety. But during this period of dollar turmoil, the flight was to commodities, most of all to petroleum, along with gold. Oil was a hedge against a weaker dollar and the risks of inflation. So as the “price” of the dollar went down against other currencies, particularly the euro, the price of oil went up.5

  More generally the financial markets and the rising tide of investor money were having increasing impact on the oil price. This is often described as speculation. But speculation is only part of the picture, for oil was no longer only a physical commodity; it was also becoming a financial instrument, a financial asset. Some called this process the “financialization” of oil. Whatever the name, it was a process that had been building up over time.6

  THE RISE OF OIL TRADING

  Into the 1970s, there really was no world oil market in which barrels were traded back and forth. Most of the global oil trade took place inside each of the integrated oil companies, among their various operating units, as oil moved from the well into tankers, and then into refineries and into gasoline stations. Throughout this long journey, the oil remained largely within the borders of the company. This was what was meant by “integration.” It was considered the natural order of the business, the way the oil industry was to be managed.

  But politics and nationalism changed all that. In the 1970s the oil-exporting countries nationalized the concessions held by the companies, which they regarded as holdovers from a more colonial era. After nationalization, the companies no longer owned the oil in the ground. The integrated links were severed. Significant amounts of oil were sold under long-term contracts. But oil also became an increasingly traded commodity, sold into a growing and variegated world oil market. Those transactions, in turn, were handled both by newly established trading divisions within the traditional companies, and by a host of new, independent commodity traders.

  A change in the United States gave a further boost to this new business of oil trading. From the early 1970s onward, the federal government controlled and set the price of oil. These price controls were originally imposed during the Nixon administration as an anti-inflation initiative. They did succeed in creating a whole new federal bureaucracy, an explosion in regulatory and litigation work for lawyers, and much political contention. But the controls did little for their stated goals of limiting inflation—and did nothing for energy security. In 1979, after a bruising political battle, President Jimmy Carter implemented a two-year phase-out of price controls. When Ronald Reagan took over as president in January 1981, he speeded things up and ended price controls immediately. It was his very first executive order.

  This shift from price controls to markets was not just a U.S. phenomenon. In Britain, the government shifted from a fixed price for setting petroleum tax rates to using spot price. As its benchmark, it used a North Sea stream called Brent.7

  FROM EGGS TO OIL : THE PAPER BA
RREL

  Now oil was becoming “just another commodity.” Although OPEC was still trying to manage prices, it had a new competitor—the global market. And, specifically, a new marketplace emerged to help buyers and sellers manage the risk of fluctuating prices. This was the New York Mercantile Exchange—the NYMEX. The exchange itself wasn’t exactly new. It had actually begun its life as the Butter and Cheese Exchange, founded in 1872 by several dozen merchants who needed a place to trade their dairy products. It soon expanded its offerings and became the Butter, Cheese, and Egg Exchange. By the 1920s, in a little-noticed innovation, egg futures were added to the trading menu, at what was now the more grandly renamed New York Mercantile Exchange.

  By the 1940s the NYMEX was also the trading place for a motley group of other commodities, ranging from yellow globe onions to apples and plywood. But the exchange’s mainstay was the Maine potato. Yet potatoes had progressively less skin in the game: for in the late 1970s, scandals hit the Maine potato contract, including the mortifying failure of the potatoes to pass the basic New York City health inspection. It looked like the exchange was going to go under. Just in time, the NYMEX started trading futures contracts in home heating oil and gasoline. This, however, was only the beginning.

  March 30, 1983, was the historic day when the exchange began trading a futures contract for light, sweet crude, tied to a stream called West Texas Intermediate—WTI—and linked back to those tanks in Cushing , Oklahoma. Now the price of oil was being set by the interaction of the floor traders at the NYMEX with other traders and hedgers and speculators all over the world. Thus was the beginning of the “paper barrel.” As technology advanced over the years, the price would be set not only daily and hourly, but eventually on a second-by-second basis.

  HEDGERS VERSUS SPECULATORS

  Today’s futures markets go back to the futures markets for agricultural products established in the nineteenth century in Midwestern cities of the United States. By availing himself of the futures market, a farmer planting his spring wheat could assure himself of his sales price for the following fall. He might lose the upside if the price of wheat shot up. But by using futures, he avoided financial ruin in case a bumper crop tanked the price.

  The petroleum futures market on the NYMEX now provided what is called a “risk-management tool” for people who produced oil or who used it. An airline would buy contracts for oil futures to protect itself against the possibility of rising prices of the physical commodity. It would put down the fraction of a cost of a barrel for the right to buy a hundred contracts—equivalent to 100,000 barrels—a year or two years from now at the current price. The price of oil—and jet fuel—might go up 50 percent a year from now. But the futures contracts would have gone up by about the same value, and the airline could close out its position, accruing the same amount as the price increase—minus the cost of buying the futures. Thus the airline would have protected itself by buying the futures, although putting the hedge in place did cost money. But that cost was, in effect, what the airline was willing to pay to insure itself against a price increase.

  For an airline, or an independent oil producer protecting itself against a fall in the price, or a home-heating oil distributor worrying about what would happen in the winter, someone needed to be on the other side of the trade. And who might that person be ? That someone was the speculator, who had no interest in taking delivery of the physical commodity but is only interested in making a profit on the trade by, as the NYMEX puts it, “successfully anticipating price movements.” If you wanted to buy a futures contract to protect against a rising price, the speculator would in effect sell it. If you wanted to sell to protect yourself against a falling price, the speculator would buy. The speculator moved in and out of trades in search of profits, offsetting one position against another. Without the speculator, the would-be hedger cannot hedge.8

  Often, it seems, the word “speculator” is confused with “manipulator.” But “speculation” is, in its use here, a technical term with rather precise meaning. The “speculator” is a “non-commercial player”—a market maker, a serious investor, or a trader acting on technical analysis. The speculator plays a crucial role. If there is no speculator, there is no liquidity, no futures market, no one on the other side of the trade, no way for a hedger—the aforementioned airline or oil producer or the farmer planting his spring wheat or the multinational company worried about currency volatility—to buy some insurance in the form of futures against the vagaries of price and fortune.

  Futures and options trading in oil rose from small amounts in the mid-1980s to very large volumes. By 2004 trading in oil futures on the NYMEX was 30 times what it had been in 1984. Similar growth was registered on the other major oil futures market. This was the ICE exchange in London, originally called the International Petroleum Exchange, where Brent, the North Sea oil stream, is traded. The Brent contract in London and the “sweet crude” contract in New York became the global standards for oil against which other crudes were benchmarked. WTI was oriented toward North America; Brent, toward the Eastern Hemisphere. Later a Dubai contract was introduced in the Middle East.

  After the stock market bust of 2000, investors wanted to find alternative investments. It was observed at the time that the prices of commodities did not move in coordination with other investment choices; that is, they were not correlated with stocks and bonds. So according to theory, if the value of a pension fund’s equity holdings declined, the value of the commodities would not. They might even go up. Thus commodities would protect portfolios against declines in stock markets and help pension funds to assure the returns on which their retirees depended. In the years that followed, diversification into commodities became a major new investment strategy among many pension funds.

  Investors were trying to purchase other forms of “insurance” as well. A large European state pension fund, for instance, was buying futures contracts to protect its portfolio against, as its chief investor officer put it, “a conflict in the Middle East”—which really meant a war involving Iran. Were such an event to occur, the value of the fund’s equity holdings would likely drop dramatically, while oil prices would likely soar. This pension fund thought it was acting as a prudent investor, hedging its portfolio against disruption and allocating among assets to protect its retirees. But, by the definition of the futures market, it was a speculator.9

  THE “BRICs”: THE INVESTMENT OPPORTUNITY OF A GENERATION

  Putting money to work in oil-based financial instruments was also seen as a way to participate on the greatest economic trend of a generation: globalization and economic growth in China, India, and other emerging markets.

  In November 2001 an economist at Goldman Sachs, Jim O’Neill, put out a research paper hatching a new concept: “the BRICs”—Brazil, Russia, India, and China. These four large-population economies, he said, were destined to grow faster than the main industrial economies. He made the startling prediction that within a few decades they would, as a group, overtake the combined GDP of the United States and the world’s five other largest economies.

  O’Neill came to the BRICs idea in the aftermath of 9/11. “I felt that if globalization were to thrive, it would no longer be American-led,” he said. “It had to be” based on the reality that “international trade lifts all.” There was also what he called the “odd insight” that provided a lightbulb moment: on flights to China, he had noticed continuing improvements in the standards and quality of service, rising toward world levels. “Rightly or wrong, I associated that with China’s involvement.” Something new was happening in the world economy.

  Initially, many people found the whole concept of BRICs wacky. They shook their heads and asked what these diverse countries could possibly have in common. “They thought it was just some kind of marketing gimmick,” said O’Neill. But by 2004 the concept of BRICs was providing a different—and powerful and compelling—framework for looking at the world economy and international growth. Competing banks, which had
previously made fun of the idea, were now launching their own BRIC funds. And in the ultimate stamp of approval, leaders of the four BRIC-anointed countries eventually started to meet for their own exclusive BRICs-only summits.

  “BRICS,” said the Financial Times, became “a near ubiquitous term, shaping how a generation of investors, financiers and policymakers view the emerging markets.” Investors started to buy equities linked to the BRICs. They also bought financial instruments linked to oil. For the growth of these countries—especially the “C,” China—was driving the demand for commodities and thus prices. Thus for investors—whether running hedge funds or pension funds, or retail investors—the commodity play was not just about oil itself, but about the booming economies that were using more and more oil.10

  TRADING PLACES

  And now there were a lot more people in the oil market—the paper barrel part of the market—investing with no intention nor any need of ever taking delivery of the physical commodity. There were pension funds and hedge funds and sovereign wealth funds. There were the “massive passives”—the commodity index funds, heavily weighted to oil and with all the derivative trading around them. There were also exchange-traded funds; there were high net-worth individuals; and there were all sorts of other investors and traders, some of them in for the long term, and some of them very short term.

  Oil was no longer just a physical commodity, required to fuel cars and airplanes. It really had become something new—and much more abstract. Now these paper barrels were also, in the form of futures and derivatives, a financial instrument, a financial asset. As such, prudent investors could diversify beyond stocks, bonds, and real estate, by shifting money into this new asset class.