Weapons of Math Destruction

by Cathy O'Neil

  So to sum up, these are the three elements of a WMD: Opacity, Scale, and Damage. All of them will be present, to one degree or another, in the examples we’ll be covering. Yes, there will be room for quibbles. You could argue, for example, that the recidivism scores are not totally opaque, since they spit out scores that prisoners, in some cases, can see. Yet they’re brimming with mystery, since the prisoners cannot see how their answers produce their score. The scoring algorithm is hidden. A couple of the other WMDs might not seem to satisfy the prerequisite for scale. They’re not huge, at least not yet. But they represent dangerous species that are primed to grow, perhaps exponentially. So I count them. And finally, you might note that not all of these WMDs are universally damaging. After all, they send some people to Harvard, line others up for cheap loans or good jobs, and reduce jail sentences for certain lucky felons. But the point is not whether some people benefit. It’s that so many suffer. These models, powered by algorithms, slam doors in the face of millions of people, often for the flimsiest of reasons, and offer no appeal. They’re unfair.

  And here’s one more thing about algorithms: they can leap from one field to the next, and they often do. Research in epidemiology can hold insights for box office predictions; spam filters are being retooled to identify the AIDS virus. This is true of WMDs as well. So if mathematical models in prisons appear to succeed at their job—which really boils down to efficient management of people—they could spread into the rest of the economy along with the other WMDs, leaving us as collateral damage.

  That’s my point. This menace is rising. And the world of finance provides a cautionary tale.

  Imagine you have a routine. Every morning before catching the train from Joliet to Chicago’s LaSalle Street station, you feed $2 into the coffee machine. It returns two quarters and a cup of coffee. But one day it returns four quarters. Three times in the next month the same machine delivers the same result. A pattern is developing.

  Now, if this were a tiny anomaly in financial markets, and not a commuter train, a quant at a hedge fund—someone like me—could zero in on it. It would involve going through years of data, even decades, and then training an algorithm to predict this one recurring error—a fifty-cent swing in price—and to place bets on it. Even the smallest patterns can bring in millions to the first investor who unearths them. And they’ll keep churning out profits until one of two things happens: either the phenomenon comes to an end or the rest of the market catches on to it, and the opportunity vanishes. By that point, a good quant will be hot on the trail of dozens of other tiny wrinkles.

  The quest for what quants call market inefficiencies is like a treasure hunt. It can be fun. And as I got used to my new job at D. E. Shaw, I found it a welcome change from academia. While I had loved teaching at Barnard, and had loved my research on algebraic number theory, I found progress agonizingly slow. I wanted to be part of the fast-paced real world.

  At that point, I considered hedge funds morally neutral—scavengers in the financial system, at worst. I was proud to go to Shaw, known as the Harvard of the hedge funds, and show the people there that my smarts could translate into money. Plus, I would be earning three times what I had earned as a professor. I could hardly suspect, as I began my new job, that it would give me a front-row seat during the financial crisis and a terrifying tutorial on how insidious and destructive math could be. At the hedge fund, I got my first up-close look at a WMD.

  In the beginning, there was plenty to like. Everything at Shaw was powered by math. At a lot of firms, the traders run the show, making big deals, barking out orders, and landing multimillion-dollar bonuses. Quants are their underlings. But at Shaw the traders are little more than functionaries. They’re called executioners. And the mathematicians reign supreme. My ten-person team was the “futures group.” In a business in which everything hinges on what will happen tomorrow, what could be bigger than that?

  We had about fifty quants in total. In the early days, it was entirely men, except for me. Most of them were foreign born. Many of them had come from abstract math or physics; a few, like me, had come from number theory. I didn’t get much of a chance to talk shop with them, though. Since our ideas and algorithms were the foundation of the hedge fund’s business, it was clear that we quants also represented a risk: if we walked away, we could quickly use our knowledge to fuel a fierce competitor.

  To keep this from happening on a large, firm-threatening scale, Shaw mostly prohibited us from talking to colleagues in other groups—or sometimes even our own office mates—about what we were doing. In a sense, information was cloistered in a networked cell structure, not unlike that of Al Qaeda. That way, if one cell collapsed—if one of us hightailed it to Bridgewater or J.P. Morgan, or set off on our own—we’d take with us only our own knowledge. The rest of Shaw’s business would carry on unaffected. As you can imagine, this wasn’t terrific for camaraderie.

  Newcomers were required to be on call every thirteen weeks in the futures group. This meant being ready to respond to computer problems whenever any of the world’s markets were open, from Sunday evening our time, when the Asian markets came to life, to New York’s closing bell at 4 p.m. on Friday. Sleep deprivation was an issue. But worse was the powerlessness to respond to issues in a shop that didn’t share information. Say an algorithm appeared to be misbehaving. I’d have to locate it and then find the person responsible for it, at any time of the day or night, and tell him (and it was always a him) to fix it. It wasn’t always a friendly encounter.

  Then there were panics. Over holidays, when few people were working, weird things tended to happen. We had all sorts of things in our huge portfolio, including currency forwards, which were promises to buy large amounts of a foreign currency in a couple of days. Instead of actually buying the foreign currency, though, a trader would “roll over” the position each day so the promise would be put off for one more day. This way, our bet on the direction of the market would be sustained but we’d never have to come up with loads of cash. One time over Christmas I noticed a large position in Japanese yen that was coming due. Someone had to roll that contract over. This was a job typically handled by a colleague in Europe, who presumably was home with his family. I saw that if it didn’t happen soon someone theoretically would have to show up in Tokyo with $50 million in yen. Ironing out that problem added a few frantic hours to the holiday.

  All of those issues might fit into the category of occupational hazard. But the real problem came from a nasty feeling I started to have in my stomach. I had grown accustomed to playing in these oceans of currency, bonds, and equities, the trillions of dollars flowing through international markets. But unlike the numbers in my academic models, the figures in my models at the hedge fund stood for something. They were people’s retirement funds and mortgages. In retrospect, this seems blindingly obvious. And of course, I knew it all along, but I hadn’t truly appreciated the nature of the nickels, dimes, and quarters that we pried loose with our mathematical tools. It wasn’t found money, like nuggets from a mine or coins from a sunken Spanish galleon. This wealth was coming out of people’s pockets. For hedge funds, the smuggest of the players on Wall Street, this was “dumb money.”

  It was when the markets collapsed in 2008 that the ugly truth struck home in a big way. Even worse than filching dumb money from people’s accounts, the finance industry was in the business of creating WMDs, and I was playing a small part.

  The troubles had actually started a year earlier. In July of 2007, “interbank” interest rates spiked. After the recession that followed the terrorist attacks in 2001, low interest rates had fueled a housing boom. Anyone, it seemed, could get a mortgage, builders were turning exurbs, desert, and prairie into vast new housing developments, and banks gambled billions on all kinds of financial instruments tied to the building bonanza.

  But these rising interest rates signaled trouble. Banks were losing trust in each other to pay back overnight loans. They were slowly coming to grips with the dange
rous junk they held in their own portfolios and judged, wisely, that others were sitting on just as much risk, if not more. Looking back, you could say the interest rate spikes were actually a sign of sanity, although they obviously came too late.

  At Shaw, these jitters dampened the mood a bit. Lots of companies were going to struggle, it was clear. The industry was going to take a hit, perhaps a very big one. But still, it might not be our problem. We didn’t plunge headlong into risky markets. Hedge funds, after all, hedged. That was our nature. Early on, we called the market turbulence “the kerfuffle.” For Shaw, it might cause some discomfort, maybe even an embarrassing episode or two, like when a rich man’s credit card is denied at a fancy restaurant. But there was a good chance we’d be okay.

  Hedge funds, after all, didn’t make these markets. They just played in them. That meant that when the market crashed, as it would, rich opportunities would emerge from the wreckage. The game for hedge funds was not so much to ride markets up as to predict the movements within them. Down could be every bit as lucrative.

  To understand how hedge funds operate at the margins, picture a World Series game at Chicago’s Wrigley Field. With a dramatic home run in the bottom of the ninth inning, the Cubs win their first championship since 1908, back when Teddy Roosevelt was president. The stadium explodes in celebration. But a single row of fans stays seated, quietly analyzing a slew of results. These gamblers don’t hold the traditional win-or-lose bets. Instead they may have bet that Yankees relievers would give up more walks than strikeouts, that the game would feature at least one bunt but no more than two, or that the Cubs’ starter would last at least six innings. They even hold bets that other gamblers will win or lose their own bets. These people wager on many movements associated with the game, but not as much on the game itself. In this, they behave like hedge funds.

  That made us feel safe, or at least safer. I remember a gala event to celebrate the architects of the system that would soon crash. The firm welcomed Alan Greenspan, the former Fed chairman, and Robert Rubin, the former Treasury secretary and Goldman Sachs executive. Rubin had pushed for a 1999 revision of the Depression-era Glass-Steagall Act. This removed the glass wall between banking and investment operations, which facilitated the orgy of speculation over the following decade. Banks were free to originate loans (many of them fraudulent) and sell them to their customers in the form of securities. That wasn’t so unusual and could be considered a service they did for their customers. However, now that Glass-Steagall was gone, the banks could, and sometimes did, bet against the very same securities that they’d sold to customers. This created mountains of risk—and endless investment potential for hedge funds. We placed our bets, after all, on market movements, up or down, and those markets were frenetic.

  At the D. E. Shaw event, Greenspan warned us about problems in mortgage-backed securities. That memory nagged me when I realized a couple of years later that Rubin, who at the time worked at Citigroup, had been instrumental in collecting a massive portfolio of these exact toxic contracts—a major reason Citigroup later had to be bailed out at taxpayer expense.

  Sitting with these two was Rubin’s protégé and our part-time partner, Larry Summers. He had followed Rubin in Treasury and had gone on to serve as president of Harvard University. Summers had troubles with faculty, though. And professors had risen up against him in part because he suggested that the low numbers of women in math and the hard sciences might be due to genetic inferiority—what he called the unequal distribution of “intrinsic aptitude.”

  After Summers left the Harvard presidency, he landed at Shaw. And I remember that when it came time for our founder, David Shaw, to address the prestigious trio, he joked that Summers’s move from Harvard to Shaw had been a “promotion.” The markets might be rumbling, but Shaw was still on top of the world.

  Yet as the crisis deepened the partners at Shaw lost a bit of their swagger. Troubled markets, after all, were entwined. For example, rumors were already circulating about the vulnerability of Lehman Brothers, which owned 20 percent of D. E. Shaw and handled many of our transactions. As the markets continued to rattle and shake, the internal mood turned fretful. We could crunch numbers with the best of the best. But what if the frightening tomorrow on the horizon didn’t resemble any of the yesterdays? What if it was something entirely new and different?

  That was a concern, because mathematical models, by their nature, are based on the past, and on the assumption that patterns will repeat. Before long, the equities group liquidated its holdings, at substantial cost. And the hiring spree for new quants, which had brought me to the firm, ended. Although people tried to laugh off this new climate, there was a growing fear. All eyes were on securitized products, especially the mortgage-backed securities Greenspan had warned us about.

  For decades, mortgage securities had been the opposite of scary. They were boring financial instruments that individuals and investment funds alike used to diversify their portfolios. The idea behind them was that quantity could offset risk. Each single mortgage held potential for default: the home owner could declare bankruptcy, meaning the bank would never be able to recover all of the money it had loaned. At the other extreme, the borrower could pay back the mortgage ahead of schedule, bringing the flow of interest payments to a halt.

  And so in the 1980s, investment bankers started to buy thousands of mortgages and package them into securities—a kind of bond, which is to say an instrument that pays regular dividends, often at quarterly intervals. A few of the home owners would default, of course. But most people would stay afloat and keep paying their mortgages, generating a smooth and predictable flow of revenue. In time, these bonds grew into an entire industry, a pillar of the capital markets. Experts grouped the mortgages into different classes, or tranches. Some were considered rock solid. Others carried more risk—and higher interest rates. Investors had reason to feel confident because the credit-rating agencies, Standard & Poor’s, Moody’s, and Fitch, had studied the securities and scored them for risk. They considered them sensible investments. But consider the opacity. Investors remained blind to the quality of the mortgages in the securities. Their only glimpse of what lurked inside came from analyst ratings. And these analysts collected fees from the very companies whose products they were rating. Mortgage-backed securities, needless to say, were an ideal platform for fraud.

  If you want a metaphor, one commonly used in this field comes from sausages. Think of the mortgages as little pieces of meat of varying quality, and think of the mortgage-backed securities as bundles of the sausage that result from throwing everything together and adding a bunch of strong spices. Of course, sausages can vary in quality, and it’s hard to tell from the outside what went into them, but since they have a stamp from the USDA saying they’re safe to eat, our worries are put aside.

  As the world later learned, mortgage companies were making rich profits during the boom by loaning money to people for homes they couldn’t afford. The strategy was simply to write unsustainable mortgages, snarf up the fees, and then unload the resulting securities—the sausages—into the booming mortgage security market. In one notorious case, a strawberry picker named Alberto Ramirez, who made $14,000 a year, managed to finance a $720,000 house in Rancho Grande, California. His broker apparently told him that he could refinance in a few months and later flip the house and make a tidy profit. Months later, he defaulted on the loan.

  In the run-up to the housing collapse, mortgage banks were not only offering unsustainable deals but actively prospecting for victims in poor and minority neighborhoods. In a federal lawsuit, Baltimore officials charged Wells Fargo with targeting black neighborhoods for so-called ghetto loans. The bank’s “emerging markets” unit, according to a former bank loan officer, Beth Jacobson, focused on black churches. The idea was that trusted pastors would steer their congregants toward loans. These turned out to be subprime loans carrying the highest interest rates. The bank sold these even to borrowers with rock-solid credit, who shou
ld have qualified for loans with far better terms. By the time Baltimore filed the suit, in 2009, more than half of the properties subject to foreclosure on Well Fargo loans were empty, and 71 percent of them were in largely African American neighborhoods. (In 2012, Wells Fargo settled the suit, agreeing to pay $175 million to thirty thousand victims around the country.)

  To be clear, the subprime mortgages that piled up during the housing boom, whether held by strawberry pickers in California or struggling black congregants in Baltimore, were not WMDs. They were financial instruments, not models, and they had little to do with math. (In fact, the brokers went to great lengths to ignore inconvenient numbers.)

  But when banks started loading mortgages like Alberto Ramirez’s into classes of securities and selling them, they were relying on flawed mathematical models to do it. The risk model attached to mortgage-backed securities was a WMD. The banks were aware that some of the mortgages were sure to default. But banks held on to two false assumptions, which sustained their confidence in the system.

  The first false assumption was that crack mathematicians in all of these companies were crunching the numbers and ever so carefully balancing the risk. The bonds were marketed as products whose risk was assessed by specialists using cutting-edge algorithms. Unfortunately, this just wasn’t the case. As with so many WMDs, the math was directed against the consumer as a smoke screen. Its purpose was only to optimize short-term profits for the sellers. And those sellers trusted that they’d manage to unload the securities before they exploded. Smart people would win. And dumber people, the providers of dumb money, would wind up holding billions (or trillions) of unpayable IOUs. Even rigorous mathematicians—and there were a few—were working with numbers provided by people carrying out wide-scale fraud. Very few people had the expertise and the information required to know what was actually going on statistically, and most of the people who did lacked the integrity to speak up. The risk ratings on the securities were designed to be opaque and mathematically intimidating, in part so that buyers wouldn’t perceive the true level of risk associated with the contracts they owned.