On that morning, I stood on a pristine green lawn, deep in the heart of Silicon Valley, at the former campus of the once-dominant computer colossus Sun Microsystems. Time moves fast in technology’s spiritual epicenter, and today the Valley’s next great company occupied these buildings. It was an organization with youth, exuberance, and the ultimate network effect—a virtual neighborhood that connected almost a billion people. As a business, it had seemingly unlimited prospects, and Bruce Aust had spent endless hours working and developing his relationships inside the company. A few months earlier, he had called me with the good news: We had landed the IPO. “No one could have done it but you,” I told him, as we celebrated over a glass of champagne. It was a happy day, and it represented much more than another name on Nasdaq’s IPO list. This was a massive affirmation of our organizational brand—and an IPO that NYSE had desperately wanted.
Upon arrival at Facebook’s headquarters that morning, Bruce and I had spent a few minutes with Mark Zuckerberg. It was my first conversation with Facebook’s youthful visionary. He was gracious, welcoming, and likable. Wearing his trademark hoodie, Zuckerberg was accompanied by Sheryl Sandberg, Facebook’s COO, who would soon become the most famous COO in the country with the publication of her best-selling book Lean In. After a few minutes of small talk, we strode outside to participate in the festivities.
Facebook stock wouldn’t actually begin trading for another couple of hours, but this was the staff’s moment of victory—a chance to recognize the extraordinary journey they had already taken to get to this point. Facebook was finally going public, and everyone wanted a piece of the action, not least the employees. Many had stayed up for a symbolic all-night hackathon, and hundreds were gathered around at the appropriately named One Hacker Way for the festivities.
“In the past eight years, all of you out there have built the largest community in the history of the world,” Zuckerberg addressed the crowd. “I can’t wait to see what you do going forward!” A cheer erupted from the gathered crowd as we began the countdown to the opening, at 6:30 a.m. Pacific time.
Five… four… three… two… one… The bell rang and the crowd cheered as our faces were beamed around the world on the wings of satellites. Zuckerberg, Sandberg, and the rest of their team were hugging and smiling, and I stood next to them with my fist raised in the air, a snapshot of celebration that I would unexpectedly have the opportunity to revisit in minute detail in the days and weeks ahead.
After the opening-bell excitement had died down, I said my good-byes and quickly jumped in a car to get back to the airport for a flight to the East Coast.
As I watched the morning sun beat back the fog on the hills above Palo Alto, I thought about the milestones in my journey with Nasdaq that had led up to this moment.
It had been almost a decade since I started my journey as CEO of a beaten-down company that was losing money and of diminished significance in a rapidly changing financial industry. In 2003, Nasdaq had been weak, distracted, bureaucratic, facing regulatory challenges, and falling technologically further and further behind. In 2012, it was strong, growing, global, highly profitable, and technologically leading-edge. Markets were changing and the power of Nasdaq’s technology companies was beginning to rival the old industrial giants that had previously dominated the market. NYSE companies like General Electric, Exxon, and Walmart were beginning to be superseded in market-share equity by Apple, Google, Amazon, Microsoft—all Nasdaq stalwarts. We had the best technology. We had the momentum. We had the expertise. We had the global reach. We had a growing software and services franchise, powering exchanges around the world. We were one of the world’s great brands. And now we had Facebook, too.
My phone rang in the car and my reverie was shattered. It was Anna Ewing, my CIO.
“Bob, we have a problem.”
I could tell from her voice it was serious. Oh shit, I thought to myself, not now. “What is it?” I asked.
“It’s the IPO,” she replied. “Something’s not working.”
And that was when it all started to go terribly wrong.
Anatomy of a Glitch
The most important periods of the trading day are the opening and closing sessions. Each day at 9:30 a.m., every buyer and seller in a stock order book is brought together in a virtual auction. The institutional dealer, the retail investor, the mutual fund manager, the pension fund advisor, the hedge fund mogul, the day trader—all their orders cross and intermix virtually and a magical event occurs: The true price is discovered. We take price discovery for granted in today’s equity markets, but it’s the most important function that stock markets perform.
In 2003, when I arrived at Nasdaq, the opening and closing auctions were woefully inadequate, a major gap in our transaction services. At 9:30, our opening bell would ring and the first transaction, large or small, would represent the opening price for the day. That price wasn’t a reliable representation of the buying and selling activity in the market. There wasn’t adequate price discovery. It was the same at closing: We didn’t have what the industry calls a “closing cross,” meaning that all the orders “cross” or are taken into account (as in an auction) in the algorithmic determination of the final price.
In 2003, some of our major customers expressed their frustration at having to mark their indices to such a poorly derived opening and closing price. Standard & Poor’s approached us and said that they were going to start using the auction of the American Stock Exchange, even with Nasdaq-listed shares. Immediately, this became an urgent issue and, frankly, an embarrassing one. I asked Adena to find a solution—her first real test under my leadership. Together with Frank Hatheway, Nasdaq’s Chief Economist, and a small team, she worked on the issue and successfully rebuilt our opening and closing auctions. Nasdaq’s transactions market was significantly improved, and our customers were relieved.
In 2005, after the INET acquisition, customers started requesting that our initial price discovery auction for IPOs be reinvented in the same manner as the opening and closing auctions. We carefully sought their feedback to develop the specs for this new IPO process. What could we do better? One customer complaint was clear: In the original opening and closing auctions, we had developed a virtual “gate” that prevented new orders and cancellations of existing orders from getting through, starting two minutes before the final auction was initiated. Customers asked us to decrease that time frame to a few seconds, arguing that two minutes was a lifetime in the increasingly fast-moving equities markets.
Somewhere along the line, our engineering team decided they had a better idea. Wouldn’t it be great, they thought, if there was no gate at all, if there was an auction that took into account all existing orders and cancellations and recalculated, if necessary, to include any extra orders or cancellations that had been initiated while the auction process was running?
So they built the IPO auction without a gate. If one person sent a cancellation in during the few seconds the auction was running, the process would reinitiate, just for that one order! Given increases in processing speed and computing horsepower, auctions were running faster and faster, and it probably seemed like a natural evolution of that process. This gateless auction was instituted with hardly anyone knowing about it—just a few individuals on the development team.
It was a classic example of overengineering. Conceptually, it was a beautiful idea—as if that mattered. It was like refueling while in flight—technically tricky, but nice if you can make it work. And it had seemed to work fine; we had already run it 450 times without a hitch. In fact, we might have run that gateless auction for another hundred years and never had a single issue. But with the particular confluence of factors that came together in the unique Facebook IPO—massive interest from retail customers, volume unseen in the history of IPOs, people panicking right before the open about the price, and an influx of sudden cancellations—it became a problem.
The first auction ran perfectly well, but several cancellations came in whil
e it was running. The logic of the programming said, Run it again. The initial cancellations were processed, but as it was running again, more cancellations arrived. So it ran the auction yet again. And like a proverbial red queen running faster and faster to stay in the same place, it couldn’t get to the end of the auction. Cancellations kept coming, and it kept running and rerunning. It couldn’t finalize.
At the time, we didn’t understand precisely what was occurring in our IPO code. Later, it would become crystal clear. But for twenty critical minutes, the IPO was unable to settle on an opening price. We issued a statement announcing the delay. We then moved the IPO over to a second matching engine, one with simpler code, and successfully opened the IPO using that engine. For that opening, we used the order book as it had been in our systems at 11:11 a.m. At 11:30 a.m. we started trading Facebook shares normally, based on that order book. By noon we had traded more than 200 million.
The problem came in the information gap we created with the rest of the Street—meaning the big institutions on Wall Street. It took Nasdaq until 1:50 p.m. to get the order confirmations for that opening auction released from the initial engine. Some of those were “too late to cancel” notices, informing customers that their cancellation orders had not been processed, given the issues with the auction. This meant that the investment banks and other institutions trading the IPO had been unable to obtain confirmation on their orders and cancellations (during that critical twenty minutes) for almost two hours, which left them blind as to their exposure. Were their orders filled or not? Had their cancellations gone through our system? Did they own Facebook or not? Many didn’t know during those intervening hours.
Like many things in technology (and life), the specific technical problem was small, brief, and quickly contained, but the damage done in the cascading aftermath was consequential and far-reaching. It rippled out across the spiderweb of Wall Street’s interconnected trading systems, infecting perceptions of the IPO, and impacting trading behavior. As the day went on, frustration with Nasdaq grew. By the end of the trading session, Facebook was still trading normally, but hundreds of angry customers whose cancellations hadn’t gone through were claiming to have lost millions of dollars in the chaos caused by that initial window of confusion.
If Facebook’s stock had gone up that day, those customers would have been thrilled with still owning the stock. But Facebook fell that first day—and a few days after—and so investors who had been unable to cancel in those initial minutes lost money (in the short term). Of course, those customers who had wanted to buy Facebook stock at the opening but were unable to get their orders filled actually avoided losing money, and they could now simply buy Facebook at a lower price. Given that it was about to go on a historic bull run, I can only hope they did.
Goldman Sachs were the smart ones that day. They saw that something was not right with our system and shut down their connection to Nasdaq, so the effect of our problems on them was minimal. Other customers, like UBS and Knight Capital, got into trouble by confirming cancellation orders to their own customers before Nasdaq had provided confirmation notices. Under normal circumstances, that might seem a formality, but not in this situation. When a buy order or a cancellation goes to an exchange, it is considered a live round, so to speak, that is still in play until a confirmation notice is actually sent. When our system came back up, we sent out “too late to cancel” notices—with the stock order confirmations. Those customers still owned the stock. But in some cases, they had already confirmed the cancellations to their own retail customers.
On the day itself, the fallout was immediate. The Street demanded explanations and compensation, and generally started taking Nasdaq’s name in vain to every journalist in town. The press was pushing for a response. Our competitors were taking shots at us. After taking it on the chin for years, NYSE didn’t miss the opportunity to press this brief advantage. Facebook was understandably not happy. By the time I landed on the East Coast, Nasdaq was under siege.
Customer Fallout
“Look, mistakes happen. We weren’t happy about it, but we understand.” Sheryl Sandberg’s generous words were welcome when we reconnected a few weeks after the IPO. I’d flown to California to meet her, and was doing everything I could to respond to all who were affected by the faulty IPO. Facebook was first on the list. Sandberg was being direct but remarkably gracious. Obviously, they weren’t happy with the IPO, but she put the best spin on it: “It’s going to be about how we both perform over time. It’s not just about one day.”
I’ve no doubt the incident was incredibly frustrating to all of Facebook’s executive team, who were counting on Nasdaq’s reputation for technical excellence when they chose to list with our exchange. It was a blow to their faith, but they stuck with us. Over time, I hope we were able to repair the damage to that important relationship.
The other urgent relationship to repair was with our customers on Wall Street—the investment banks, underwriters, and institutions that were directly affected by the IPO delay and the confusion in orders. How much money had actually been lost during the melee? Lots of numbers were getting thrown around, anywhere from tens of millions of dollars to several hundred million. We were determined to take full accountability, but we couldn’t just hand over what people claimed they lost. We needed to determine a reasonable and fair process for resolving claims and determining our liability. Eric Noll, a bright guy who was Chris Concannon’s successor as EVP of Transaction Services, worked hard to determine a process for estimating compensation. The SEC also had their own inquiry underway, and they were going to have to approve whatever we came up with.
What was our actual legal liability? Arguably, very little. We had a clause in our rulebook explicitly stating that our liability for trading losses was very limited. But what good would legality do us if we damaged our brand and enraged all of our business partners and customers? After consultation with the Board, we decided to change our policy for this particular situation. It was a delicate decision—we didn’t want to open the door to a flood of spurious legal claims that had nothing to do with Facebook and set a precedent that we would regret down the line. But specifically for the IPO and resulting fallout, we decided to make good on the claims against us, where reasonable.
The SEC strongly emphasized that we should adjudicate our claims in as objective a manner as possible and deal with all customers on an equal footing. They wanted to ensure against us giving some customers special dispensation in exchange for future favors or business. This meant we weren’t allowed to consult with our customers in the process. Eric analyzed the situation, and we came up with a plan—but largely in a vacuum. The Street panned it. Many felt it didn’t adequately address the liability. So Ed Knight headed back to the SEC and convinced them that we had to be able to deal directly with the customers who were impacted. They relented, and we modified our approach.
Over the next weeks, I ended up talking extensively with the leaders of Wall Street financial institutions that had been affected—firms like UBS, Citadel, Morgan Stanley, and Goldman Sachs. While many individual investors were impacted by the IPO, their trades don’t come to us directly; they come through these financial intermediaries. Those were our direct customers, and we had to work with them to find a solution. It was not an easy negotiation, and I remember long, intensive days spent working to develop a fair formula and “get to yes.”
Finally, we came to an agreement. We decided that during a specific window of time, if you had sent in an order to sell existing shares or had initiated a buy order and then followed up with a subsequent cancellation, we would respect those orders as if they had been confirmed and pay whatever money was lost as a result of the subsequent drop in the stock price. In other words, in the brief period of time that our system was malfunctioning, we compensated people as if it had been working perfectly. We even assumed infinite market capacity to absorb all of those sells and cancellations as part of the agreement. Nasdaq calculated an estimated
liability and put money aside to cover the possible claims. The SEC approved it. The Financial Industry Regulatory Authority (FINRA) and its CEO, Rick Ketchum, graciously agreed to audit the process.
I particularly remember one of the last conversations during this period, with the founder and CEO of Citadel, Ken Griffin. By this time, everything had been negotiated, and almost all the major players had accepted our terms. It had been an exhausting process, involving millions of dollars in losses, which adds just a little emotion to such negotiations. In most cases I was sitting across the table from colleagues, rivals, or friends.
After I explained the final agreement, Griffin paused for a moment on the other end of the line. “Ken, does that all makes sense?” I asked. He replied, “Yes, everything looks good. I appreciate your work on this.” Then he added, in a very slow and deliberate way, “We’re good here, Bob.”
Perhaps it was just the timing, or the way he said it. It had that sense of finality—a signal the process had come to an end. Nasdaq had tried to make good on its obligations to customers and investors—without giving away the farm. Now we could all get back to being colleagues and, in some cases, competitors.
Loser of the Week
In the week after the Facebook IPO, the unfortunate image of my opening-bell fist pump next to Zuckerberg was played on an endless loop on business media channels—my Icarus moment, exposed for all the world to see. I had become used to being in the public eye, and I had been through brief periods in the media superstorm (especially during the attempted takeovers of LSE and NYSE), but this was a darker version. It was hard to escape, and took its toll over the days and weeks that followed. “Robert Greifeld: Fist-Pumping while the Exchange Burned” read one caption.
The Saturday after the IPO was the celebration for my daughter Katie’s eighteenth birthday. It was a special moment for her, and as our family and friends celebrated on the lawn of our home, I remember that the beautiful late-spring setting was a marked contrast to the dark clouds swirling around in my own mind. The life of a CEO is demanding, but usually I was able to be fully engaged and present for such an important family milestone. On this day, however, it was impossible to let work just be work. Julia stayed close to my side, a comfort amid the gloom that seemed impossible to shake, even as our daughter crossed a threshold into adulthood.
Market Mover Page 19