Amazon Unbound

by Brad Stone

  George also instituted a dramatic change in the Alexa group’s structure. It had been a functional organization, with centralized engineering, product management, and marketing teams. But that wasn’t growing smoothly or fast enough for Bezos’s liking. George instead reorganized Alexa around the Amazonian ideal of fast-moving “two pizza” teams, each devoted to a specific Alexa domain, like music, weather, lighting, thermostats, video devices, and so on.

  Each team was run by a so-called “single-threaded leader” who had ultimate control and absolute accountability over their success or failure. (The phrase comes from computer science terminology; a single-threaded program executes one command at a time.) Alexa, like Amazon itself, became a land of countless CEOs, each operating autonomously. To yoke them all together, George oversaw the creation of a “north star” document, to crystalize the strategy of a global, voice-enabled computing platform.

  Meanwhile Bezos approved all these changes and stayed intimately involved, attending product reviews and reading the Friday night compilation of updates from all the various two-pizza teams, and responding with detailed questions or problems that the groups would then have to fix over the weekend. Alexa execs, like leaders elsewhere in Amazon, became frequent recipients of the CEO’s escalation emails, in which he forwarded a customer complaint accompanied by a single question mark and then expected a response within twenty-four hours. He was also the chief evangelist for Alexa within the company. “What are you doing for Alexa?” he asked other executives, as he had for AWS years before. Everyone in the company had to include Alexa in the OP1 documents they presented to the S-team, describing their plans for the coming year.

  At the end of 2016, after eight million U.S. households had purchased an Echo or Echo Dot, device exec Dave Limp announced internally that Amazon had become the top-selling speaker company in the world. It validated the entire crusade. But of course, Bezos wanted to become the top AI firm in the world, and in that respect, he was about to have significant competition.

  That fall, Google introduced the Google Home smart speaker. It looked considerably more stylish, “like something you might plant a succulent in,” said Wired. It also had a crisper sound and, predictably, searched the web and retrieved answers with aplomb. The Alexa team had “gone into every holiday season just waiting for either Apple or Google to announce something, and when they didn’t, we would just high-five each other,” said former Alexa exec Charlie Kindel. Even though those companies were allergic to what they considered copycat products, eventually they couldn’t resist the fast-growing smart speaker market.

  That added to the pressure on the Alexa team to move faster and stay ahead with new features and variations on the hardware. In early 2017, a Swedish customer emailed Bezos to ask why Amazon was waiting to develop language-specific versions of Alexa before introducing the Echo in Europe. Why couldn’t they just sell it everywhere in English first? This was actually on the product road map but wasn’t a priority. According to one executive, Bezos got that email at 2 a.m. Seattle time, and by the following morning there were a half dozen independent groups working to sell Alexa in eighty new countries.

  Later, Alexa execs would say that Bezos’s close involvement made their lives more difficult but also produced immeasurable results. Jeff “gave us the license and permission to do some of the things we needed to do to go faster and to go bigger,” Toni Reid said. “You can regulate yourself quite easily or think about what you’re going to do with your existing resources…. Sometimes, you don’t know what the boundaries are. Jeff just wanted us to be unbounded.”

  * * *

  But there were drawbacks to the frenetic speed and growth. For years the Alexa smartphone app looked like something a design student had come up with during a late-night bender. Setting up an Echo, or networking Echos throughout the home, was more complicated than it needed to be. It was also difficult and confusing for users to phrase commands in the right way to trigger third-party skills and specialized features.

  The decentralized and chaotic approach of countless two-pizza teams run by single-threaded leaders was manifested in aspects of the product that had become overly complex. Basic tasks, like setting up a device and connecting it to smart home appliances, had become “painful, very painful to the customer,” agreed Tom Taylor, a sardonic and even-keeled Amazon executive who took over from Mike George as leader of the Alexa unit in 2017. He set out to “find all the places that customers are suffering from our organizational structure.”

  There was plenty of turmoil that Taylor and his colleagues couldn’t quell. In March 2018, a bug caused Alexas around the world to randomly emit crazed, unprompted laughter. A few months later, an Echo inadvertently recorded the private conversation of a couple in Portland, Oregon, and inexplicably sent the recording to one of the husband’s employees in Seattle, whose phone number was in his address book. Amazon said the device thought it had heard its wake word and then a series of commands to record and forward the conversation. It was an “extremely rare occurrence,” the company said, and that, “as unlikely as this string of events is, we are evaluating options to make this case even less likely.” After those incidents, employees had to write a “correction of error” report, which analyzes an incident in detail and tries to get to its underlying root causes by going through a series of iterative questions and answers called “the five whys.” The memo went all the way to Bezos, describing what had happened and recommending how the process that created the problem in the first place could be fixed.

  Some errors could not be undone, such as Alexa’s penchant for assassinating Santa Claus in the minds of younger users. One such incident took place during the Alexa Prize, an Amazon contest among universities to build artificial chatbots that could carry on a sophisticated multipart conversation. When Alexa users said, “Alexa, let’s chat,” they got to talk to one of the chatbots and rate its performance. During the first competition in 2017, the chatbot from the University of Washington was retrieving some of its answers from Reddit, an online discussion board, and errantly informed a child that Santa was a myth. The parents complained and the chatbot was temporarily pulled from the rotation (but later won the $500,000 grand prize).

  The periodic problems with Alexa underscored how far it had come and how far it still had to go. By 2019, Amazon had sold more than 100 million Echo devices. In the span of a decade, a product spawned by Bezos’s love of science fiction and infatuation with invention had become a universally recognized product whose miscues and challenges to conventional notions of privacy were widely covered by the media.

  Yet Alexa still wasn’t conversational, in the way Bezos and Rohit Prasad had originally hoped. And though it had spawned a small cottage industry of startups and other companies pinning their hopes on voice-enabled services and devices, not many people used Alexa’s third-party add-ons or “Skills,” and developers still weren’t seeing much revenue, compared to the way they did on the app stores of Apple and Google.

  Bezos fervently believed all that would come in the next few years. Awed employees and Amazon fans who had watched him visualize Alexa and will it into existence believed the CEO could practically see the future. But in at least one respect, he did not.

  In 2016, he was reviewing the Echo Show, the first Alexa device with a video screen. Executives who worked on the project recalled that on several occasions when Bezos demoed the prototype, he spent the first few minutes asking Alexa to play videos that ridiculed a certain GOP presidential candidate.

  “Alexa, show me the video, ‘Donald Trump says “China,” ’ ” he asked, or “Alexa, play Stephen Colbert’s monologue from last night.” Then “he would laugh like there’s no tomorrow,” said a vice president who was in the demos.

  Bezos had no idea what was coming.

  CHAPTER 2 A Name Too Boring to Notice

  In November 2012, when Donald Trump was still the host of a reality TV show and Alexa prototypes were about to start moving into the homes
of employees, Jeff Bezos was asked by the TV interviewer Charlie Rose a question that had become a recurring favorite of journalists: Will Amazon ever buy or open physical stores? “Only if we can have a truly differentiated idea,” he replied. “We want to do something that is uniquely Amazon. We haven’t found it yet, but if we can find that idea, we would love to.”

  The answer was only partially true. Because inside Amazon, a small team was already converging on a novel concept for a chain of physical stores, under the direction of Bezos himself. They were getting ready to make what would turn out to be one of the most quixotic and expensive bets in the company’s history.

  At the time, Bezos was not only observing how advancements in processing power and decreases in computing costs were helping computers understand human speech. He was also tracking the potential for computers with cameras to actually see—to recognize and understand images and video. Earlier that year, he had circulated among Amazon’s senior engineers an article in the New York Times that described how a Google supercomputer had pored over ten million images and taught itself to recognize cats. “Jeff had faith that this was a really important trend that we should pay attention to,” said Joseph Sirosh, chief technology officer of Amazon’s retail business at the time. “Just as he got really enthusiastic about computer voice recognition, he was also really excited about computer vision.”

  The allure of computer vision, along with his interest in pressing Amazon’s advantage in the cloud to push the frontiers of artificial intelligence, again sparked the fertile imagination of Amazon’s founder. More than 90 percent of retail transactions were conducted in physical stores, according to the U.S. Census Bureau. Perhaps there was a way to tap this vast reservoir of sales with a completely self-service physical store that harnessed emerging technologies like computer vision and robotics.

  In 2012, Bezos pitched this broad idea at an off-site meeting to the S-team. Bezos alone handpicked members for the leadership council, and they held such brainstorm sessions every year, usually at some nearby corporate retreat, to spark new ideas and reaffirm the importance of “thinking big.” Members were required to write a paper with an inventive idea that might expand Amazon’s business.

  Judging from the executives he assigned to follow up on his challenge, Bezos considered the opportunity of entering physical retail with a self-service store to be significant. To lead the project, he appointed Steve Kessel, the deputy who had started the Kindle business nearly a decade before. Kessel, a Dartmouth graduate and recreational hockey player who had worked for Amazon since 1999, had the first discussions about the job while he was on sabbatical with his family in the south of France. His new task, in Amazonspeak, was to have a “single-threaded focus” on creating a new line of innovative stores. To manage the project, Kessel in turn lured back Gianna Puerini, a vice president who over the years had run Amazon’s home page and recommendations businesses.

  Puerini, the wife of S-team member Brian Valentine, was happily retired at the time, restoring and flipping houses in the Seattle area, and had no plans or financial need to return to work. She said she found Kessel’s pitch compelling. “When I asked Steve ‘Why me?’ a key part of his answer was that while we had a lot in common, he thought we would approach problems from different angles and look at things in different ways,” Puerini said. “I loved that he acknowledged the diversity of perspective and thought process…. I think I emailed Steve that night and said, ‘I’m in!’ ”

  Bezos’s technical advisor at the time, Dilip Kumar, the successor to Greg Hart in the coveted aide-de-camp role, would join Kessel and Puerini in early 2013 to run engineering. Because Bezos thought traditional retailers played their appointed roles well, the group had to meet a high bar before they could proceed. “Jeff was very particular that he didn’t want to just build any store. He wanted the store to be disruptive—something that no one had attempted before, something that would change the way brick-and-mortar retail had been done for hundreds of years,” said Bali Raghavan, who joined as one of the first engineering directors.

  The project was to be kept a secret even from other Amazon employees. So the team set up shop above a sporting goods store in a nondescript six-floor building on Westlake Avenue. One of Puerini’s first tasks was selecting a code name so boring that no one would pay any attention to them. For the next few years, the team would go by the initials, IHM, short for the nonsensical “Inventory Health Management.” Much later, after years of laborious progress toward their ambitious goal, the project would be known by the name of the peculiar store they would create and try to bring to nearly every major city in North America: Amazon Go.

  In those early weeks of brainstorming, the IHM crew considered whether they should develop Macy’s-style department stores, electronics stores, or Walmart-style supercenters. Bezos had no particular opinion about what they should sell, just that he wanted to disrupt traditional retail. One discarded idea involved two-floor outlets, with mobile robots swarming over an upper level packed with merchandise. Conveyor belts and other robots would then deliver them to customers’ waiting vehicles below.

  Amazon executives like to say and repeat compulsively that they start with the needs of the customer and “work backwards.” Ruminating on the act of shopping in regular stores, Puerini’s team made lists of their advantages, such as the instant gratification of walking out with desired items. They also made a list of the drawbacks, chief among them the frustration of waiting in checkout lines. People “are busy. They probably have something they’d rather be doing,” Puerini said.

  After months of research into customer needs and feasible technology, Amazon’s crack team of type A disrupters felt that the waiting problem was one they could solve with technology. The PR FAQs from the time—with Bezos’s handwriting scrawled in the margin, according to people who saw drafts—coined a trademark for a system that didn’t yet exist: “Just Walk Out technology.” Having envisioned the outcome, they would now try to invent a system that would allow shoppers to select items from shelves and get automatically charged without ever queueing up to pay.

  An excited Bezos approved the approach in 2013, not knowing that it would take five arduous and expensive years of research to bring it to fruition. “I think in the beginning, even the scientists were a little suspicious whether or not they could actually pull the thing off,” said Doug Herrington, the Amazon senior vice president in charge of the North American e-commerce business.

  The Amazon Go engineers initially considered using RFID chips in the product packaging to track which items were removed from shelves or asking customers to use their smartphones to scan product barcodes. But Bezos didn’t want them to take an easy path; he wanted them to innovate in the field of computer vision, which he saw as important to Amazon’s future. So they settled on the idea of cameras in the ceiling and algorithms behind the scenes that would try to spot when customers selected products and charge them for it. Scales hidden inside the shelves would provide another reliable sensor to determine when products were being removed and corroborate who was buying what.

  Over the next few years, Dilip Kumar recruited experts from outside Amazon, such as the University of Southern California’s renowned computer vision scientist, Gérard Medioni, as well as engineers from inside who worked on complex technologies like Amazon’s pricing algorithms. They were entering the hottest crucible at Amazon—a Jeff project, one that had the attention of the ever-curious and demanding boss. During a normal week, they would put in seventy to eighty hours, pushing against impending deadlines and the boundaries of science. At nights and over the weekends, they answered email, wrote six-page narrative memos, and like their counterparts on the concurrent Alexa and Fire Phone projects, prepared incessantly for regular reviews with Bezos. “We were all living in a cave,” said engineering director Bali Raghavan.

  Toward the end of 2013, they decided to focus on groceries. Americans on average shopped for clothes and electronics just a few times a yea
r. They shopped for food an average 1.7 times a week in 2013, according to the Food Marketing Institute, magnifying the inconvenience of waiting in line. The Go team started to hire executives with grocery experience, who were asked not to change their LinkedIn profiles and were furnished with burner phones and credit cards with no link to Amazon. “In the beginning, it was very 007-like, and it felt cool and important,” said Steve Lamontagne, a veteran of the Albertsons and SuperValu grocery chains. “It’s a lonely way to work though, particularly not being able to leverage the contacts you’ve established over decades.”

  The Go team presented every few weeks to Jeff Bezos. One notable meeting took place late in the day on June 24, 2014. Team members remember it because Amazon reported weak quarterly earnings that day and its stock sank 10 percent—the biggest drop in a year when Amazon was beset by the failure of the Fire Phone and unusually tepid sales growth. But Bezos was unperturbed. While he could be a remorseless boss capable of terrifying employees when they failed to meet his exacting standards, he seemed to have an unusual wellspring of patience for those at the company who practiced the challenging art of invention. “If there was any time the guy should have been agitated, that was it,” said Lamontagne. “Every time I was in a room with him, he never asked us, ‘How much is this going to cost me?’ or ‘Can we make money in X amount of time?’ He would look at us and say, ‘I know this is really hard and there is a lot of fatigue that comes with inventing something new. You’re heading in the right direction.’ ”

  Go execs envisioned large-scale stores of about thirty thousand square feet, roughly the size of a suburban supermarket. After a few months, they decided such a megamarket was overly ambitious and cut the proposed store in half. The midsized grocery store would offer a mix of services, not only rows of shelves with grab-and-go items, but counters with cheesemongers, baristas, and meat butchers. Employees envisioned a warm and welcoming experience and became attached to the idea of selling hot meals and coffee. Puerini’s team designed the first concept store in one of their conference rooms, using children’s blocks and standard-issue Amazon bookshelves and door desks to conceptualize how customers might behave in such an environment.