Machines of Loving Grace

by John Markoff

  Tether’s automotive Grand Challenge drew garage innovators and eager volunteers out of the woodwork. In military terms it was a “force multiplier,” allowing the agency to get many times the innovation it would get from traditional contracting efforts. At its heart, however, the specific challenge that Tether chose to pursue had been cooked up more than a decade earlier inside the same university research community that he now disfavored. The guiding force behind the GM robot SUV that would win the Urban Challenge in 2007 was a Carnegie Mellon roboticist who had been itching to win this prize for more than a decade.

  In the fall of 2005, Tether’s second robot race through the California desert had just ended at the Nevada border and Stanford University’s roboticists were celebrating. Stanley, the once crash-prone computerized Volkswagen Touareg, had just pulled off a come-from-behind victory and rolled under a large banner before a cheering audience of several thousand.

  Just a few feet away in another tent, however, the atmosphere had the grim quality of a losing football team’s locker room. The Carnegie Mellon team had been the odds-on favorite, with two robot vehicle entries and a no-nonsense leader, a former marine and rock climber, William L. “Red” Whittaker. His team had lost the race due to a damnable spell of bad luck. Whittaker had barnstormed into the first DARPA race eighteen months earlier with another heavily funded GM Humvee, only to fail when the car placed a wheel just slightly off road on a steep climb. Trapped in the sand, it was out of the competition. Up to then, Whittaker’s robot had been head and shoulders above the others. So when he returned the second time with a two-car fleet and a squad of photo analysts to pore over the course ahead of the competition, he had easily been cast as the odds-on favorite.

  Once again, however, bad luck struck. His primary vehicle led Stanley until late in the race, when it malfunctioned, slowing dramatically and allowing the Stanford team to sail by and grab the $2 million prize. After the second loss, Whittaker stood in the tent in front of his team and gave an inspiring speech worthy of any college football coach. “On any Sunday . . .” he told his team, echoing the words of the losing coach. The loss was especially painful because the leaders of the Stanford team, Sebastian Thrun and Mike Montemerlo, were former CMU roboticists who had defected to Stanford, where they organized the rival, winning effort. Years later the loss still rankled. Outside of Whittaker’s office at the university is a portrait of the ill-fated team of robot car designers. In the hallway Whittaker would greet visitors and replay the failure in detail.

  The defeat was particularly striking because Red Whittaker had in many ways been viewed widely as the nation’s premier roboticist. By the time of the Grand Challenges he had already become a legend for designing robots capable of going places where humans couldn’t go. For decades he combined a can-do attitude with an adventurer’s spirit. His parents had both flown planes with a bit of barnstorming style. His father, an air force bomber pilot, sold mining explosives after the war. His mother, a chemist, was a pilot, too. When he was a young man, she had once flown him under a bridge.3

  His Pennsylvania upbringing led him to develop a style of robotics that pushed in the direction of using the machines primarily as tools to extend an adventurer’s reach, a style in the tradition of Yvon Chouinard, the legendary climber who designed and made his own climbing hardware, or Jacques Cousteau, the undersea explorer who made his own breathing equipment. With a degree in civil engineering from Princeton and a two-year tour as a marine sergeant, the six-foot-four Whittaker pioneered “field” robotics—building machines that left the laboratory and moved around in the world.

  In Red Whittaker’s robotic world, however, humans were still very much in the loop. In every case he used them to extend his reach as an adventurer. He had built machines used in nuclear power plant catastrophes at both Three Mile Island and Chernobyl. In the late 1980s he designed a huge nineteen-foot-tall robot called Ambler that was intended to walk on Mars. He sent a robot into a volcano and had been one of the first roboticists in the United States to explore the idea of an autonomous car as part of Carnegie Mellon’s Navlab project.

  “This is not the factory of the future,” he was fond of pointing out. “The ideas that make it in the factory don’t make it in the outside world.”4

  As a young man Whittaker had variously been a rower, wrestler, boxer, and mountain climber. His love of adventure had not been without personal pain, however. He spent a decade of his life rock climbing, sneaking away from his robot projects to spend time in Yosemite and the Himalayas. He even soloed the east wall of the Matterhorn in winter conditions. He had begun climbing casually as a member of a local explorer’s club in Pittsburgh. It only became a passion when he met another young climber, after seeing a notice on a bulletin board: “Expert climber willing to teach the right guy,” the note read, adding: “You must have a car.”

  The two would become inseparable climbing partners over the next decade.

  That magic time for Whittaker came to a sudden end one summer when they were climbing in Peru. His Pittsburgh friend was climbing with another young climber. The two were roped together and the younger man slipped and pulled both men down a tumbling set of ledges for almost a thousand feet. Whittaker, who was off-rope during the accident, was able to rescue the young climber, but his friend was killed by the fall. Whittaker returned to Pittsburgh shaken by the accident. It would take months before he mustered up the courage to go over to the home where the young man had lived with his parents and clean out the dead climber’s room.

  The death left its mark. Whittaker stopped climbing, but still hungered for some sort of challenging adventure. He began to build ever more exotic robots, capable of performing tasks ranging from simple exploration to sophisticated repair, to extend his adventures into volcanoes, and ultimately, perhaps, to the moon and Mars. Even when he had been climbing on Earth in the 1970s and 1980s, it was becoming more and more difficult to find virgin territory. With the possibility of “virtual exploration,” new vistas would open up indefinitely and Whittaker could again dream of climbing and rappelling, this time perhaps with a humanoid robot stand-in on another world.

  Whittaker redeemed his bitter loss to Stanford’s Stanley several years later in the third Grand Challenge, in 2007. His General Motors–backed “Boss” would win the final Urban Driving Challenge.

  One of the most enduring bits of Silicon Valley lore recalls how Steve Jobs recruited Pepsi CEO John Sculley to Apple by asking him if he wanted to spend the rest of his life selling sugar water. Though some might consider it naive, the Valley’s ethos is about changing the world. That is at the heart of the concept of “scale,” which is very much a common denominator in motivating the region’s programmers, hardware hackers, and venture capitalists. It is not enough to make a profit, or to create something that is beautiful. It has to have an impact. It has to be something that goes under 95 percent of the world’s Christmas trees, or offers clean water or electricity to billions of people.

  Google’s chief executive Larry Page took the Steve Jobs approach in recruiting Sebastian Thrun. Thrun was a fast-rising academic who had spent a sabbatical year at Stanford in 2001, which opened his eyes to the world that Silicon Valley offered beyond the walls of academia. There was more out there besides achieving tenure, publishing, and teaching students.

  He returned to Stanford as an assistant professor in 2003. He attended the first DARPA Grand Challenge as an observer. The self-driving car competition completely changed his perspective: he realized that there were great thinkers outside of his cloistered academic community who cared deeply about changing the world. In between, during his short return to CMU, he had sent a note to Whittaker offering to help their software effort, but was rebuffed. Thrun had brought a group of students with him from CMU when he returned to Stanford, including Mike Montemerlo, whose father was a NASA roboticist. Montemerlo gave a presentation on the first DARPA contest. At the end of his presentation his final slide asked, “Should we at Stanfor
d enter the Grand Challenge?” And then he answered his own question in a large font. “NO!” There were a dozen reasons not to do it. They would have no chance of winning, it was too hard, it would cost too much money. Thrun looked at Montemerlo and it was obvious that although on paper he was the quintessential pessimist, everything in his demeanor was saying yes.

  Sebastian Thrun (left) and Mike Montemerlo (right) in front of the Stanford University autonomous vehicle while it was being tested to take part in DARPA’s Urban Challenge in 2007. (Photo courtesy of the author)

  Soon afterward Thrun threw himself into the DARPA competition with passion. For the first time in his life he felt like he was focusing on something that was genuinely likely to have broad impact. Living in the Arizona desert for weeks on end, surviving on pizza, the team worked on the car until it was able to drive the backcountry roads flawlessly.

  Montemerlo and Thrun made a perfect team of opposites. Montemerlo was fundamentally conservative, and Thrun was extraordinarily risk-inclined. As head of software, Montemerlo would build his conservative assumptions into his programs. When he wasn’t looking, Thrun would go through the code and comment out the limitations to make the car go faster. It would infuriate the younger researcher. But in the end it was a winning combination.

  Larry Page had said to Thrun that if you really focus on something you can achieve amazing things. He was right. After Stanley captured the $2 million DARPA prize, Thrun took Page’s words to heart. The two men had become friends after Thrun helped the Google cofounder debug a home robot that Page had been tinkering with. Thrun borrowed the device and brought it back able to navigate inside Page’s home.

  Navigation, a necessity for autonomous robots, had become Thrun’s particular expertise. At CMU and later at Stanford he worked to develop SLAM, the mapping technique pioneered at Stanford Research Institute by the designers of the first mobile robots beginning in the 1960s. Thrun had helped make the technique fast and accurate and had paved the way for using it in autonomous cars. At Carnegie Mellon he had begun to attract national attention for a variety of mobile robots. In 1998 at the Smithsonian in D.C., he showcased Minerva, a mobile museum tour guide that was connected to the Web and could interact with museum guests and travel up to three and a half miles per hour. He worked with Red Whittaker to send robots into mines, which relied heavily on SLAM techniques. Thrun also tried to integrate mobile and autonomous robots in nursing and elder-care settings, with little success. It turned out to be a humbling experience, which gave him a deep appreciation of the limitations of using technologies to solve human problems. In 2002, in a team effort between the two universities, Thrun pioneered a new flavor of SLAM that was dubbed FastSLAM, which could be used in real-world situations where it was necessary to locate thousands of objects. It was an early example of a new wave of artificial intelligence and robotics that increasingly relied on probabilistic statistical techniques rather than on rule-based inference.

  At Stanford, Thrun would rise quickly to become director of the revitalized Stanford Artificial Intelligence Laboratory that had originally been created by John McCarthy in the 1960s. But he also quickly became frustrated by the fragmented life of an academic, dividing time between teaching, public speaking, grant writing, working on committees, doing research, and mentoring. In the wake of his 2005 DARPA Grand Challenge victory Thrun had also become more visible in high-technology circles. His talks described the mass atrocities committed by human drivers that resulted in more than one million killed and maimed each year globally. He personalized the story. A close friend had been killed in an automobile accident when Thrun was a high school student in his native Germany. Many people he was close to lost friends in accidents. More recently, a family member of a Stanford faculty secretary was crippled for life after a truck hit her car. In an instant she went from being a young girl full of life and possibility to someone whose life was forever impaired. Thrun’s change-the-world goals gave him a platform at places like the TED Conference.

  After building two vehicles for the DARPA Challenge contests, he decided to leave Stanford. Page offered him the opportunity to do things at “Google scale,” which meant that his work would touch the entire world. He secretly set up a laboratory modeled vaguely on Xerox PARC, the legendary computer science laboratory that was the birthplace of the modern personal computer, early computer networks, and the laser printer, creating projects in autonomous cars and reinventing mobile computing. Among other projects, he helped launch Google Glass, which was an effort to build computing capabilities including vision and speech into ordinary glasses.

  Unlike laboratories of the previous era that emphasized basic science, such as IBM Research and Bell Labs, Google’s X Lab was closer in style to PARC, which had been established to vault the copier giant, restyled “the Document Company,” into the computer industry—to compete directly with IBM. The X Lab was intended to push Google into new markets. Google felt secure in its Web search monopoly so, with a profit stream that by the end of 2013 was more than $1 billion a month, the search company funded ambitious R & D projects that might have nothing to do with the company’s core business. Google was famous for its 70-20-10 rule, which gave its engineers free time to pursue their own side projects. Employees are supposed to spend 10 percent of their time on projects entirely unrelated to the company’s core business. Its founders Sergey Brin and Larry Page believed deeply in thinking big. They called their efforts “moon shots”: not pure science, but research projects that were hopefully destined to have commercial rather than purely scientific impact.

  It was a perfect environment for Thrun. His first project in 2008 had been to create the company’s fleet of Street View cars that systematically captured digital images of homes and businesses on every street in the nation. The next year he began an even more ambitious effort: a self-driving car that would travel on public streets and highways. He was both cautious and bold in the car project. A single accident might destroy the Google car, so at the outset he ensured that a detailed safety regime was in place. He was acutely aware that if there was any indication in the program that Google had not been incredibly careful, it would be a disaster. He never let an untrained driver near the wheel of the small Toyota Prius fleet on which the system was being developed. The cars would eventually drive more than a half-million miles without an accident, but Thrun understood that even a single error every fifty thousand to a hundred thousand miles was too high an error rate. At the same time he believed that there was a path forward that would allow Google to redefine what it meant to be in a car.

  Like the automotive industry, Thrun and his team believed in the price/volume curve, which suggested that costs would go down the more a company manufactured a particular thing. Sure, today a single experimental lidar laser radar might cost tens of thousands of dollars, but the Google engineers had faith that in a few years it would be so cheap that it would not be a showstopper in the bill of materials of some future car. In the trade-off between cost and durability, Thrun always felt it would make sense to design and build more reliable systems now and depend on mass manufacturing technologies for price reductions to kick in later. The pricey laser guidance systems didn’t actually contain that many parts, so there was little reason to believe that prices couldn’t come down rapidly. It had already happened with radar, which had once been an esoteric military and aviation technology but in recent years had begun showing up in motion detectors and premium automobiles.

  Thrun evinced an engineer’s worldview and tended toward a libertarian outlook. He held a pro-business point of view that the global corporation was an evolutionary step beyond the nation-state. He also subscribed to the belief, commonplace in the Valley, that within three decades as much as 90 percent of all jobs will be made obsolete by advancing AI and robotic technologies. Indeed, Thrun believed that most people’s jobs are actually pretty useless and unfulfilling. There are countless manual labor jobs—everything from loading and unloading trucks to driving th
em—that could vanish over the coming decade. He also believed that much of the bureaucratic labor force is actively counterproductive. Those people make other people’s work harder. Thrun had a similar contempt for what he perceived as Detroit’s hidebound car industry that could have easily used technology to radically reshape transportation systems and make them safer, but did little and was content to focus on changing the shape of a car’s tail fins each year. By 2010 he had a deep surprise in store for an industry that did not change easily and was largely unfamiliar with Silicon Valley culture.5

  The DARPA races created ripples in Detroit, the cradle of the American automotive industry, but the industry kept to its traditional position that cars were meant to be driven by people and should not operate autonomously. By and large the industry had generally resisted computer technology. Many car manufacturers adhered to a “computers are buggy” philosophy. However, engineers elsewhere in the country were beginning to think about transportation through the new lens of cheap sensors, the microprocessor, and the Internet.

  In the spring of 2010, rumors about an experimental Google car began to float around Silicon Valley. Initially they sounded preposterous. The company, nominally a provider of Internet search, was supposedly hiding the cars in plain sight. Google engineers, so the story went, had succeeded in robotically driving from San Francisco to Los Angeles on freeways at night! The notion immediately elicited both guffaws and pointed reminders that such an invention would be illegal, even if it was possible. How could they get away with something so crazy?

  Of course, Google’s young cofounders Sergey Brin and Larry Page had by then perfected a public image for wild schemes based on AI and other futuristic technologies to transform the world. Eric Schmidt, the company’s chief executive officer beginning in 2001, would tell reporters that his role was one of adult supervision—persuading the cofounders which of their ideas should be kept above and which below the “bar.” The cofounders famously considered the idea of a space elevator. New, incredibly strong material had recently been developed, and this material was so strong that, rather than using a rocket, it would be possible to build a cable that reached from the Earth into orbit to inexpensively hoist people and materials into space. When queried about the idea Schmidt would pointedly state that this was one of the ideas that was being considered, but was—for the moment at least—“below the bar.”