Machines of Loving Grace

by John Markoff

  Decades earlier, when Raibert began his graduate studies at MIT, he had set out to study neurophysiology. One day he followed a professor back to the MIT AI Lab. He walked into a room where one of the researchers had a robot arm lying in pieces on the table. Raibert was captivated. From then on he wanted to be a roboticist. Several years later, as a newly minted engineer, Raibert got a job at NASA’s Jet Propulsion Laboratory in Pasadena. When he arrived, he felt like a stranger in a strange land. Robots, and by extension their keepers, were definitely second-class citizens compared to the agency’s stars, the astronauts. JPL had hired the brand-new MIT Ph.D. as a junior engineer into a project that proved to be stultifyingly boring.

  Out of self-preservation, Raibert started following the work of Ivan Sutherland, who by 1977 was already a legend in computing. Sutherland’s 1962 MIT Ph.D. thesis project “Sketchpad” had been a major step forward in graphical and interactive computing, and he and Bob Sproull codeveloped the first virtual reality head-mounted display in 1968. Sutherland went to Caltech in 1974 as founding chair of the university’s new computer science department, where he was instrumental in working with physicist Carver Mead and electrical engineer Lynn Conway on a new model for designing and fabricating integrated circuits with hundreds of thousands of logic elements and memory—a 1980s advance that made possible the modern semiconductor industry.

  Alongside his older brother Bert, Sutherland had actually come to robotics in high school, during the 1950s. The two boys had the good fortune to be tutored by Edmund C. Berkeley, an actuary and computing pioneer who had written Giant Brains, or Machines That Think in 1949. In 1950, Berkeley had designed Simon, which, although it was constructed with relays and a total memory of four two-bit numbers, could arguably be considered the first personal computer.2 The boys modified it to do division. Under Berkeley’s guidance, the Sutherland brothers worked on building a maze-solving mouselike robot and Ivan built a magnetic drum memory that was capable of storing 128 two-bit numbers for a high school science project, which got Ivan a scholarship to Carnegie Institute of Technology.

  Once in college, the brothers continued to work on a “mechanical animal.” They went through a number of iterations of a machine called a “beastie,” which was based on dry cell batteries and transistors and was loosely patterned after Berkeley’s mechanical squirrel named Squee.3 They spent endless hours trying to program the beastie to play tag.

  Decades later, as the chair of Caltech’s computer science department in the 1970s, Sutherland, long diverted into computer graphics, had seemingly left robot design interests behind him. When Raibert heard Sutherland lecture, he was riveted by the professor’s musings on what might soon be possible in the field. Raibert left the auditorium feeling entirely fired up. He set about breaking down the bureaucratic wall that protected the department chair by sending Sutherland several polite emails, and also leaving a message with his secretary.

  His initial inquiries ignored, Raibert became irritated. He devised a plan. For the next two and a half weeks, he called Sutherland’s office every day at two P.M. Each day the secretary answered and took a message. Finally a gruff Sutherland returned his call. “What do you want?” he shouted. Raibert explained that he was anxious to collaborate with Sutherland and wanted to propose some possible projects. When they finally met in 1977, Raibert had prepared three ideas and Sutherland, after listening to the concept of a one-legged walking—hopping, actually—robot, brusquely declared: “Do that one!”

  Sutherland would become Raibert’s first rainmaker. He took him along on a visit to DARPA (where Sutherland had worked for two years just after Licklider) and to the National Science Foundation, and they came away with a quarter million dollars in research funding to get the project started. The two worked together on early walking robots at Caltech, and several years later Sutherland persuaded Raibert to move with him to Carnegie Mellon, where they continued with research on walking machines.

  Ultimately Raibert pioneered a remarkable menagerie of robots that hopped, walked, twirled, and even somersaulted. The two had adjoining offices at CMU and coauthored an article on walking machines for Scientific American in January 1983. Raibert would go on to set up the Leg Laboratory at CMU in 1981 and then move the laboratory to MIT while he held a faculty position there from 1986 to 1992. He left MIT to found Boston Dynamics. Another young MIT professor, Gill Pratt, would continue to work in the Leg Lab, designing walking machines and related technologies enabling robots to work safely in partnership with humans.

  Raibert pioneered walking machines, but it was his CMU colleague Red Whittaker who almost single-handedly created “field robotics,” machines that moved freely in the physical world. DARPA’s autonomous vehicle contest had its roots in Red Whittaker’s quixotic scheme to build a machine that could make its way across an entire state. The new generation of mobile walking rescue robots had their roots in the work that he did in building some of the first rescue robots three and a half decades ago.

  Whittaker’s career took off with the catastrophe at Three Mile Island Nuclear Generating Station on March 28, 1979. He had just received his Ph.D. when there was a partial meltdown in one of the two nuclear reactors at the site. The crisis exposed how unprepared the industry was to cope with the loss of control of a reactor’s radioactive fuel. It would be a half decade before robots built by Whittaker and his students would enter the most severely damaged areas of the reactor and help with the cleanup.

  Whittaker’s opportunity came when two giant construction firms, having spent $1 billion, failed to get into the basement of the crippled reactor to inspect it and begin the cleanup. Whittaker sent the first CMU robot, which his team assembled in six months and dubbed “Rover,” into Three Mile Island in April of 1984. It was a six-wheeled contraption outfitted with lights, and a camera tethered to its controller. It was lowered into the basement, where it traversed water, mud, and debris, successfully gathering the first images of consequences of the disaster. The robot was later modified to perform inspections and conduct sampling.4

  The success of this robot set the tone for Whittaker’s can-do style of tackling imposing problems. After years of bureaucratic delays, his first company, Redzone Robotics, supplied a robot to help with the cleanup at Chernobyl, the 1986 nuclear power plant disaster in Ukraine. By the early 1990s Whittaker was working on a Mars robot for NASA. The Mars robot was large and heavy, so it was unlikely to make the first mission. Instead, Whittaker plotted to find an equally dramatic project back on Earth. Early driverless vehicle research was beginning to show promise, so the CMU researchers started experimenting letting vehicles loose on Pittsburgh’s streets. What about driving across an entire state? Whittaker thought that that idea, which he called “the Grand Traverse,” would prove that robots were ready to perform in the real world and not just in the laboratory. “Give me two years and a half-dozen graduate students and we could make it happen,” he boasted to the New York Times in 1991.5 A decade and a half later at DARPA, Tony Tether lent credence to this idea by underwriting the first autonomous vehicle Grand Challenge.

  Although the roboticists finally made rapid progress in building useful robots in the early 1990s, it was only after decades of disappointment. The technology failure at Three Mile Island initially cast a pall over the robotics industry. In the June 1980 issue of Omni magazine, Marvin Minsky wrote a long manifesto calling for the development of telepresence technologies—mobile robots outfitted with video cameras, displays, microphones, and speakers that allow their operator to be “present” from a remote location anywhere in the connected world. Minsky used his manifesto to rail against the shortcomings of the world of robotics:

  Three Mile Island really needed telepresence. I am appalled by the nuclear industry’s inability to deal with the unexpected. We all saw the absurd inflexibility of present day technology in handling the damage and making repairs to that reactor. Technicians are still waiting to conduct a thorough inspection of the damaged plant—and to
absorb a year’s allowable dose of radiation in just a few minutes. The cost of repair and the energy losses will be $1 billion; telepresence might have cut this expense to a few million dollars.

  The big problem today is that nuclear plants are not designed for telepresence. Why? The technology is still too primitive. Furthermore, the plants aren’t even designed to accommodate the installation of advanced telepresence when it becomes available. A vicious circle!6

  The absence of wireless networking connectivity was the central barrier to the development of remote-controlled robots at the time. But Minsky also focused on the failure of the robotics community to build robots with the basic human capabilities to grasp, manipulate, and maneuver. He belittled the state of the art of robotic manipulators used by nuclear facility operators, calling them “little better than pliers” and noted that they were not a match for human hands. “If people had a bit more engineering courage and tried to make these hands more like human hands, modeled on the physiology of the palm and fingers, we could make nuclear reactor plants and other hazardous facilities much safer.”7

  It was an easy criticism to make, yet when the article was reprinted three decades later in IEEE Spectrum in 2010, the field had made surprisingly little progress. Robotic hands like those Minsky had called for still did not exist. In 2013 Minsky bemoaned the fact that even at the 2011 Fukushima meltdowns, there wasn’t yet a robot that could easily open a door in an emergency. It was also clear that he remained bitter over the fact that the research community had largely chosen the vision charted by Rod Brooks, which involved hunting for emergent complex behaviors by joining simple components.

  One person who agreed with Minsky was Gill Pratt, who had taken over as director of the MIT Leg Lab after Marc Raibert. Later a professor and subsequently dean at Olin College in Needham, Massachusetts, Pratt arrived at DARPA in early 2010 as a program manager in charge of two major programs. One, the ARM program, for Autonomous Robotic Manipulation, involved building the robotic hands whose absence Minsky had noted. ARM hands were specified to possess a humanlike functionality for a variety of tasks: picking up objects, grasping and controlling tools designed for humans, and operating a flashlight. A second part of ARM funded efforts to connect the human brain to robotic limbs, which would give wounded soldiers and the disabled—amputees, paraplegics, and quadriplegics—new freedoms. A parallel project to ARM, called Synapse, focused on developing biologically inspired computers that could better translate a machine’s perception into robotic actions.

  Pratt represented a new wave at DARPA, arriving shortly after the Obama administration had replaced Tony Tether with Regina Dugan as the agency’s director. Tether had moved DARPA away from its historically close relationship with academia by shifting funding to classified military contractors. Dugan and Pratt tried to repair the damage by quickly reestablishing closer relations with university campuses. Pratt’s research before arriving at DARPA had focused on building robots that could navigate the world outside of the lab. The challenge was giving the robots practical control over the relatively gentle forces that they would encounter in the physical world. The best way to do this, he found, was to insert some elastic material between the robot’s components and the gear train that drives them. The approach tried to mimic the function played by biological tendons located between a muscle and a joint. The springy tendon material can stretch and, when measured, indicates how much force was being applied to it. Until then the direct mechanical connection between the components that made up the arms and legs used by robots gave them both power and precision that was too inflexible—and potentially dangerous—for navigating the unpredictable physical world populated by vulnerable and litigious humans.

  Pratt had not initially considered human-robot collaboration. Instead, he was interested in how the elderly safely move about in the world. Typically, the infirm used walkers and wheelchairs. As he explored the contact between humans and the tools they use, he realized that elasticity offered the humans a measure of protection against unyielding obstacles. More elastic robots, Pratt concluded, could make it possible for humans to work close to the machines without fear of being injured.

  They were working with Cog, an early humanoid robot designed by Rodney Brooks’s robot laboratory during the 1990s. A graduate student, Matt Williamson, was testing the robot’s arm. A bug in the code caused the arm to repeatedly slap the test fixture. When Brooks inserted himself between the robot and the test bench, he became the first human to ever be spanked by a robot. It was a gentle whipping and—fortunately for his graduate students—Brooks survived. Pratt’s research was an advance both in biomimicry and human-robot collaboration. Brooks adopted “elastic actuation” as a central means of making robots safe for people to work with.

  When Pratt arrived at DARPA he was keenly aware that despite decades of research, most robots were still kept inside labs, not just for human safety but also to protect the robot’s software from an uncontrolled environment. He had been at DARPA for a little more than a year when on March 12, 2011, the tsunami struck the Fukushima Daiichi Nuclear Power Plant. The workers inside the plant had been able to control the emergency for a short period, but then high radiation leakage forced them to flee before they could oversee a safe shutdown of the reactors. DARPA became peripherally involved in the crisis because humanitarian assistance and disaster relief is a Pentagon responsibility. (The agency tried to help out in the wake of the 9/11 attacks by sending robots to search for survivors at the World Trade Center.) DARPA officials coordinated a response at Fukushima by contacting U.S. companies who had provided assistance at Three Mile Island and Chernobyl. A small armada of U.S. robots was sent to Japan in an effort to get into the plant and make repairs, but by the time power plant personnel were trained to use them it was too late to avoid the worst damage. This was particularly frustrating because Pratt could see that a swift deployment of robots would almost certainly have been helpful and limited the damage. “The best the robots could do was help survey the extensive damage that had already occurred and take radiation readings; the golden hours for early intervention to mitigate the extent of the disaster had long since passed,” he wrote.8

  The failure led to the idea of the DARPA Robotics Challenge, which was announced in April 2012. By sponsoring a grand challenge on the scale of Tether’s autonomous vehicle contest, Pratt sought to spark innovations in the robotics community that would facilitate the development of autonomous machines that could operate in environments that were hostile for humans. Teams would build and program a robot to perform a range of eight tasks9 that might be expected in a power plant emergency, but most of them would not build the robots from scratch: Pratt had contracted with Boston Dynamics to supply Atlas humanoid robots as a joint platform to jump-start the competition.

  In the dark it is possible to make out the blue glow of an unblinking eye staring into the evening gloom. This light is a retina scanner that uses the eye as a digital fingerprint. These pricey electronic sentinels are not yet commonplace, but they do show up in certain ultra–high security locations. Passing beneath their gaze is a bit like passing before the unblinking eye of some cybernetic Cerberus. The scanner isn’t the only bit of info-security decor. The home itself is a garden of robotic delights. Inside in the foyer, a robotic arm gripping a mallet strikes a large gong to signal a new arrival. There are wheeled, flying, crawling, and walking machines everywhere. To a visitor, it feels like the scene in the movie Blade Runner in which detective Rick Deckard arrives at the home of the gene-hacker J. F. Sebastian and finds himself in a menagerie of grotesque, quirky synthetic creatures.

  The real-life J.F. lording over this lair is Andy Rubin, a former Apple engineer who in 2005 joined Google to jump-start the company’s smartphone business. At the time the world thought of Google as an unstoppable company, since it had rapidly become one of the globe’s dominant computing technology companies. Inside Google, however, the company’s founders were deeply concerned that their advan
tage in Web search, and thus their newly gained monopoly, might be threatened by the rapid shift away from desktop to handheld mobile computers. The era of desktop computing was giving way to a generation of more intimate machines in what would soon come to be known as the post-PC era. The Google founders were fearful that if Microsoft was able to replicate its desktop monopoly in the emerging world of phones, they would be locked out and would lose their search monopoly. Apple had not yet introduced the iPhone, so they had no way of knowing how fundamentally threatened Microsoft’s desktop stranglehold would soon be.

  In an effort to get ahead, Google acquired Rubin’s small start-up firm to build its own handheld software operating system as a defense against Microsoft. Google unveiled Android in November 2007, ten months after the iPhone first appeared. During the next half decade, Rubin enjoyed incredible success, displacing not just Microsoft, but Apple, Blackberry, and Palm Computing as well. His strategy was to build an open-source operating system and offer it freely to the companies who had once paid hefty licenses to Microsoft for Windows. Microsoft found it impossible to compete with free. By 2013 Google’s software would dominate the world of mobile phones in terms of market share.

  Early in his career, Rubin had worked at Apple Computer as a manufacturing engineer after a stint at Zeiss in Europe programming robots. He left Apple several years later with an elite group of engineers and programmers to build one of the early handheld computers at General Magic. General Magic’s efforts to seed the convergence of personal information, computing, and telephony became an influential and high-profile failure in the new mobile computing world.

  Andy Rubin went on a buying spree for Google when the company decided to develop next-generation robotics technologies. Despite planning a decade-long effort, he walked away after just a year. (Photo courtesy of Jim Wilson/New York Times/Redux)