by John Markoff
Licklider, like McCarthy a half decade later, was confident that the advent of “Strong” artificial intelligence—a machine capable of at least matching wits and self-awareness with a human—was likely to arrive relatively soon. The period of man-machine “symbiosis” might only last for less than two decades, he wrote, although he allowed that the arrival of truly smart machines that were capable of rivaling thinking humans might not happen for a decade, or perhaps fifty years.
Ultimately, although he posed the question of whether humans will be freed or enslaved by the Information Age, he chose not to directly address it. Instead he drew a picture of what has become known as a “cyborg”—part human, part machine. In Licklider’s view human operators and computing equipment would blend together seamlessly to become a single entity. That vision has since been both celebrated and reviled. But it still begs the unanswered question—will we be masters, slaves, or partners of the intelligent machines that are appearing today?
Consider the complete spectrum of human-machine interactions from simple “FAQbots” to Google Now and Apple’s Siri. Moving into the unspecified future in the movie Her, we see an artificial intelligence, voiced by Scarlett Johansson, capable of carrying on hundreds of simultaneous, intimate, human-level conversations. Google Now and Siri currently represent two dramatically different computer-human interaction styles. While Siri intentionally and successfully mimics a human, complete with a wry sense of humor, Google Now opts instead to function as a pure information oracle, devoid of personality or humanity.
It is tempting to see the personalities of the two competing corporate chieftains in these contrasting approaches. At Apple, Steve Jobs saw the potential in Siri before it was even capable of recognizing human speech and focused his designers on natural language as a better way to control a computer. At Google, Larry Page, by way of contrast, has resisted portraying a computer in human form.
How far will this trend go? Today it is anything but certain. Although we are already able to chatter with our cars and other appliances using limited vocabularies, computer speech and voice understanding is still a niche in the world of “interfaces” that control the computers that surround us. Speech recognition clearly offers a dramatic improvement in busy-hand, busy-eye scenarios for interacting with the multiplicity of Web services and smartphone applications that have emerged. Perhaps advances in brain-computer interfaces will prove to be useful for those unable to speak or when silence or stealth is needed, such as card counting in blackjack. The murkier question is whether these cybernetic assistants will eventually pass the Turing test, the metric first proposed by mathematician and computer scientist Alan Turing to determine if a computer is “intelligent.” Turing’s original 1951 paper has spawned a long-running philosophical discussion and even an annual contest, but today what is more interesting than the question of machine intelligence is what the test implies about the relationship between humans and machines.
Turing’s test consisted of placing a human before a computer terminal to interact with an unknown entity through typewritten questions and answers. If, after a reasonable period, the questioner was unable to determine whether he or she was communicating with a human or a machine, then the machine could be said to be “intelligent.” Although it has several variants and has been widely criticized, from a sociological point of view the test poses the right question. In other words, it is relevant with respect to the human, not the machine.
In the fall of 1991 I covered the first of a series of Turing test contests sponsored by a New York City philanthropist, Hugh Loebner. The event was first held at the Boston Computer Museum and attracted a crowd of computer scientists and a smattering of philosophers. At that point the “bots,” software robots designed to participate in the contest, weren’t very far advanced beyond the legendary Eliza program written by computer scientist Joseph Weizenbaum during the 1960s. Weizenbaum’s program mimicked a Rogerian psychologist (a human-centered form of psychiatry focused on persuading a patient to talk his or her way toward understanding his or her actual feelings) and he was horrified to discover that his students had become deeply immersed in intimate conversations with his first, simple bot.
But the judges for the original Loebner contest in 1991 fell into two broad categories: computer literate and computer illiterate. For human judges without computer expertise, it turned out that for all practical purposes the Turing test was conquered in that first year. In reporting on the contest I quoted one of the nontechnical judges, a part-time auto mechanic, saying why she was fooled: “It typed something that I thought was trite, and when I responded it interacted with me in a very convincing fashion,”5 she said. It was a harbinger of things to come. We now routinely interact with machines simulating humans and they will continue to improve in convincing us of their faux humanity.
Today, programs like Siri not only seem almost human; they are beginning to make human-machine interactions in natural language seem routine. The evolution of these software robots is aided by the fact that humans appear to want to believe they are interacting with humans even when they are conversing with machines. We are hardwired for social interaction. Whether or not robots move around to assist us in the physical world, they are already moving among us in cyberspace. It’s now inevitable that these software bots—AIs, if only of limited capability—will increasingly become a routine part of daily life.
Intelligent software agents such as Apple’s Siri, Microsoft’s Cortana, and Google Now are interacting with hundreds of millions of people, by default defining this robot/human relationship. Even at this relatively early stage Siri has a distinctly human style, a first step toward the creation of a generation of likable and trusted advisors. Will it matter whether we interact with these systems as partners or keep them as slaves? While there is an increasingly lively discussion about whether intelligent agents and robots will be autonomous—and if they are autonomous, whether they will be self-aware enough that we need to consider questions of “robot rights”—in the short term the more significant question is how we treat these systems and what the design of those interactions says about what it means to be human. To the extent that we treat these systems as partners it will humanize us. Yet the question of what the relationship between humans and machines will be has largely been ignored by much of the modern computing world.
Jonathan Grudin, a computer scientist at Microsoft Research, has noted that the separate disciplines of artificial intelligence and human-computer interaction rarely speak to one another.6 He points to John McCarthy’s early explanation of the direction of artificial intelligence research: “[The goal] was to get away from studying human behavior and consider the computer as a tool for solving certain classes of problems. Thus AI was created as a branch of computer science and not as a branch of psychology.”7 McCarthy’s pragmatic approach can certainly be justified by the success the field has had in the past half decade. Artificial intelligence researchers like to point out that aircraft can fly just fine without resorting to flapping their wings—an argument that asserts that to duplicate human cognition or behavior, it is not necessary to comprehend it. However, the chasm between AI and IA has only deepened as AI systems have become increasingly facile at human tasks, whether it is seeing, speaking, moving boxes, or playing chess, Jeopardy!, or Atari video games.
Terry Winograd was one of the first to see the two extremes clearly and to consider the consequences. His career traces an arc from artificial intelligence to intelligence augmentation. As a graduate student at MIT in the 1960s, he focused on understanding human language in order to build a software equivalent to Shakey—a software robot capable of interacting with humans in conversation. Then, during the 1980s, in part because of his changing views on the limits of artificial intelligence, he left the field—a shift in perspective moving from AI to IA. Winograd walked away from AI in part because of a series of challenging conversations with a group of philosophers at the University of California. A member of a small
group of AI researchers, he engaged in a series of weekly seminars with Berkeley philosophers Hubert Dreyfus and John Searle. The philosophers convinced him that there were real limits to the capabilities of intelligent machines. Winograd’s conversion coincided with the collapse of a nascent artificial intelligence industry known as the “AI Winter.” Several decades later, Winograd, who was faculty advisor for Google cofounder Larry Page at Stanford, famously counseled the young graduate student to focus on the problem of Web search rather than self-driving cars.
In the intervening decades Winograd had become acutely aware of the importance of the designer’s point of view. The separation of the fields of AI and human-computer interaction, or HCI, is partly a question of approach, but it’s also an ethical stance about designing humans either into or out of the systems we create. More recently at Stanford Winograd helped create an academic program focusing on “Liberation Technologies,” which studies the construction of computerized systems based on human-centered values.
Throughout human history, technology has displaced human labor. Locomotives and tractors, however, didn’t make human-level decisions. Increasingly, “thinking machines” will. It is also clear that technology and humanity coevolve, which again will pose the question of who will be in control. In Silicon Valley it has become fashionable to celebrate the rise of the machines, most clearly in the emergence of organizations like the Singularity Institute and in books like Kevin Kelly’s 2010 What Technology Wants. In an earlier book in 1994, Out of Control, Kelly came down firmly on the side of the machines. He described a meeting between AI pioneer Marvin Minsky and Doug Engelbart:
When the two gurus met at MIT in the 1950s, they are reputed to have had the following conversation:
Minsky: We’re going to make machines intelligent. We are going to make them conscious!
Engelbart: You’re going to do all that for the machines? What are you going to do for the people?
This story is usually told by engineers working to make computers more friendly, more humane, more people centered. But I’m squarely on Minsky’s side—on the side of the made. People will survive. We’ll train our machines to serve us. But what are we going to do for the machines?8
Kelly is correct to point out that there are Minsky and Engelbart “sides.” But to say that people will “survive” belittles the consequences. He is basically echoing Minsky, who is famously said to have responded to a question about the significance of the arrival of artificial intelligence by saying, “If we’re lucky, maybe they’ll keep us as pets.”
Minsky’s position is symptomatic of the chasm between the AI and IA camps. The artificial intelligence community has until now largely chosen to ignore the consequences of the systems it considers merely powerful tools, dispensing with discussions of morality. As one of the engineers who is building next-generation robots told me when I asked about the impact of automation on people: “You can’t think about that; you just have to decide that you are going to do the best you can to improve the world for humanity as a whole.”
During the past half century, McCarthy’s and Engelbart’s philosophies have remained separate and their central conflict stands unresolved. One approach supplants humans with an increasingly powerful blend of computer hardware and software. The other extends our reach intellectually, economically, and socially using the same ingredients. While the chasm between these approaches has been little remarked, the explosion of this new wave of technology, which now influences every aspect of modern life, will encapsulate the repercussions of this divide.
Will machines supplant human workers or augment them? On one level, they will do both. But once again, that is the wrong question to ask, and it provides only a partial answer. Both software and hardware robots are flexible enough that they can ultimately become whatever we program them to be. In our current economy, how robots—both machines and intelligent systems—are designed and how they are used is overwhelmingly defined by cost and benefit, and costs are falling at an increasingly rapid rate. In our society, economics dictate that if a task can be done more cheaply by machine—software or hardware—in most cases it will be. It’s just a matter of when.
The decision to come down on either side of the debates is doubly difficult because there are no obvious right answers. Although driverless cars will displace millions of jobs, they will also save many lives. Today, decisions about implementing technologies are made largely on the basis of profitability and efficiency, but there is an obvious need for a new moral calculus. The devil, however, is in more than the details. As with nuclear weapons and nuclear power, artificial intelligence, genetic engineering, and robotics will have society-wide consequences, both intended and unintended, in the next decade.
2|A CRASH IN THE DESERT
On a desert road near Florence, Arizona, one morning in the fall of 2005, a Volkswagen Touareg was kicking up a dust cloud, bouncing along at a steady twenty to twenty-five miles per hour, carrying four passengers. To the casual observer there was nothing unusual about the way the approaching vehicle was being driven. The road was particularly rough, undulating up and down through a landscape dotted with cactus and scrubby desert vegetation. The car bounced and wove, and all four occupants were wearing distinctive crash helmets. The Touareg was plastered with decals like a contestant in the Baja 1000 off-road race. It was also festooned with five curious sensors perched at the front of the roof, each with an unobstructed view of the road. Other sensors, including several radars, also sprouted from the roof. A video camera peered out through the windshield. A tall whip antenna set at the back of the vehicle, in combination with the sensors, conspired to give a postapocalyptic vibe reminiscent of a Mad Max movie.
The five sensors on the roof were actually mechanical contraptions, each rapidly sweeping an infrared laser beam back and forth over the road ahead. The beams, invisible to the eye, constantly reflected off the gravel road and the desert surrounding the vehicle. Bouncing back to sensors, the lasers provided a constantly changing portrait of the surrounding landscape accurate to the centimeter. Even small rocks on the road hundreds of feet ahead could not escape the unblinking gaze of the sensors, known as lidar.
The Touareg was even more peculiar inside. The driver, Sebastian Thrun, a roboticist and artificial intelligence researcher, wasn’t driving. Instead he was gesturing with his hands as he chatted with the other passengers. His eyes rarely watched the road. Most striking of all: his hands never touched the steering wheel, which twitched back and forth as if controlled by some unseen ghost.
Sitting behind Thrun was another computer researcher, Mike Montemerlo, who wasn’t driving either. His eyes were buried in the screen of a laptop computer that was displaying the data from the lasers, radars, and cameras in a God’s-eye view of the world around the car in which potential obstacles appeared as a partial rainbow of blips on a radar screen. It revealed an ever-changing cloud of colored dots that in aggregate represented the road unfolding ahead in the desert.
The car, named Stanley, was being piloted by an ensemble of software programs running on five computers installed in the trunk. Thrun was a pioneer of an advanced version of a robotic navigation technique known as SLAM, which stands for simultaneous localization and mapping. It had become a standard tool for robots to find their way through previously unexplored terrain. The wheel continued to twitch back and forth as the car rolled along the rutted road lined with cactus and frequent outcroppings of boulders. Immediately to Thrun’s right, between the front seats, was a large red E-Stop button to override the car’s autopilot in an emergency. After a half-dozen miles, the robotic meanderings of the Touareg felt anticlimactic. Stanley wasn’t driving down the freeway, so as the desert scenery slid by, it seemed increasingly unnecessary to wear crash helmets for what was more or less a Sunday drive in the country.
The car was in training to compete in the Pentagon’s second Grand Challenge, an ambitious autonomous vehicle contest intended to jump-start technology planned for fu
ture robotic military vehicles. At the beginning of the twenty-first century, Congress instructed the U.S. military to begin designing autonomous vehicles. Congress even gave the Pentagon a specific goal: by 2015, one-third of the army’s vehicles were supposed to go places without human drivers present. The directive wasn’t clear as to whether both autonomous and remotely teleoperated vehicles would satisfy the requirement. In either case the idea was that smart vehicles would save both money and soldiers’ lives. But by 2004, little progress had been made, and Tony Tether, the then controversial director of the Pentagon’s blue-sky research arm, DARPA, the Defense Advanced Research Projects Agency, came up with a high-profile contest as a gambit to persuade computer hackers, college professors, and publicity-seeking corporations to innovate where the military had failed. Tether was a product of the military-industrial complex, and the contest itself was a daring admission that the defense contracting world was not able to get the job done. By opening the door for ragtag teams of hobbyists, Tether ran the risk of undermining the classified world dominated by the Beltway Bandits that surround Washington, D.C., and garner the lion’s share of military research dollars.
The first Grand Challenge contest, held in 2004, was something of a fiasco. Vehicles tipped over, drove in circles, and ignominiously knocked down fences. Even the most successful entrant had gotten stuck in the dust just seven miles from the starting line in a 120-mile race, with one wheel spinning helplessly as it teetered off the edge of the road. When the dust settled, a reporter flying overhead in a light plane saw brightly colored vehicles scattered motionless over the desert floor. At the time it seemed obvious that self-driving cars were still years away, and Tether was criticized for organizing a publicity stunt.