by Dan Ariely
It surprised me to see some confederates being coy with their judges. Asked what kind of engineer he is, Dave, to my left, answered, “A good one. :)” And Doug, to my right, responded to a question about what brought him to Brighton with “if I tell you, you’ll know immediately that I’m human ;-)” For my money, wit is very successful, but coyness is a double-edged sword. You show a sense of humor, but you jam the cogs of the conversation. Probably the most dangerous thing a confederate can do in a Turing Test is stall. It’s suspect—as the guilty party would tend to be the one running out the clock—and it squanders your most precious resource: time.
The humans in a Turing Test are strangers, limited to a medium that is slow and has no vocal tonality, and without much time. A five-second Turing Test would be an easy win for the machines: the judges, barely able to even say “hello,” simply wouldn’t be able to get enough data from their respondents to make any kind of judgment. A five-hour test would be an easy win for the humans. The Loebner Prize organizers have tried different time limits since the contest’s inception, but in recent years they’ve mostly adhered to Turing’s original prescription of five minutes: around the point when conversation starts to get interesting.
A big part of what I needed to do as a confederate was simply to make as much engagement happen in those minutes as I physically and mentally could. Rather than adopt the terseness of a deponent, I offered the prolixity of a writer. In other words, I talked a lot. I stopped typing only when to keep going would have seemed blatantly impolite or blatantly suspicious. The rest of the time, my fingers were moving. I went out of my way to embody that maxim of “A bore is a man who, being asked ‘How are you?’ starts telling you how he is.”
JUDGE: Hi, how’s things?
CONFEDERATE: hey there
CONFEDERATE: things are good
CONFEDERATE: a lot of waiting, but . . .
CONFEDERATE: good to be back now and going along
CONFEDERATE: how are you?
When we’d finished, and my judge was engaged in conversation with one of my computer counterparts, I strolled around the table, seeing what my comrades were up to. Looking over at my fellow confederate Dave’s screen, I noticed his conversation began as if he were on the receiving end of an interrogation, and he was answering in a kind of minimal staccato:
JUDGE: Are you from Brighton?
CONFEDERATE: No, from the US
JUDGE: What are you doing in Brighton?
CONFEDERATE: On business
JUDGE: How did you get involved with the competition?
CONFEDERATE: I answered an e-mail.
Like a good deponent, he let the questioner do all the work. When I saw how stiff Dave was being, I confess I felt a certain confidence—I, in my role as the world’s worst deponent, was perhaps in fairly good shape as far as the Most Human Human award was concerned.
This confidence lasted approximately sixty seconds, or enough time for me to continue around the table and see what another fellow confederate, Doug, and his judge had been saying.
JUDGE: Hey Bro, I’m from TO.
CONFEDERATE: cool
CONFEDERATE: leafs suck
CONFEDERATE: ;-)
JUDGE: I am just back from a sabbatical in the CS Dept. at U of T.
CONFEDERATE: nice!
JUDGE: I remember when they were a great team.
JUDGE: That carbon date me, eh?
CONFEDERATE: well, the habs were a great team once, too . . .
CONFEDERATE: *sigh*
JUDGE: YEH, THEY SUCK TOO.
CONFEDERATE: (I’m from Montreal, if you didn’t guess)
Doug and his judge had just discovered that they were both Canadian. They let rip with abbreviations and nicknames and slang and local references. And then they started to talk about hockey.
I was in trouble.
Six months after the 2009 contest, a video appeared on YouTube of a man having a shockingly cogent conversation with a bot about Shakespeare’s Hamlet. Some suspected it might herald a new age for chatbots, and for AI. Others, including myself, were unimpressed. Sophisticated behavior doesn’t necessarily indicate a mind. It might indicate just a memory. As Dali so famously put it, “The first man to compare the cheeks of a young woman to a rose was obviously a poet; the first to repeat it was possibly an idiot.”
For instance, Richard Wallace, the three-time Most Human Computer winner, recounts an “AI urban legend” in which “a famous natural language researcher was embarrassed . . . when it became apparent to his audience of Texas bankers that the robot was consistently responding to the next question he was about to ask . . . [His] demonstration of natural language understanding . . . was in reality nothing but a simple script.”
The moral of the story: no demonstration is ever sufficient. Only interaction will do. In the 1997 contest, one judge gets taken for a ride by Catherine, waxing political and really engaging in the topical conversation “she” has been programmed to lead about the Clintons and Whitewater. In fact, everything is going swimmingly until the very end, when the judge signs off:
JUDGE: it looks like i’ve got to go, Catherine, its been nice talking with you
JUDGE: do you need water or something before the 5 minute rounds?
At which point the program erupts into gibberish.
COMPUTER: OK, yes on balance . . . Time to get off this one I think and onto something more down to earth!
JUDGE: like what?
COMPUTER: Almost certainly! let’s move on
COMPUTER: I suppose it depends on where you’re coming from, but as the song goes in My Fair Lady, “Why can’t a woman be more like a man?”
Defeat from the jaws of victory.
We so often think of intelligence, of AI, in terms of sophistication, or complexity of behavior. But in so many cases, it’s impossible to say much with certainty about the program itself, because any number of different pieces of software—of wildly varying levels of “intelligence”—could have produced that behavior.
No, I think sophistication, complexity of behavior, is not it at all. For instance, you can’t judge the intelligence of an orator by the eloquence of his prepared remarks; you must wait until the Q&A and see how he fields questions. The computation theorist Hava Siegelmann once described intelligence as “a kind of sensitivity to things.” These Turing Test programs that hold forth may produce interesting output, but they’re rigid and inflexible. They are, in other words, insensitive—occasionally fascinating talkers that cannot listen.
As computing technology in the twenty-first century moves increasingly toward mobile devices, we’ve seen the 1990s’ explosive growth in processor speed taper off and product development become less about raw computing horsepower than about the overall design of the product and its fluidity, reactivity, and ease of use. This fascinating shift in computing emphasis may be the cause, effect, or correlative of a healthier view of human intelligence—an understanding, not so much that it is complex and powerful, per se, as that it is reactive, responsive, sensitive, nimble. Our computers, flawed mirrors that they are, have helped us see that about ourselves.
The Most Human Human
The Most Human Computer award in 2009 goes to David Levy and his program, Do-Much-More. Levy, who also won in ’97, with Catherine, is an intriguing guy: he was one of the big early figures in the digital-chess scene of the ’70s and ’80s, and was one of the organizers of the Marion Tinsley–Chinook checkers matches that preceded the Kasparov–Deep Blue showdowns in the ’90s. He’s also the author of the recent nonfiction book Love and Sex with Robots, to give you an idea of the sorts of things that are on his mind when he’s not competing for the Loebner Prize.
Levy stands up, to applause, accepts the award from Philip Jackson and Hugh Loebner, and makes a short speech about the importance of AI for a bright future and the importance of the Loebner Prize for AI. I know what’s next on the agenda, and my stomach knots. I’m certain that Doug’s gotten it; he and the judge were ta
lking Canada thirty seconds into their conversation.
Ridiculous Canadians and their ice hockey, I’m thinking. Then I’m thinking how ridiculous it is that I’m even allowing myself to get this worked up about some silly award. Then I’m thinking how ridiculous it is to fly 5,000 miles just to have a few minutes’ worth of IM conversations. Then I’m thinking how maybe it’ll be great to be the runner-up; I can compete again in 2010 in Los Angeles, with the home-field cultural advantage, and finally prove—
“And the results here show also the identification of the humans,” Jackson announces, “and from the ranking list we can see that ‘Confederate 1,’ which is Brian Christian, was the most human.”
And he hands me the certificate for the Most Human Human award.
I didn’t know how to feel, exactly. It seemed strange to treat the award as meaningless or trivial, but did winning really represent something about me as a person? More than anything, I felt that together, my fellow confederates and I had avenged the mistakes of 2008 in dramatic fashion. That year, the twelve judges decided five times that computer programs were more human than confederates. In three of those instances, the judge was fooled by a program named Elbot, which was the handiwork of a company called Artificial Solutions, one of many new businesses leveraging chatbot technology. One more deception, and Elbot would have tricked 33 percent of that year’s dozen judges—surpassing Turing’s 30 percent mark and making history. After Elbot’s victory at the Loebner Prize and the publicity that followed, the company seemingly decided to prioritize the Elbot software’s more commercial applications; at any rate, it had not entered the ’09 contest as the returning champion.
In some ways a closer fight would have been more dramatic. Between us, we confederates hadn’t permitted a single vote to go the machines’ way. Whereas 2008 was a nail biter, 2009 was a rout. We think of science as an unhaltable, indefatigable advance. But in the context of the Turing Test, humans—dynamic as ever—don’t allow for that kind of narrative. We don’t provide the kind of benchmark that sits still.
As for the prospects of AI, some people imagine the future of computing as a kind of heaven. Rallying behind an idea called the Singularity, people like Ray Kurzweil (in The Singularity Is Near) and his cohort of believers envision a moment when we make smarter-than-us machines, which make machines smarter than themselves, and so on, and the whole thing accelerates exponentially toward a massive ultra-intelligence that we can barely fathom. Such a time will become, in their view, a kind of a techno-Rapture, in which humans can upload their consciousness onto the Internet and get assumed—if not bodily, than at least mentally—into an eternal, imperishable afterlife in the world of electricity.
Others imagine the future of computing as a kind of hell. Machines black out the sun, level our cities, seal us in hyperbaric chambers, and siphon our body heat forever.
I’m no futurist, but I suppose, if anything, I prefer to think of the long-term future of AI as a kind of purgatory: a place where the flawed but good-hearted go to be purified—and tested—and come out better on the other side.
Who would have imagined that the computer’s earliest achievements would be in the domain of logical analysis, a capacity once held to be what made us most different from everything else on the planet? That it could fly a plane and guide a missile before it could ride a bike? That it could create plausible preludes in the style of Bach before it could make plausible small talk? That it could translate before it could paraphrase? That it could spin half-discernible essays on postmodern theory before it could be shown a chair and say, as most toddlers can, “chair”?
As computers have mastered rarefied domains once thought to be uniquely human, they simultaneously have failed to master the ground-floor basics of the human experience—spatial orientation, object recognition, natural language, adaptive goal-setting—and in so doing, have shown us how impressive, computationally and otherwise, such minute-to-minute fundamentals truly are.
We forget how impressive we are. Computers are reminding us.
One of my best friends was a barista in high school. Over the course of a day, she would make countless subtle adjustments to the espresso being made, to account for everything from the freshness of the beans to the temperature of the machine to the barometric pressure’s effect on the steam volume, meanwhile manipulating the machine with an octopus’s dexterity and bantering with all manner of customers on whatever topics came up. Then she went to college and landed her first “real” job: rigidly procedural data entry. She thought back longingly to her barista days—when her job actually made demands of her intelligence.
Perhaps the fetishization of analytical thinking, and the concomitant denigration of the creatural—that is, animal—and bodily aspects of life are two things we’d do well to leave behind. Perhaps at last, in the beginnings of an age of AI, we are starting to center ourselves again, after generations of living slightly to one side—the logical, left-hemisphere side. Add to this that humans’ contempt for “soulless” animals, our unwillingness to think of ourselves as descended from our fellow “beasts,” is now challenged on all fronts: growing secularism and empiricism, growing appreciation for the cognitive and behavioral abilities of organisms other than ourselves, and, not coincidentally, the entrance onto the scene of an entity with considerably less soul than we sense in a common chimpanzee or bonobo—in this way AI may even turn out to be a boon for animal rights.
Indeed, it’s entirely possible that we’ve seen the high-water mark of our left-hemisphere bias. I think the return of a more balanced view of the brain and mind—and of human identity—is a good thing, one that brings with it a changing perspective on the sophistication of various tasks.
It’s my belief that only experiencing and understanding truly disembodied cognition—only seeing the coldness and deadness and disconnectedness of something that really does deal in pure abstraction, divorced from sensory reality—can snap us out of it. Only this can bring us, quite literally, back to our senses.
In a 2006 article about the Turing Test, the Loebner Prize cofounder Robert Epstein writes, “One thing is certain: whereas the confederates in the competition will never get any smarter, the computers will.” I agree with the latter and couldn’t disagree more strongly with the former.
When the world-champion chess player Garry Kasparov defeated Deep Blue, rather convincingly, in their first encounter in 1996, he and IBM readily agreed to return the next year for a rematch. When Deep Blue beat Kasparov (rather less convincingly) in ’97, Kasparov proposed another rematch for ’98, but IBM would have none of it. The company dismantled Deep Blue, which never played chess again.
The apparent implication is that—because technological evolution seems to occur so much faster than biological evolution (measured in years rather than millennia)—once the Homo sapiens species is overtaken, it won’t be able to catch up. Simply put: the Turing Test, once passed, is passed forever. I don’t buy it.
Rather, IBM’s odd anxiousness to get out of Dodge after the ’97 match suggests a kind of insecurity on its part that I think proves my point. The fact is, the human race got to where it is by being the most adaptive, flexible, innovative, and quick-learning species on the planet. We’re not going to take defeat lying down.
No, I think that while the first year that computers pass the Turing Test will certainly be a historic one, it will not mark the end of the story. Indeed, the next year’s Turing Test will truly be the one to watch—the one where we humans, knocked to the canvas, must pull ourselves up; the one where we learn how to be better friends, artists, teachers, parents, lovers; the one where we come back. More human than ever.
Contributors’ Notes
Michael Behar is a freelance journalist based in Boulder, Colorado, where he lives with his wife and two-year-old son. He writes about adventure travel, the environment, and innovations in science and technology for more than twenty-five national publications, including Outside, Wired, Skiing, Men’s Journal,
Popular Science, and OnEarth, where he is a contributing editor. He is the former science editor for National Geographic and served as a senior editor at Wired from 1995 to 2000. In addition to this anthology, he’s been featured in The Best American Travel Writing 2006 and The Best of Technology Writing 2008.
Deborah Blum is a Pulitzer Prize–winning science journalist, a blogger for Wired, and the author of five books, most recently The Poisoner’s Handbook: Murder and the Birth of Forensic Medicine in Jazz Age New York, a New York Times paperback bestseller. She writes for publications ranging from Scientific American to the literary journal Tin House and is a founding editor of the acclaimed e-book review site Download the Universe. She teaches journalism at the University of Wisconsin–Madison.
Brendan Buhler writes about galaxies, genetics, parasites, clean energy, the skull-strewn wilderness, people who keep leopards as pets, and jumbo Elvis impersonators. Politics, too, but cockroaches and whimsy preferred. His work has appeared in the Los Angeles Times and California and Sierra magazines. The chapter he contributed to 101 Places Not to See Before You Die was about the annual porn convention in Vegas, an experience that will tarnish your very soul with the orange cheese dust of vileness.