Final Jeopardy

by Stephen Baker

  The thirty-six-year-old Jennings, with his featherweight build, is far smaller than Rutter. He has an easy laugh and a self-effacing style. He hadn’t yet found a Blu-ray player, he said, to watch the video of Watson in action. But he had clearly been reading everything he could find about the computer, including technical articles. Jennings double-majored in computer science and English at Brigham Young University and later worked as a computer programmer in Salt Lake City. When he heard about the match against Watson, he said, it excited him. “I said, ‘Wow, we get to see if a computer can play Jeopardy,’” he said. “I was more interested in the geeky part.”

  As Jennings studied up on Watson’s algorithms and its massive parallel processing, he couldn’t help comparing the computer to his own mind. “Many of the tricks that I used in Jeopardy are things that I read Watson does,” he said.

  He gave an example. One Jeopardy clue asked for the name of two of Jesus’ disciples whose names are both top-ten baby male names and end in the same letter. “I remember thinking,” he said, “that that’s not the kind of thing you can know. The only way to do it is to break it down, make a list, do the Venn diagram of it. And I come to find out that Watson does exactly that. It’s very good at decomposing questions, so it does the two fact sets in parallel. Then it does the Venn diagram to see if there’s anything on both lists.” Jennings paused for a moment, then said, “Matthew and Andrew, by the way. I got that one right at the last minute. I was about to put James and Judas, but I don’t think Judas is a popular baby name, for some reason …”

  Jennings reflected on traveling across the country, to IBM’s lab in Yorktown, to take on Rutter and Watson. “It’s a little different to be the road team,” he said. “I’m not playing in the familiar studio where I have the muscle memory and the good times and the million-dollar check. I’m picturing a very sterile lab from the fifties, people running around in white coats… .”

  Jennings had been following Watson’s record against its sparring partners, and the trend looked worrisome. In the beginning of the matches, he said, Watson was winning 64 percent of the time against standard Jeopardy players. “Now they’ve fine-tuned it, and it’s 67 percent against Tournament of Champions players. I know it can still be beat,” he said. “But I think to myself: Could I win 67 percent of my games against Tournament of Champions players? That’s not something I’ve ever done. I rattled off a very long streak, but it was against rookie players.” He said that he was going into the match feeling, for the first time, like an underdog.

  Jennings is well known for his disarming modesty. In previous games, it could be argued, it may have benefited him as a psychological tactic. Rivals encountered a likable and unassuming young man who seemed almost surprised at his own success. By the time they looked at the scoreboard, he was annihilating them.

  Such tactics wouldn’t mean much in the coming showdown. Jennings and Rutter would be facing a foe impervious to nerves and psychological maneuvering. And while millions tuned in to what promised to be an epic knowledge battle between two men and a thinking machine, the drama would leave Watson unmoved. The machine, unlike everyone else, had no stake in the outcome.

  11. The Match

  DAVID FERRUCCI HAD driven the same stretch hundreds of times from his suburban home to IBM’s Yorktown labs, or a bit farther to Hawthorne. For fifteen or twenty minutes along the Taconic Parkway he went over his endless to-do list. How could his team boost Watson’s fact-checking in Final Jeopardy? Could any fix ensure that the machine’s bizarre speech defect would never return? Was the pun-detection algorithm performing up to par? There were always more details, plenty to fuel both perfectionism and paranoia—and Ferrucci had a healthy measure of both.

  But this January morning was different. As he drove past frozen fields and forests, the pine trees heavy with fresh snow, all of his lists were history. After four years, his team’s work was over. Within hours, Watson alone would be facing Ken Jennings and Brad Rutter, with Ferrucci and the machine’s other human trainers reduced to spectators. Ferrucci felt his eyes well up. “My whole team would be judged by this one game,” he said later. “That’s what killed me.”

  The day before, at a jam-packed press conference, IBM had unveiled Watson to the world. The event took place on a glittering new Jeopardy set mounted over the previous two weeks by an army of nearly a hundred workers. It resembled the set in Culver City: the same jumbo game board to the left, the contestants’ lecterns to the right, with Alex Trebek’s podium in the middle. In front was a long table for the Jeopardy officials, where Harry Friedman would sit, Rocky Schmidt at his side, followed by a line of writers and judges, all of them with monitors, phones, and a pile of old-fashioned reference books. Every piece was in place. But this East Coast version was plastered with IBM branding. The shimmering blue wall bore the company’s historic slogan, Think, in a number of languages. Stretched across the shiny black floor was a logo that looked at first like Batman’s emblem. But closer study revealed the planet Earth, with each of the continents bulging, as if painted by Fernando Botero. This was Chubby Planet, the symbol of IBM’s Smarter Planet campaign and the model for Watson’s avatar. In the negotiations with Jeopardy over the past two years, IBM had lost out time and again on promotional guarantees. It had seemed that Harry Friedman and his team held all the cards. But now that the match was assured and on Big Blue’s home turf, not a single branding opportunity would be squandered.

  The highlight of the press event came when Jennings and Rutter strode across the stage for a five-minute, fifteen-clue demonstration. In this test run, Watson held its own. In fact, it ended the session ahead of Jennings, $4,400 to $3,400. Rutter trailed with $1,200. Within hours, Internet headlines proclaimed that Watson had vanquished the humans. It was as if the game had already been won.

  If only it were true. The demo match featured just a handful of clues and included no Final Jeopardy—Watson’s Achilles’ heel. What’s more, after the press departed that afternoon, Watson and the human champs went on to finish that game and play another round—“loosening their thumbs,” in the language of Jeopardy. In these games Ferrucci saw a potential problem: Ken Jennings. It was clear, he said, that Jennings had prepped heavily for the match. He had a sense of Watson’s vulnerabilities and an aggressive betting strategy specially honed for the machine. Brad Rutter was another matter altogether. Starting out, Ferrucci’s team had been more concerned about Rutter than Jennings. His speed on the buzzer was the stuff of legend. Yet he appeared relaxed, almost too relaxed, as if he could barely be bothered to buzz. Was he saving his best for the match?

  In the first of the two practice games, Jennings landed on all three Daily Doubles. Each time he bet nearly everything he had. This was the same strategy Greg Lindsay had followed to great effect in three sparring games a year earlier. The rationale was simple. Even with its mechanical finger slowing it down by a few milliseconds, Watson was lightning fast on the buzzer. The machine was likely to win more than its share of the regular Jeopardy clues. So the best chance for the humans was to pump up their winnings on the four clues that hinged on betting, not buzzing: the Daily Doubles and Final Jeopardy. Thanks to his aggressive betting, Jennings ended the first full practice game with some $50,000, a length ahead of Watson, which scored $39,000. Jennings was fired up. When he clinched the match, he pointed to the computer and exclaimed, “Game over!” Rutter finished a distant third, with about $10,000. In the second game, Jennings and Watson were neck and neck to the end, when Watson edged ahead in Final Jeopardy. Again, Rutter coasted to third place. Ferrucci said that he and his team left the practice rounds thinking, “Ken’s really good—but what’s going on with Brad?”

  When Ferrucci pulled in to the Yorktown labs the morning of the match, the site had been transformed. The visitors’ parking lot was cordoned off for VIPs. Security guards noted every person entering the building, checking their names against a list. And in the vast lobby, usually manned by one lonely guard, IBM’s lu
minaries and privileged guests circled around tables piled with brunch fare. Ferrucci made his way to Watson’s old practice studio, now refashioned as an exhibition room. There he gave a half-hour talk about the computer to a gathering of IBM clients, including J. P. Morgan, American Express, and the pharmaceutical giant Merck & Co. Ferrucci recalled the distant days when a far stupider Watson responded to a clue about a famous French bacteriologist by saying: “What is ‘How Tasty Was My Little Frenchman’?” (That was the title of a 1971 Brazilian comedy about cannibals in the Amazon.)

  His next stop, the makeup room, revealed his true state of mind. The makeup artist was a woman originally from Italy, like much of Ferrucci’s family. As she began to work on his face she showered him with warmth and concern—acting “motherly.” This rekindled his powerful feelings about his team and the end of their journey, and before he knew it, tears were streaming down his face. The more the woman comforted him, the worse it got. Ferrucci finally stanched the flow and got the pancake on his face, but he knew he was a mess. He hunted down Scott Brooks, the lighthearted press officer. Maybe some jokes, he thought, “would take the lump out of my throat.” Brooks laughed and warned him that people might compare him to the new U.S. Speaker of the House, John Boehner, whose frequent tears had recently earned him the sobriquet “Weeper of the House.”

  This irritated the testy Ferrucci and, to his relief, knocked him out of his fragile mood. He joined his team for a last lunch, all of them seated at a long table in the cafeteria. As they were finishing, just a few minutes before 1 P.M., a roaring engine interrupted their conversations. It was IBM’s chairman, Sam Palmisano, landing in his helicopter. The hour had come. Ferrucci walked down the sunlit corridor to the auditorium.

  Ken Jennings woke up that Friday morning in the Crown Plaza Hotel in White Plains. He’d slept well, much better than he usually did before big Jeopardy matches. He had good reason to feel confident. He had destroyed Watson in one of the practice rounds. Afterward, he said, Watson’s developers told him that the game had featured a couple of “train wrecks”—categories in which Watson appeared disoriented. Children’s Literature was one. For Jennings, train wrecks signaled the machine’s vulnerability. With a few of them in the big match, he could stand up tall for humans and perhaps extend his legend from Jeopardy to the broader realm of knowledge. “Given the right board,” he said, “Watson is beatable.” The stakes were considerable. While IBM would give all of Watson’s winnings to charity, a human winner would earn a half-million-dollar prize, with another half-million to give to the charity of his choice. Finishing in second and third place was worth $150,000 and $100,000, with equal amounts for the players’ charities.

  A little after eleven, a car service stopped by the hotel, picked up Jennings and his wife, Mindy, and drove them to IBM’s Yorktown laboratory. Jennings carried three changes of clothes so that he could dress differently for each session, simulating three different days. As soon as he stepped out of the car, Jeopardy officials whisked him past the crush of people in the lobby toward the staircase. Jeopardy had cleared out a few offices in IBM’s Human Resources Department, and Jennings was given one as a dressing room.

  On short visits to the East Coast, Brad Rutter liked to sleep late in order to stay in sync with West Coast time. But the morning of the match, he found himself awake at seven, which meant he faced four and a half hours before the car came for him. Rutter was at the Ritz-Carlton in White Plains, about a half mile from Jennings. He ate breakfast, showered, and then killed time until 11:30. Unlike Jennings, Rutter had grounds for serious concern. In the practice rounds, he had been uncharacteristically slow. The computer had exquisite timing, and Jennings seemed to hold his own. Rutter, who had never lost a game of Jeopardy, was facing a flameout unless he could get to the buzzer faster.

  Shortly after Rutter arrived at IBM, he and Jennings played one last practice round with Watson. To Rutter’s delight, his buzzer thumb started to regain its old magic, and he beat both Jennings and the machine. Now, in the three practice matches, each of the players had registered a win. But Jennings and Rutter noticed something strange about Watson. Its game strategy, Jennings said, “seemed naive.” Just like beginning Jeopardy players, Watson started with the easy, low-dollar clues and moved straight down the board. Why wasn’t it hunting for Daily Doubles? In the Blu-ray Discs given to them in November, Jennings and Rutter had seen that Watson skipped around the high-dollar clues, hunting for the single Daily Double on the first Jeopardy board and the two in Double Jeopardy. Landing on Daily Doubles was vital. It gave a player the means to build a big lead. And once the Daily Doubles were off the board, the leader was hard to catch. But in the practice rounds, Watson didn’t appear to be following this strategy.

  The two players were led to a tiny entry hall behind the auditorium. As the event began, shortly after 1 P.M., they waited. They listened as IBM introduced Watson to its customers. “You know how they call time-outs before a guy kicks a field goal?” Jennings said. “We were joking that they were doing the same thing to us. Icing us.” Through the door they heard speeches by John Kelly, the chief of IBM Research, and Sam Palmisano. Harry Friedman, who decades earlier had earned $5 a joke as a writer for Hollywood Squares, delivered one of his own. “I’ve lived in Hollywood for a long time,” he told the crowd, “so I know something about Artificial Intelligence.” When Ferrucci was called to the stage, the crowd rose for a standing ovation. “I already cried in makeup,” he said. “Let’s not repeat that.”

  Finally, it was time for Jeopardy, and Jennings and Rutter were summoned to the stage. They walked down the narrow aisle of the auditorium, Jennings leading in a business suit and yellow tie, the taller, loose-gaited Rutter following him, his collar unbuttoned. They settled at their lecterns, Jennings on the far side, Rutter closer to the crowd. Between them, its circular black screen dancing with colorful jagged lines, sat Watson.

  The show began with the familiar music. A fill-in for the legendary announcer, Johnny Gilbert (who hadn’t made the trip from Culver City), introduced the contestants and Alex Trebek. Even then, Jennings and Rutter had to wait while an IBM video told the story of the Watson project. In a second video, Trebek asked Ferrucci about the machinery behind the bionic player—now up to 2,880 processing cores. Then Trebek gave viewers a tutorial on Watson’s answer panel. It would reveal the statistical confidence that the computer had in each of its top responses. It was a window into Watson’s thinking.

  Trebek, in fact, had been a late convert to the answer panel. Like the rest of the Jeopardy team, he was loath to stray from the show’s time-honored formulas. People knew what to expect from the game: the precise movements of the cameras, the familiar music, voices, and categories. Wouldn’t the intrusion of an electronic answer panel distract them and ultimately make the game less enjoyable to watch? Trebek raised that concern on a visit to IBM in November. But the prospect of televising the game without Watson’s answer panel horrified Ferrucci. Millions of viewers, he believed, would simply conclude that the machine had been fed all the answers. They wouldn’t appreciate what Watson went through to arrive at the correct response. So while Trebek was eating lunch that day, Ferrucci had his technicians take down the answer panel. When the afternoon sessions began, it took only one game for Trebek to ask for it to be restored. Later, he said, watching Watson without the panel’s analysis was “boring as hell.”

  Finally, it was time to play. A hush settled over the auditorium. Ferrucci, sitting between David Gondek and Eric Brown, laced his hands tightly and made a steeple with his index fingers. He watched as Trebek, with a wave of his arm, revealed the six categories for the first round of Jeopardy. One was Literary Character APB. Trebek explained that APB stood for “all points bulletin.” This clarification was lost on the deaf Watson, which irked Ferrucci and the IBM team. Other categories were Beatles People, Olympic Oddities, Name that Decade, Final Frontiers, and Alternate Meanings. None of them looked especially vexing for the computer.
>
  Rutter had won the draw, so he started and chose Alternate Meanings for $200. “A four-letter word for vantage point,” Trebek read, “or belief.” Rutter, famous for his prowess with the buzzer, won this first clue and responded correctly: “What is view?”

  He asked for the $400 clue in the same category. Trebek read: “Four-letter word for the iron fitting on the hoof of a horse, or a card-dealing box in a casino.”

  Watson won the buzz and uttered its first syllables for an audience of millions, answering correctly: “What is a shoe?” It pronounced the final word meekly, as if unsure of itself or perhaps embarrassed. Still, with that response, Watson had $400—positive winnings against the greatest of human players. That alone was a threshold that four years earlier had appeared daunting to many—including some in the audience.

  With control of the board, Watson pursued the merciless strategy mapped out by David Gondek and his team. Departing from its passive approach in the practice rounds, it moved straight to the high-dollar boxes, hunting for the Daily Double. “Let’s try Literary Character APB for eight hundred,” Watson said. The zinging sound of space guns echoed through the auditorium, announcing that the machine, on its first try, had landed on the Daily Double. The two APPLAUSE signs flashed over the stage, but they were hardly needed. This was Watson’s crowd.

  In truth, the Daily Double in the first round of Jeopardy is not terribly important, especially this early in the game. The players at this stage have very little money to bet. It’s in Double Jeopardy, when the end is in sight and the contestants have piled up much higher winnings, that a laggard can vault toward victory, winning $10,000 or even more with a single bet. Watson’s Daily Double strategy was less about padding its own lead than keeping these dangerous wild cards from its rivals.