The Big Nine
Page 4
The following year, the group gathered on the top floor of Dartmouth’s math department and researched complexity theory, natural language simulation, neural networks, the relationship of randomness to creativity, and learning machines. On the weekdays they met in the main math classroom for a general discussion before dispersing to tackle the more granular tasks. Professors Allen Newell, Herbert Simon, and Cliff Shaw came up with a way to discover proofs of logical theorems and simulated the process by hand—a program they called Logic Theorist—at one of the general sessions. It was the first program to mimic the problem-solving skills of a human. (Eventually, it would go on to prove 38 of the first 52 theorems in Alfred North Whitehead and Bertrand Russell’s Principia Mathematica, a standard text on the foundations of mathematics.) Claude Shannon, who had several years earlier proposed teaching computers to play chess against humans, got the opportunity to show a prototype of his program, which was still under construction.20
McCarthy and Minsky’s expectations for groundbreaking advancements in AI didn’t materialize that summer at Dartmouth. There wasn’t enough time—not to mention enough compute power—to evolve AI from theory to practice.21 However, that summer did set in motion three key practices that became the foundational layer for AI as we know it today:
1. AI would be theorized, built, tested, and advanced by big technology companies and academic researchers working together.
2. Advancing AI required a lot of money, so commercializing the work in some way—whether working through partnerships with government agencies or the military or building products and systems that could be sold—was going to be required.
3. Investigating and building AI relied on a network of interdisciplinary researchers, which meant establishing a new academic field from scratch. It also meant that those in the field tended to recruit people they already knew, which kept the network relatively homogenous and limited its worldview.
There was another interesting development that summer. While the group coalesced around the question raised by Turing—Can machines think?—they were split on the best approach to prove his answer, which was to build a learning machine. Some of the members favored a biological approach. That is, they believed that neural nets could be used to imbue AI with common sense and logical reasoning—that it would be possible for machines to be generally intelligent. Other members argued that it would never be possible to create such a complete replica of human thinking structures. Instead, they favored an engineering approach. Rather than writing commands to solve problems, a program could help the system “learn” from a data set. It would make predictions based on that data, and a human supervisor would check answers—training and tweaking it along the way. In this way, “machine learning” was narrowly defined to mean learning a specific task, like playing checkers.
Psychologist Frank Rosenblatt, who was at the Dartmouth workshop, wanted to model how the human brain processed visual data and, as a result, learn how to recognize objects. Drawing on the research from that summer, Rosenblatt created a system called Perceptron. His intent was to construct a simple framework program that would be responsive to feedback. It was the first artificial neural network (ANN) that operated by creating connections between multiple processing elements in a layered arrangement. Each mechanical neuron would take in lots of different signal inputs and then use a mathematical weighting system to decide which output signal to generate. In this parallel structure, multiple processors could be accessed at once—meaning that it was not only fast, it could process a lot of data continuously.
Here’s why this was so important: while it didn’t necessarily mean that a computer could “think,” it did show how to teach a computer to learn. We humans learn through trial and error. Playing a C scale on the piano requires striking the right keys in the right sequence. At the beginning, our fingers, ears, and eyes don’t have the correct pattern memorized, but if we practice—repeating the scale over and over, making corrections each time—we eventually get it right. When I took piano lessons and mangled my scales, my teacher corrected me, but if I got them right, I earned a sticker. The sticker reinforced that I’d made the right decisions while playing. It’s the same with Rosenblatt’s neural network. The system learned how to optimize its response by performing the same functions thousands of times, and it would remember what it learned and apply that knowledge to future problems. He’d train the system using a technique called “back propagation.” During the initial training phase, a human evaluates whether the ANN made the correct decision. If it did, the process is reinforced. If not, adjustments were made to the weighting system, and another test was administered.
In the years following the workshop, there was remarkable progress made on complicated problems for humans, like using AI to solve mathematical theorems. And yet training AI to do something that came simply—like recognizing speech—remained a vexing challenge with no immediate solution. Before their work on AI began, the mind had always been seen as a black box. Data went in, and a response came back out with no way to observe the process. Early philosophers, mathematicians, and scientists said this was the result of divine design. Modern-era scientists knew it was the result of hundreds of thousands of years of evolution. It wasn’t until the 1950s, and the summer at Dartmouth, that researchers believed they could crack open the black box (at least on paper) and observe cognition. And then teach computers to mimic our stimulus-response behavior.
Computers had, until this point, been tools to automate tabulation. The first era of computing, marked by machines that could calculate numbers, was giving way to a second era of programmable computers. These were faster, lighter systems that had enough memory to hold instruction sets within the computers. Programs could now be stored locally and, importantly, written in English rather than complicated machine code. It was becoming clear that we didn’t need automata or humanistic containers for AI applications to be useful. AI could be housed in a simple box without any human characteristics and still be extremely useful.
The Dartmouth workshop inspired British mathematician I. J. Good to write about “an ultraintelligence machine” that could design ever better machines than we might. This would result in a future “intelligence explosion, and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make.”22
A woman did finally enter the mix, at least in name. At MIT, computer scientist Joseph Weizenbaum wrote an early AI system called ELIZA, a chat program named after the ingenue in George Bernard Shaw’s play Pygmalion.23 This development was important for neural networks and AI because it was an early attempt at natural language processing, and the program accessed various prewritten scripts in order to have conversations with real people. The most famous script was called DOCTOR,24 and it mimicked an empathetic psychologist using pattern recognition to respond with strikingly humanistic responses.
The Dartmouth workshop had now generated international attention, as did its researchers, who’d unexpectedly found themselves in the limelight. They were nerdy rock stars, giving everyday people a glimpse into a fantastical new vision of the future. Remember Rosenblatt, the psychologist who’d created the first neural net? He told the Chicago Tribune that soon machines wouldn’t just have ELIZA programs capable of a few hundred responses, but that computers would be able to listen in on meetings and type out dictation, “just like a office secretary.” He promised not only the largest “thinking device” ever built, but one that would be operational within just a few months’ time.25
And Simon and Newell, who built the Logic Theorist? They started making wild, bold predictions about AI, saying that within ten years—meaning by 1967—computers would
• beat all the top-ranked grandmasters to become the world’s chess champion,
• discover and prove an important new mathematical theorem, and
• write the kind of music that even the harshest critics would still value.26
Mea
ntime, Minsky made predictions about a generally intelligent machine that could do much more than take dictation, play chess, or write music. He argued that within his lifetime, machines would achieve artificial general intelligence—that is, computers would be capable of complex thought, language expression, and making choices.27
The Dartmouth workshop researchers wrote papers and books. They sat for television, radio, newspaper, and magazine interviews. But the science was difficult to explain, and so oftentimes explanations were garbled and quotes were taken out of context. Wild predictions aside, the public’s expectations for AI became more and more fantastical, in part because the story was misreported. For example, Minsky was quoted in Life magazine saying: “In from three to eight years we will have a machine with the general intelligence of an average human being. I mean a machine that will be able to read Shakespeare, grease a car, play office politics, tell a joke, have a fight.”28 In that same article, the journalist refers to Alan Turing as “Ronald Turing.” Minsky, who was clearly enthusiastic, was likely being cheeky and didn’t mean to imply that walking, talking robots were just around the corner. But without the context and explanation, the public perception of AI started to warp.
It didn’t help that in 1968, Arthur Clarke and Stanley Kubrick decided to make a movie about the future of machines with the general intelligence of the average person. The story they wanted to tell was an origin story about humans and thinking machines—and they brought Minsky on board to advise. If you haven’t guessed already, it’s a movie you already know called 2001: A Space Odyssey, and it centered around a generally intelligent AI named HAL 9000, who learned creativity and a sense of humor from its creators—and threatened to kill anyone who wanted to unplug it. One of the characters, Victor Kaminski, even got his name from Minsky.
It’s fair to say that by the middle of the 1960s, AI had entered the zeitgeist, and everyone was fetishizing the future. Expectations for the commercial success of AI were on the rise, too, due to an article published in an obscure trade journal that covered the radio industry. Titled simply “Cramming More Components onto Integrated Circuits,” the article, written by Intel cofounder Gordon Moore, laid out the theory that the number of possible transistors that could be placed on an integrated circuit board for the same price would double every 18 to 24 months. This bold idea became known as Moore’s law, and very early on his thesis appeared to be accurate. Computers were becoming more and more powerful and capable of myriad tasks, not just solving math problems. It was fuel for the AI community because it meant that their theories could move into serious testing soon. It also raised the fascinating possibility that human-made AI processors could ultimately exceed the powers of the human mind, which has a biologically limited storage capacity.
All the hype, and now this article, funneled huge investment into AI—even if those outside the Dartmouth network didn’t quite understand what AI really was. There were no products to show yet, and there were no practical ways to scale neural nets and all the necessary technology. Because people now believed in the possibility of thinking machines, that was enough to secure significant corporate and government investment. For example, the US government funded an ambitious AI program for language translation. It was the height of the Cold War, and the government wanted an instantaneous translation system of Russian for greater efficiency, cost savings, and accuracy. It seemed as though machine learning could provide a solution by way of a translation program. A collaboration between the Institute of Languages and Linguistics at Georgetown University and IBM produced a Russian-English machine translation system prototype that had a limited 250-word vocabulary and specialized only in organic chemistry. The successful public demonstration caused many people to leap to conclusions, and machine translation hit the front page of the New York Times—along with half a dozen other newspapers.
Money was flowing—between government agencies, universities, and the big tech companies—and for a time, it didn’t look like anyone was monitoring the tap. But beyond those papers and prototypes, AI was falling short of promises and predictions. It turned out that making serious headway proved a far greater challenge than its modern pioneers anticipated.
Soon, there were calls to investigate the real-world uses and practical implementation of AI. The National Academy of Sciences had established an advisory committee at the request of the National Science Foundation, the Department of Defense, and the Central Intelligence Agency. They found conflicting viewpoints on the viability of AI-powered foreign language translation and ultimately concluded that “there has been no machine translation of general scientific text, and none is in immediate prospect.”29 A subsequent report produced for the British Science Research Council asserted that the core researchers had exaggerated their progress on AI, and it offered a pessimistic prognosis for all of the core research areas in the field. James Lighthill, a British applied mathematician at Cambridge, was the report’s lead author; his most damning criticism was that those early AI techniques—teaching a computer to play checkers, for example—would never scale up to solve bigger, real-world problems.30
In the wake of the reports, elected officials in the US and UK demanded answers to a new question: Why are we funding the wild ideas of theoretical scientists? The US government, including DARPA, pulled funding for machine translation projects. Companies shifted their priorities away from time-intensive basic research on general AI to more immediate programs that could solve problems. If the early years following the Dartmouth workshop were characterized by great expectations and optimism, the decades after those damning reports became known as the AI Winter. Funding dried up, students shifted to other fields of study, and progress came to a grinding halt.
Even McCarthy became much more conservative in his projections. “Humans can do this kind of thing very readily because it’s built into us,” McCarthy said.31 But we have a much more difficult time understanding how we understand speech—the physical and cognitive processes that make language recognition possible. McCarthy liked to use a birdcage example to explain the challenge of advancing AI. Let’s say that I asked you to build me a birdcage, and I didn’t give you any other parameters. You’d probably build an enclosure with a top, bottom, and sides. If I gave you an additional piece of information—the bird is a penguin—then you might not put a top on it. Therefore, whether or not the birdcage requires a top depends on a few things: the information I give you and all of the associations you already have with the word “bird,” like the fact that most birds fly. We have built-in assumptions and context. Getting AI to respond the same way we do would require a lot more explicit information and instruction.32 The AI Winter would go on to last for three decades.33
What Came Next: Learning to Play Games
While funding had dried up, many of the Dartmouth researchers continued their work on AI—and they kept teaching new students. Meanwhile, Moore’s law continued to be accurate, and computers became ever more powerful.
By the 1980s, some of those researchers figured out how to commercialize aspects of AI—and there was now enough compute power and a growing network of researchers who were finding that their work had commercial viability. This reignited interest and, more importantly, the flow of cash into AI. In 1981, Japan announced a 10-year-long plan to develop AI called Fifth Generation. That prompted the US government to form the Microelectronics and Computer Technology Corporation, a research consortium designed to ensure national competitiveness. In the UK, funding that had been cut in the wake of that damning report on AI’s progress by James Lighthill got reinstated. Between 1980 and 1988, the AI industry ballooned from a few million dollars to several billion.
Faster computers, loaded with memory, could now crunch data more effectively, and the focus was on replicating the decision-making processes of human experts, rather than building all-purpose machines like the fictional HAL 9000. These systems were focused primarily on using neural nets for narrow tasks, like playing games. And throughout the ’90
s and early 2000s, there were some exciting successes. In 1994, an AI called CHINOOK played six games of checkers against world champion Marlon Tinsley (all draws). CHINOOK won when Tinsley withdrew from the match and relinquished his championship title.34 In 1997, IBM’s Deep Blue supercomputer beat world chess champion Garry Kasparov, who buckled under the stress of a six-game match against a seemingly unconquerable opponent. In 2004, Ken Jennings won a statistically improbable 74 consecutive games on Jeopardy!, setting a Guinness World Record at that time for the most cash ever won on a game show. So when he accepted a match against IBM’s Watson in 2011, he felt confident he was going to win. He’d taken classes on AI and assumed that the technology wasn’t advanced enough to make sense of context, semantics, and wordplay. Watson crushed Jennings, who started to lose confidence early on in the game.
What we knew by 2011 was that AI now outperformed humans during certain thinking tasks because it could access and process massive amounts of information without succumbing to stress. AI could define stress, but it didn’t have an endocrine system to contend with.
Still, the ancient board game Go was the high-water mark for AI researchers, because it could be played using conventional strategy alone. Go is a game that originated in China more than 3,000 years ago and is played using simple enough rules: two players take turns placing white and black stones on an empty grid. Stones can be captured when they are surrounded by the opposite color or when there are no other open spaces or “liberties.” The goal is to cover territory on the board, but that requires psychology and an astute understanding of the opponent’s state of mind.