by John Markoff
Later in the meeting, LeCun cornered Sejnowski and the two scientists compared notes. The conversation would lead to the creation of a small fraternity of researchers who would go on to formulate a new model for artificial intelligence. LeCun finished his thesis work on an approach to training neural networks known as “back propagation.” His addition made it possible to automatically “tune” the networks to recognize patterns more accurately.
After leaving school LeCun looked around France to find organizations that were pursuing similar approaches to AI. Finding only a small ministry of science laboratory and a professor who was working in a related field, LeCun obtained funding and laboratory space. His new professor told him, “I’ve no idea what you’re doing, but you seem like a smart guy so I’ll sign the papers.” But he didn’t stay long. First he went off to Geoff Hinton’s neural network group at the University of Toronto, and when the Bell Labs offer arrived he moved to New Jersey, continuing to refine his approach known as convolutional neural nets, initially focusing on the problem of recognizing handwritten characters for automated mail-sorting applications. French-born Canadian Yoshua Bengio, a bright MIT-trained computer scientist, joined him at Bell Labs and worked on the character recognition software, and later on machine vision technology that would be used by the NCR Corporation to automatically read a sizable proportion of all the bank checks circulating in the world.
Yet despite their success, for years the neural network devotees were largely ignored by the mainstream of academic computer science. Thinking of themselves as the “three musketeers,” Hinton, LeCun, and Bengio set out to change that. Beginning in 2004 they embarked on a “conspiracy”—in LeCun’s words—to boost the popularity of the networks, complete with a rebranding campaign offering more alluring concepts of the technology such as “deep learning” and “deep belief nets.” LeCun had by this time moved to New York University, partly for closer ties with neuroscientists and with researchers applying machine-learning algorithms to the problem of vision.
Hinton approached a Canadian foundation, the Canadian Institute for Advanced Research, for support to organize a research effort in the field and to hold several workshops each year. Known as the Neural Computation and Adaptive Perception project, it permitted him to handpick the most suitable researchers in the world across a range of fields stretching from neuroscience to electrical engineering. It helped crystallize a community of people interested in the neural network research.
Terry Sejnowski, Yann LeCun, and Geoffrey Hinton (from left to right), three scientists who helped revive artificial intelligence by developing biologically inspired neural network algorithms. (Photo courtesy of Yann LeCun)
This time they had something else going for them—the pace of computing power had accelerated, making it possible to build neural networks of vast scale, processing data sets orders of magnitude larger than before. It had taken almost a decade, but by then the progress, power, and value of the neural network techniques was indisputable. In addition to raw computing power, the other missing ingredient had been large data sets to use to train the networks. That would change rapidly with the emergence of the global Internet, making possible a new style of centralized computing power—cloud computing—as well as the possibility of connecting that capacity to billions of mobile sensing and computing systems in the form of smartphones. Now the neural networks could be easily trained on millions of digital images or speech samples readily available via the network.
As the success of their techniques became more apparent, Hinton began to receive invitations from different computer companies all looking for ways to increase the accuracy of a wide variety of consumer-oriented artificial intelligence services—speech recognition, machine vision and object recognition, face detection, translation and conversational systems. It seemed like the list was endless. As a consultant Hinton had introduced the deep learning neural net approach early on at Microsoft, and he was vindicated in 2012, when Microsoft’s head of research Richard Rashid gave a lecture in a vast auditorium in Tianjin, China. As the research executive spoke in English he paused after each sentence, which was then immediately translated by software into spoken Chinese in a simulation of his own voice. At the end of the talk, there was silence and then stunned applause from the audience.
The demonstration hadn’t been perfect, but by adding deep learning algorithm techniques the company had adopted from Hinton’s research, it had been able to reduce recognition errors by more than 30 percent. The following year a trickle of interest in neural networks turned into a torrent. The easy availability of Internet data sets and low-cost crowd-sourced labor provided both computing and human resources for training purposes.
Microsoft wasn’t alone. A variety of new neural net and other machine-learning techniques have led to a dramatic revival of interest in AI in Silicon Valley and elsewhere. Combining the new approach to AI with the Internet has meant that it is now possible to create a new service based on computer vision or speech recognition and then use the Internet and tens of millions of smartphone users to immediately reach a global audience.
In 2010 Sebastian Thrun had come to Google to start the Google X Laboratory, which was initially framed inside the company as Google’s version of Xerox’s Palo Alto Research Center. It had a broad portfolio of research projects, stretching from Thrun’s work in autonomous cars to efforts to scale up neural networks, loosely identified as “brain” projects, evoking a new wave of AI.
The Human Brain Project was initially led by Andrew Ng, who had been a colleague with Thrun at the resurrected Stanford Artificial Intelligence Laboratory. Ng was an expert in machine learning and adept in some of the deep learning neural network techniques that Hinton and LeCun had pioneered. In 2011, he began spending time at Google building a machine vision system and the following year it had matured to the point where Google researchers presented a paper on how the network performed in an unsupervised learning experiment using YouTube videos. Training itself on ten million digital images found on YouTube, it performed far better than any previous effort by roughly doubling accuracy in recognizing objects from a challenging list of twenty thousand distinct items. It also taught itself to recognize cats, which is not surprising since there is an overabundance of cat images on YouTube. The Google brain assembled a dreamlike digital image of a cat by employing a hierarchy of memory locations to successively cull out general features after being exposed to millions of images. The scientists described the mechanism as a cybernetic cousin to what takes place in the brain’s visual cortex. The experiment was made possible by Google’s immense computing resources that allowed the researchers to turn loose a cluster of sixteen thousand processors on the problem—which of course is still a tiny fraction of the brain’s billions of neurons, a huge portion of which are devoted to vision.
Whether or not Google is on the trail of a genuine artificial “brain” has become increasingly controversial. There is certainly no question that the deep learning techniques are paying off in a wealth of increasingly powerful AI achievements in vision and speech. And there remains in Silicon Valley a growing group of engineers and scientists who believe they are once again closing in on “Strong AI”—the creation of a self-aware machine with human or greater intelligence.
Ray Kurzweil, the artificial intelligence researcher and barnstorming advocate for technologically induced immortality, joined Google in 2013 to take over the brain work from Ng, shortly after publishing How to Create a Mind, a book that purported to offer a recipe for creating a working AI. Kurzweil, of course, has all along been one of the most eloquent backers of the idea of a singularity. Like Moravec, he posits a great acceleration of computing power that would lead to the emergence of autonomous superhuman machine intelligence, in Kurzweil’s case pegging the date to sometime around 2023. The idea became codified in Silicon Valley in the form of the Singularity University and the Singularity Institute, organizations that focused on dealing with the consequences of that exponentia
l acceleration.
Joining Kurzweil are a diverse group of scientists and engineers who believe that once they have discovered the mechanism underlying the biological human neuron, it will be simply a matter of scaling it up to create an AI. Jeff Hawkins, a successful Silicon Valley engineer who had founded Palm Computing with Donna Dubinsky, coauthored On Intelligence in 2004, which argued that the path to human-level intelligence lay in emulating and scaling up neocortex-like circuits capable of pattern recognition. In 2005, Hawkins formed Numenta, one of a growing list of AI companies pursuing pattern recognition technologies. Hawkins’s theory has parallels with the claims that Kurzweil makes in How to Create a Mind, his 2012 effort to lay out a recipe for intelligence. Similar paths have been pursued by Dileep George, a Stanford-educated artificial intelligence researcher who originally worked with Hawkins at Numenta and then left to form his own company, Vicarious, with the goal of developing “the next generation of AI algorithms,” and Henry Markram, the Swiss researcher who has enticed the European Union into supporting his effort to build a detailed replica of the human brain with one billion euros in funding.
In 2013 a technology talent gold rush that was already under way reached startling levels. Hinton left for Google because the resources available in Mountain View dwarfed what he had access to at the University of Toronto. There is now vastly more computing power available than when Sejnowski and Hinton first developed the Boltzmann Machine approach to neural networks, and there is vastly more data to train the networks on. The challenge now is managing a neural network that might have one billion parameters. To a conventional statistician that’s a nightmare, but it has spawned a sprawling “big data” industry that does not shy away from monitoring and collecting virtually every aspect of human behavior, interaction, and thought.
After his arrival at Google, Hinton promptly published a significant breakthrough in making more powerful and efficient learning networks by discovering how to keep the parameters from effectively stepping on each other’s toes. Rather than have an entire network process the whole image simultaneously, in the new model a subset is chosen, a portion of the image is processed, and the weights of the connections are updated. Then another random set is picked and the image is processed again. It offers a way to use randomness to reinforce the influence of each subset. The insight might be biologically inspired, but it’s not a slavish copy. By Sejnowski’s account, Hinton is an example of an artificial intelligence researcher who pays attention to the biology but is not constrained by it.
In 2012 Hinton’s networks, trained on a huge farm of computers at Google, did remarkably well at recognizing individual objects, but they weren’t capable of “scene understanding.” For example, the networks could not recognize the sentence: “There is a cat sitting on the mat and there is a person dangling a toy at the cat.” The holy grail of computer vision requires what AI researchers call “semantic understanding”—the ability to interpret the scene in terms of human language. In the 1970s the challenge of scene understanding was strongly influenced by Noam Chomsky’s ideas about generative grammar as a context for objects and a structure for understanding their relation within a scene. But for decades the research went nowhere.
However, late in 2014, the neural network community began to make transformative progress in this domain as well. Around the country research groups reported progress in combining the learning properties of two different types of neural networks, one to recognize patterns in human language and the other to recognize patterns in digital images. Strikingly, they produced programs that could generate English-language sentences that described images at a high level of abstraction.44 The advance will help in applications that improve the results generated by Internet image search applications. The new approach also holds out the potential for creating a class of programs that can interact with humans with a more sophisticated level of understanding.
Deep learning nets have made significant advances, but for Hinton, the journey is only now beginning. He said recently that he sees himself as an explorer who has landed on a new continent and it’s all very interesting, but he has only progressed a hundred yards inland and it’s still looking very interesting—except for the mosquitoes. In the end, however, it’s a new continent and the researchers still have no idea what is really possible.
In late 2013, LeCun followed Hinton’s move from academia to industry. He agreed to set up and lead Facebook’s AI research laboratory in New York City. The move underscored the renewed corporate enthusiasm for artificial intelligence. The AI Winter was only the dimmest of memories. It was now clearly AI Spring.
Facebook’s move to join the AI gold rush was an odd affair. It began with a visit by Mark Zuckerberg, Facebook cofounder and chief executive, to an out-of-the-way technical conference called Neural Information Processing Systems, or NIPS, held in a Lake Tahoe hotel at the end of 2013. The meeting had long been a bone-dry academic event, but Zuckerberg’s appearance to answer questions was a clear bellwether. Not only were the researchers unused to appearances by high-visibility corporate tycoons, but Zuckerberg was accompanied by uniformed guards, lending the event a surreal quality. The celebrity CEO filled the room he was in and several other workshops were postponed as a video feed was piped into an overflow room. “The tone changed rapidly: accomplished professors became little more than lowly researchers shuffling into the Deep Learning workshop to see a Very Important Person speak,”45 blogged Alex Rubinsteyn, a machine-learning researcher who was an attendee at the NIPS meeting.
In the aftermath of the event there was an alarmed back-and-forth inside the tiny community of researchers about the impact of commercialization of AI on the culture of the academic research community. It was, however, too late to turn back. The field has moved on from the intellectual quarrels in the 1950s and 1960s over the feasibility of AI and the question of the correct path. Today, a series of probabilistic mathematical techniques have reinvented the field and transformed it from an academic curiosity into a force that is altering many aspects of the modern world.
It has also created an increasingly clear choice for designers. It is now possible to design humans into or out of the computerized systems that are being created to grow our food, transport us, manufacture our goods and services, and entertain us. It has become a philosophical and ethical choice, rather than simply a technical one. Indeed, the explosion of computing power and its accessibility everywhere via wireless networks has reframed with new urgency the question addressed so differently by McCarthy and Engelbart at the dawn of the computing age.
In the future will important decisions be made by humans or by the deep learning–style algorithms? Today, the computing world is demarcated between those who focus on creating intelligent machines and those who focus on how human capabilities can be extended by the same machines. Will it surprise anyone that the differing futures emerging from those opposing stances must be very different worlds?
5|WALKING AWAY
As a young navy technician in the 1950s, Robert Taylor had a lot of experience flying, even without a pilot’s license. He had become a favorite copilot of the real pilots, who needed both lots of flight hours and time to study for exams. So they would take Taylor along in their training jets, and after they took off he would fly the plane—gently—while the real pilots studied in the backseat. He even practiced instrument landing approaches, in which the plane is guided to a landing by radio communications, while the pilot wears a hood blocking the view of the outside terrain.
As a young NASA program administrator in the early 1960s, Taylor was confident when he received an invitation to take part in a test flight at a Cornell University aerospace laboratory. On arrival they put him in an uncomfortable anti-g suit and plunked him down in the front seat of a Lockheed T-33 jet trainer while the real pilot sat behind him. They took off and the pilot flew up to the jet’s maximum altitude, almost fifty thousand feet, then Taylor was offered the controls to fly around a bit. After a while the pi
lot said, “Let’s try something a little more interesting. Why don’t you put the plane in a dive?” So Taylor pushed the joystick forward until he thought he was descending steeply enough, then he began to ease the stick backward. Suddenly he froze in panic. As he pulled back, the plane entered a steeper dive. It felt like going over the top of a roller-coaster ride. He pulled the stick back farther but the plane was still descending almost vertically.
Finally he said to the pilot behind him, “Okay, you’ve got it, you better take over!” The pilot laughed, leveled the plane out, and said, “Let’s try this again.” They tried again and this time when he pushed the stick forward the plane went unexpectedly upward. As he pushed a bit harder, the plane tilted up farther. This time he panicked, about to stall the plane, and again the pilot leveled the plane out.
Taylor should have guessed. He had had such an odd piloting experience because he was unwittingly flying a laboratory plane that the air force researchers were using to experiment with flight control systems. The air force invited Taylor to Cornell because as a NASA program manager he had granted, unsolicited, $100,000 to a flight research group at Wright-Patterson Air Force Base.