Machines of Loving Grace

by John Markoff

  Governed by a decidedly socialist outlook, the group eschewed the idea of personal computing. Computing should be a social and shared experience, Community Memory decreed. It was an idea before its time. Twelve years before AOL and the Well were founded, and seven years before dial-up BBSes became popular, Community Memory’s innovators built and operated bulletin boards, social media, and electronic communities from other people’s cast-offs. The first version of the project lasted only until 1975 before shutting down.

  Felsenstein had none of the anti-PC bias shared by both his radical friends and John McCarthy. Thus unencumbered, he became one of the pioneers of personal computing. Not only was Felsenstein one of the founding members of the Homebrew Computer Club, he also designed the Sol-20, an early hobbyist computer released in 1976, followed up in 1981 with the Osborne 1, the first mass-produced portable computer. Indeed, Felsenstein had a broad view of the impact of computing on society. He had grown up in a household where Norbert Wiener’s The Human Use of Human Beings held a prominent place on the family bookshelf. His father had considered himself not merely a political radical but a modernist as well. Felsenstein would later write that his father, Jake, “was a modernist who believed in the perfectibility of man and the machine as the model for human society. In play with his children he would often imitate a steam locomotive in the same fashion other fathers would imitate animals.”9

  The discussion of the impact of technology had been a common trope in the Felsenstein household in the late fifties and the early sixties before Felsenstein left for college. The family discussed the impact of automation and the possibility of technological unemployment with great concern. Lee had even found and read a copy of Wiener’s God and Golem, Inc., published the year that Wiener had unexpectedly died while visiting Stockholm and consisting mostly of his premonitions, both dire and enthusiastic, about the consequences of machines and automation on man and society. To Felsenstein, Wiener was a personal hero.

  Despite his early interest in robots and computing, Felsenstein had never been enthralled with rule-based artificial intelligence. Learning about Engelbart’s intelligence amplification ideas would change the way he thought about computing. In the mid-1970s Engelbart’s ideas were in the air among computer hobbyists in Silicon Valley. Felsenstein, and a host of others like him, were dreaming about what they could do with their own computers. In 1977, the second year of the personal computer era, he listened to a friend and fellow computer hobbyist, Steve Dompier, talk about the way he wanted to use a computer. Dompier described a future user interface that would be designed like a flight simulator. The user would “fly” through a computer file structure, much in the way 3-D computer programs now simulate flying over virtual terrain.

  Felsenstein’s thinking would follow in Dompier’s footsteps. He developed the idea of “play-based” interaction. Ultimately he extended the idea to both user interface design and robotics. Traditional robotics, Felsenstein decided, would lead to machines that would displace humans, but “golemics,” as he described it, using a term first introduced by Norbert Wiener, was the right relationship between human and machine. Wiener had used “golemic” to describe the pretechnological world. In his “The Golemic Approach,”10 Felsenstein presented a design philosophy for building automated machines in which the human user was incorporated into a system with a tight feedback loop between the machine and the human. In Felsenstein’s design, the human should retain a high level of skill to operate the system. It was a radically different approach compared to conventional robotics, in which human expertise was “canned” in the robot while the human remained passive.

  For Felsenstein, the automobile was a good analogy for the ideal golemic device. Automobiles autonomously managed a good deal of their own functions—automatic transmission, braking, and these days advanced cruise control and lane keeping—but in the end people maintained control of their cars. The human, in NASA parlance, remained very much in the loop.

  Felsenstein first published his ideas as a manifesto in the 1979 Proceedings of the West Coast Computer Faire. The computer hobbyist movement in the mid-1970s had found its home at this annual computer event, which was created and curated by a former math teacher and would-be hippie named Jim Warren. When Felsenstein articulated his ideas, the sixties had already ended, but he remained very much a utopian: “Given the application of the golemic outlook, we can look forward, I believe, to a society in which rather than bringing about the displacement of people from useful and rewarding work, machines will effect a blurring of the distinction between work and play.”11 Still, when Felsenstein wrote his essay in the late 1970s it was possible that the golem could evolve either as a collaborator or as a Frankenstein-like monster.

  Although he was instrumental in elevating personal computing from its hobbyist roots into a huge industry, Felsenstein was largely forgotten until very recently. During the 1990s he had worked as a design engineer at Interval Research Corporation, and then set up a small consulting business just off California Avenue in Palo Alto, down the street from where Google’s robotics division is located today. Felsenstein held on to his political ideals and worked on a variety of engineering projects ranging from hearing aids to parapsychology research tools. He was hurled back onto the national stage in 2014 when he became a target for Evgeny Morozov, the sharp-penned intellectual from Belarus who specializes in quasi-academic takedowns of Internet highfliers and exposing post-dot-com era foibles. In a New Yorker essay12 aiming at what he found questionable about the generally benign and inclusive Maker Movement, Morozov zeroed in on Felsenstein’s Homebrew roots and utopian ideals as expressed in a 1995 oral history. In this interview, Felsenstein described how his father had introduced him to Tools for Conviviality by Ivan Illich, a radical ex-priest who had been an influential voice for the political Left in the 1960s and 1970s counterculture. Felsenstein had been attracted to Illich’s nondogmatic attitude toward technology, which contrasted “convivial,” human-centered technologies with “industrial” ones. Illich had written largely before the microprocessor had decentralized computing and he saw computers as tools for instituting and maintaining centralized, bureaucratic control. In contrast, he had seen how radio had been introduced into Central America and rapidly became a bottom-up technology that empowered, instead of oppressed, people. Felsenstein believed the same was potentially true for computing.13

  Morozov wanted to prove that Felsenstein and by extension the Maker Movement that carries on his legacy are naive to believe that society could be transformed through tools alone. He wrote that “society is always in flux” and further that “the designer can’t predict how various political, social, and economic systems will come to blunt, augment, or redirect the power of the tool that is being designed.” The political answer, Morozov argued, should have been to transform the hacker movements into traditional political campaigns to capture transparency and democracy.

  It is an impressive rant, but Morozov’s proposed solution was as ineffective as the straw man he set up and sought applause for tearing down. He focused on Steve Jobs’s genius in purportedly not caring whether the personal computing technology he was helping pioneer in the mid-1970s was open or not. He gave Jobs credit for seeing the computer as a powerful augmentation tool. However, Morozov entirely missed the codependency between Jobs the entrepreneur and Wozniak the designer and hacker. It might well be possible to have one without the other, but that wasn’t how Apple became so successful. By focusing on the idea that Illich was only interested in simple technologies that were within the reach of nontechnical users, Morozov rigged an argument so he would win.

  However, the power of “convivial” technologies, which was Illich’s name for tools that are under individual control, remains a vitally important design point that is possibly even more relevant today. Evidence of this was apparent in an interaction between Felsenstein and Illich, when the radical scholar visited Berkeley in 1986. Upon meeting him, Illich mocked Felsenstein for trying t
o substitute communication using computers for direct communication. “Why do you want to go deet-deet-deet to talk to Pearl over there? Why don’t you just go talk to Pearl?” Illich asked.

  Felsenstein responded: “What if I didn’t know that it was Pearl that I wanted to talk to?”

  Illich stopped, thought, and said, “I see what you mean.”

  To which Felsenstein replied: “So you see, maybe a bicycle society needs a computer.”

  Felsenstein had convinced Illich that their communication could create community even if it was not face-to-face. Given the rapid progress in robotics, Felsenstein and Illich’s insight about design and control is even more important today. In Felsenstein’s world, drudgery would be the province of machines and work would be transformed into play. As he described it in the context of his proposed “Tom Swift Terminal,”14 which was a hobbyist system that foreshadowed the first PCs, “if work is to become play, then tools must become toys.”

  Today, Microsoft’s corporate campus is a sprawling set of interlocking walkways, buildings, sports fields, cafeterias, and parking garages dotted with fir trees. In some distinct ways it feels different from the Googleplex in Silicon Valley. There are no brightly colored bicycles, but the same cadres of young tech workers who could easily pass for college or even high school students amble around the campus.

  When you approach the elevator in the lobby of Building 99, where the firm’s corporate research laboratories are housed, the door senses your presence and opens automatically. It feels like Star Trek: Captain Kirk never pushed a button either. The intelligent elevator is the brainchild of Eric Horvitz, a senior Microsoft research scientist and director of Microsoft’s Redmond Research Center. Horvitz is well known among AI researchers as one of the first generation of computer scientists to use statistical techniques to improve the performance of AI applications.

  He, like many others, began with an intense interest in understanding how human minds work. He obtained a medical degree at Stanford during the 1980s, and soon immersed himself further in graduate-level neurobiology research. One night in the laboratory he was using a probe to insert a single neuron into the brain of a rat. Horvitz was thrilled. It was a dark room and he had an oscilloscope and an audio speaker. As he listened to the neuron fire, he thought to himself, “I’m finally inside. I am somewhere in the midst of vertebrate thought.” At the same moment he realized that he had no idea what the firing actually suggested about the animal’s thought process. Glancing over toward his laboratory bench he noticed a recently introduced Apple IIe computer with its cover slid off to the side. His heart sank. He realized that he was taking a fundamentally wrong approach. What he was doing was no different from taking the same probe and randomly sticking it inside the computer in search of an understanding of the computer’s software.

  He left medicine, shifting his course of study, and started taking cognitive psychology and computer science courses. He adopted Herbert Simon, the Carnegie Mellon cognitive scientist and AI pioneer, as an across-the-country mentor. He also became close to Judea Pearl, the UCLA computer science professor who had pioneered an approach to artificial intelligence breaking with the early logic- and rule-based approach, instead focusing on recognizing patterns by building nesting webs of probabilities. This approach is not conceptually far from the neural network ideas so harshly criticized by Minsky and Papert in the 1960s. As a result, during the 1980s at Stanford, Horvitz was outside the mainstream in computer science research. Many mainstream AI researchers thought his interest in probability theory was dated, a throwback to an earlier generation of “control theory” methods.

  After he arrived at Microsoft Research in 1993, Horvitz was given a mandate to build a group to develop AI techniques to improve the company’s commercial products. Microsoft’s Office Assistant, a.k.a. Clippy, was first introduced in 1997 to help users master hard-to-use software, and it was largely a product of the work of Horvitz’s group at Microsoft Research. Unfortunately, it became known as a laughingstock failure in human-computer interaction design. It was so widely reviled that Microsoft’s thriller-style promotional video for Office 2010 featured Clippy’s gravestone, dead in 2004 at the age of seven.15

  The failure of Clippy offered a unique window into the internal politics at Microsoft. Horvitz’s research group had pioneered the idea of an intelligent assistant, but Microsoft Research—and hence Horvitz’s group—was at that point almost entirely separate from Microsoft’s product development department. In 2005, after Microsoft had killed the Office Assistant technology, Steven Sinofsky, the veteran head of Office engineering, described the attitude toward the technology during program development: “The actual feature name used in the product is never what we named it during development—the Office Assistant was famously named TFC during development. The ‘C’ stood for clown. I will let your active imagination figure out what the TF stood for.”16 It was clear that the company’s software engineers had no respect for the idea of an intelligent assistant from the outset. Because Horvitz and his group couldn’t secure enough commitment from the product development group for Clippy, Clippy fell by the wayside.

  The original, more general concept of the intelligent office assistant, which Horvitz’s research group had described in a 1998 paper, was very different from what Microsoft later commercialized. The final shipping version of the assistant omitted software intelligence that would have prevented the assistant from constantly popping up on the screen with friendly advice. The constant intrusions drove many users to distraction and the feature was irreversibly—perhaps prematurely—rejected by Microsoft’s customers. However, the company chose not to publicly explain why the features required to make Clippy work well were left out. A graduate student once asked Horvitz this after a public lecture and the response given was that the features had bloated Office 97 to such an extent that it would no longer fit on its intended distribution disk.17 (Before the Internet offered feature updates, leaving something out was the only practical option.)

  Such are the politics of large corporations, but Horvitz would persist. Today, a helpful personal assistant—who resides inside a computer monitor—greets visitors to his fourth-floor glass-walled corner cubicle. The monitor is perched on a cart outside his office, and the display shows the cartoon head of someone who looks just like Max Headroom, the star of the British television series about a stuttering artificial intelligence that incorporated the dying memories of Edison Carter, an earnest investigative reporter. Today Horvitz’s computerized greeter can inform visitors of where he is, set up appointments, or suggest when he’ll next be available. It tracks almost a dozen aspects of Horvitz’s work life, including his location and how busy he is likely to be at any moment during the day.

  Horvitz has remained focused on systems that augment humans. His researchers design applications that can monitor a doctor and patient or other essential conversation, offering support so as to eliminate potentially deadly misperceptions. In another application, his research team maintains a book of morbid transcripts from plane crashes to map what can go wrong between pilots and air traffic control towers. The classic and tragic example of miscommunication between pilots and air traffic control is the Tenerife Airport disaster of 1977, during which two 747 jetliners were navigating a dense fog without ground radar and collided while one was taxiing and the other was taking off, killing 583 people.18 There is a moment in the transcript where two people attempt to speak at the same time, causing interference that renders a portion of the conversation unintelligible. One goal in the Horvitz lab is to develop ways to avoid these kinds of tragedies. When developers integrate machine learning and decision-making capabilities into AI systems, Horvitz believes that those systems will be able to reason about human conversations and then make judgments about what part of a problem people are best capable to solve and what part should be filtered through machines. The ubiquitous availability of cheap computing and the Internet has made it easier for these systems to show resul
ts and gain traction, and there are already several examples of this kind of augmentation on the market today. As early as 2005, for example, two chess amateurs used a chess-playing software program to win a match against chess experts and individual chess-playing programs.

  Horvitz is continuing to deepen the human-machine interaction by researching ways to couple machine learning and computerized decision-making with human intelligence. For example, his researchers have worked closely with the designers of the crowd-sourced citizen science tool called Galaxy Zoo, harnessing armies of human Web surfers to categorize images of galaxies. Crowd-sourced labor is becoming a significant resource in scientific research: professional scientists can enlist amateurs, who often need to do little more than play elaborate games that exploit human perception, in order to help scientists map tricky problems like protein folding.19 In a number of documented cases teams of human experts have exceeded the capability of some of the most powerful supercomputers.

  By assembling ensembles of humans and machines and designating a specific research task for each group, scientists can create a powerful hybrid research team. The computers possess staggering image recognition capabilities and they can create tables of the hundreds of visual and analytic features for every galaxy currently observable by the world’s telescopes. That was very inexpensive but did not yield perfect results. In the next version of the program, dubbed Galaxy Zoo 2, computers with machine-learning models would interpret the images of the galaxies in order to present accurate specimens to human classifiers, who could then catalog galaxies with much less effort than they had in the past. In yet another refinement, the system would add the ability to recognize the particular skills of different human participants and leverage them appropriately. Galaxy Zoo 2 was able to automatically categorize the problems it faced and knew which people could contribute to solving which problem most effectively.