Machines of Loving Grace

by John Markoff

  Kay has effectively translated Hegel for the modern age. Today, a wide variety of companies are developing conversational computers like Siri. Kay argues that as a consequence, designers should aim to create programs that function as colleagues rather than servants. If we fail, history hints at a disturbing consequence. Kay worried that building intelligent “assistants” might only recapitulate the problem the Romans faced by letting their Greek slaves do their thinking for them. Before long, those in power were unable to think independently.

  Perhaps we have already begun to slip down a similar path. For example, there is growing evidence that reliance on GPS for directions and for correction of navigational errors hinders our ability to remember and reason spatially, which are more generally useful survival skills.13 “When people ask me, ‘are computers going to take over the world?’” Kay said, “For most people they already have, because they have ceded authority to them in so many different ways.”

  That hints at a second great problem: the risk of ceding individual control over everyday decisions to a cluster of ever more sophisticated algorithms. Not long ago, Randy Komisar, a veteran Silicon Valley venture capitalist, sat in a meeting listening to someone describe a Google service called Google Now, the company’s Siri competitor. “What I realized was that people are dying to have an intelligence tell them what they should be doing,” he said. “What food they should be eating, what people they should be meeting, what parties they should be going to.” For today’s younger generation, the world has been turned upside down, he concluded. Rather than using computers to free them up to think big thoughts, develop close relationships, and exercise their individuality and creativity and freedom, young people were suddenly so starved for direction that they were willing to give up that responsibility to an artificial intelligence in the cloud. What started out as Internet technologies that made it possible for individuals to share preferences efficiently has rapidly transformed into a growing array of algorithms that increasingly dictate those preferences for them. Now the Internet seamlessly serves up life directions. They might be little things like finding the best place nearby for Korean barbecue based on the Internet’s increasingly complete understanding of your individual wants and needs, or big things like an Internet service arranging your marriage—not just the food, gifts, and flowers, but your partner, too.

  The tension inherent in AI and IA perspectives was a puzzle to me when I first realized that Engelbart and McCarthy had set out to invent computer technologies with radically different goals in mind. Obviously they represent both a dichotomy and a paradox. For if you augment a human with computing technology, you inevitably displace humans as well. At the same time, choosing one side or another in the debate is an ethical choice, even if the choice isn’t black or white. Terry Winograd and Jonathan Grudin have separately described the rival communities of scientists and engineers that emerged from that early work. Both men have explored the challenge of fusing the two contradictory approaches. In particular, in 2009 Winograd set out to build a Program on Liberation Technology at Stanford to find ways that computing technologies could improve governance, enfranchise the poor, support human rights, and implement economic development, along with a host of other aims.

  Of course, there are limits to this technology. Winograd makes the case that whether computing technologies are deployed to extend human capabilities or to replace them is more a consequence of the particular economic system in which they are created and used than anything inherent in the technologies themselves. In a capitalist economy, if artificial intelligence technologies improve to the point that they can replace new kinds of white-collar and professional workers, they will inevitably be used in that way. That lesson carries forward in the differing approaches of the software engineers, AI researchers, roboticists, and hackers who are the designers of these future systems. It should be obvious that Bill Joy’s warning that “the future doesn’t need us” is just one possible outcome. It is equally apparent that the world transformed by these technologies doesn’t have to play out catastrophically.

  A little over a century ago, Thorstein Veblen wrote an influential critique of the turn-of-the-century industrial world, The Engineers and the Price System. He argued that, because of the power and influence of industrial technology, political power would flow to engineers, who could parlay their deep knowledge of technology into control of the emerging industrial economy. It certainly didn’t work out that way. Veblen was speaking to the Progressive Era, looking for a middle ground between Marxism and capitalism. Perhaps his timing was off, but his basic point, as echoed a half century later at the dawn of the computer era by Norbert Wiener, may yet prove correct. Today, the engineers who are designing the artificial intelligence–based programs and robots will have tremendous influence over how we will use them. As computer systems are woven more deeply into the fabric of everyday life, the tension between augmentation and artificial intelligence has become increasingly salient.

  What began as a paradox for me has a simple answer. The solution to the contradiction inherent in AI versus IA lies in the very human decisions of engineers and scientists like Bill Duvall, Tom Gruber, Adam Cheyer, Terry Winograd, and Gary Bradski, who all have intentionally chosen human-centered design.

  At the dawn of the computing age, Wiener had a clear sense of the significance of the relationship between humans and their creations—smart machines. He recognized the benefits of automation in eliminating human drudgery, but he also worried that the same technology might subjugate humanity. The intervening decades have only sharpened the dichotomy he first identified.

  This is about us, about humans and the kind of world we will create.

  It’s not about the machines.

  ACKNOWLEDGMENTS

  After reporting on Silicon Valley since 1976, in 2010 I left that beat at the New York Times and moved to the paper’s science section. The events and ideas in this book have their roots in two series that I participated in at the paper while I reported on robotics and artificial intelligence. “Smarter Than You Think” appeared during 2010 and “The iEconomy” in 2012.

  Glenn Kramon, who has worked with me as an editor since we were both at the San Francisco Examiner in the mid-eighties, coined the “Smarter Than You Think” rubric. I am a reporter who values good editors, and Glenn is one of the best.

  The case I made to the paper’s editors in 2010 and the one I describe here is that just as personal computing and the Internet have transformed the world during the past four decades, artificial intelligence and robotics will have an even larger impact during the next several. Despite the fact that our machines are increasingly mimicking our physical and intellectual capabilities, they are still entirely man-made. How they are made will determine the shape of our world.

  Gregg Zachary and I have been both competitors and collaborators for decades, and he remains a close friend with an encyclopedic knowledge of the impact of technology on society. John Kelley, Michael Schrage, and Paul Saffo are also friends who have each had innumerable conversations with me about the shape and consequences of future computing technologies. I have for years had similar conversations with Randy Komisar, Tony Fadell, and Steve Woodward on long bike rides. Jerry Kaplan, who has returned to the world of artificial intelligence after a long hiatus, has real insight into the way it will change the modern world.

  John Brockman, Max Brockman, and Katinka Matson are more than wonderful agents; they are good friends. At HarperCollins my editor, Hilary Redmon, understood that if my last book borrowed its title from a song this one should come from a poem. Her colleague Emma Janaskie was tremendously helpful in navigating all the details that go into producing a book.

  Special thanks to Iris Litt and Margaret Levi, who as directors of the Center for Advanced Study in Behavioral Studies at Stanford University, allowed me to join the community of social scientists in the hills overlooking Silicon Valley. Thanks also to Phil Taubman for introducing me to the Center.

&n
bsp; When I was unable to obtain a visa to report in China in 2012, John Dulchinos pointed me to Drachten and my first factory of the future. In my reporting travels Frank Levy and David Mindell at the Massachusetts Institute of Technology took time to discuss the effects of robotics on the workplace and the economy. Larry Smarr, director of the California Institute for Telecommunications and Information Technology, has frequently hosted me and is always a decade or two ahead in seeing where computing is heading. Mark Stahlman was generous in offering insights on Norbert Wiener and his impact.

  Mark Seiden, whose real-world computing experience stretches back to the first interactive computers, took time away from his work to help with editing, offering technical insight. Anders Fernstedt delved into the archives for gems from Norbert Wiener that had been lost for far too long. He painstakingly went through several of my drafts, offering context and grammar tips.

  Finally, to Leslie Terzian Markoff for sharing it all with me.

  NOTES

  In cases where quotes are not attributed, they are based on the author’s interviews.

  PREFACE

  1.This distinction was famously made by Richard Stallman, an iconoclastic software developer who pioneered the concept of freely shared software.

  1|BETWEEN HUMAN AND MACHINE

  1.John Markoff, What the Dormouse Said: How the Sixties Counterculture Shaped the Personal Computer Industry (New York: Viking, 2005), 282.

  2.Moshe Y. Vardi, “The Consequences of Machine Intelligence,” Atlantic, October 25, 2012, http://www.theatlantic.com/technology/archive/2012/10/the-consequences-of-machine-intelligence/264066.

  3.Frank Levy and Richard J. Murnane, “Dancing with Robots: Human Skills for Computerized Work,” http://content.thirdway.org/publications/714/Dancing-With-Robots.pdf.

  4.J. C. R. Licklider, “Man-Computer Symbiosis,” IRE Transactions on Human Factors in Electronics HFE-1 (March 1960): 4–11, http://groups.csail.mit.edu/medg/people/psz/Licklider.html.

  5.John Markoff, “Can Machines Think? Humans Match Wits,” New York Times, November 9, 1991, http://www.nytimes.com/1991/11/09/us/can-machines-think-humans-match-wits.html.

  6.Jonathan Grudin, “AI and HCI: Two Fields Divided by a Common Focus,” AI Magazine, Winter 2009, http://research.microsoft.com/apps/pubs/default.aspx?id=138574.

  7.John McCarthy, book review of B. P. Bloomfield, The Question of Artificial Intelligence: Philosophical and Sociological Perspectives, in Annals of the History of Computing 10, no. 3 (1988): 224–229.

  8.Kevin Kelly, Out of Control: The Rise of Neo-Biological Civilization (New York: Perseus, 1994), 33–34.

  2|A CRASH IN THE DESERT

  1.Jerry Kaplan, presentation at Stanford University Probabilistic AI lunch meeting, May 6, 2013.

  2.Defense Science Board, “The Role of Autonomy in DoD Systems,” U.S. Department of Defense, July 2012, http://www.acq.osd.mil/dsb/reports/AutonomyReport.pdf.

  3.James R. Hagerty, “A Roboticist’s Trip from Mines to the Moon,” Wall Street Journal, July 2, 2011, http://www.wsj.com/articles/SB10001424052702304569504576405671616928518.

  4.John Markoff, “The Creature That Lives in Pittsburgh,” New York Times, April 21, 1991, http://www.nytimes.com/1991/04/21/business/the-creature-that-lives-in-pittsburgh.html.

  5.John Markoff, “Google Cars Drive Themselves, in Traffic,” New York Times, October 9, 2010, http://www.nytimes.com/2010/10/10/science/10google.html?pagewanted=all.

  6.“Electronic Stability Control Systems for Heavy Vehicles,” National Highway Traffic Safety Administration, 2012, http://www.nhtsa.gov/Laws+&+Regulations/Electronic+Stability+Control+(ESC).

  7.John Markoff, “Police, Pedestrians and the Social Ballet of Merging: The Real Challenges for Self-Driving Cars,” New York Times, May 29, 2014, http://bits.blogs.nytimes.com/2014/05/29/police-bicyclists-and-pedestrians-the-real-challenges-for-self-driving-cars/?_php=true&_type=blogs&_r=0.

  8.Lawrence D. Burns, William C. Jordan, and Bonnie A. Scarborough, “Transforming Personal Mobility,” The Earth Institute, Columbia University, January 27, 2013, http://sustainablemo bility.ei.columbia.edu/files/2012/12/Transforming-Personal-Mobility-Jan-27-20132.pdf.

  9.William Grimes, “Philippa Foot, Renowned Philosopher, Dies at 90,” New York Times, October 9, 2010, http://www.nytimes.com/2010/10/10/us/10foot.html.

  10.“Transportation and Material Moving Occupations,” Occupational Outlook Handbook, Bureau of Labor Statistics, http://www.bls.gov/ooh/transportation-and-material-moving/home.htm.

  11.Vannevar Bush, “As We May Think,” Atlantic Monthly, July 1, 1945, http://www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881.

  12.Peter Norvig, keynote address, NASA Innovative Advanced Concepts Conference, Stanford, California, February 5, 2014.

  3|A TOUGH YEAR FOR THE HUMAN RACE

  1.John Markoff, “Skilled Work, without the Worker,” New York Times, August 18, 2012, http://www.nytimes.com/2012/08/19/business/new-wave-of-adept-robots-is-changing-global-industry.html.

  2.Ibid.

  3.Norbert Wiener, Collected Works with Commentaries, ed. Pesi Masani (Cambridge, MA: MIT Press, 1985), 272.

  4.“Father of Cybernetics Norbert Wiener’s Letter to UAW President Walter Reuther,” August 13, 1949, https://libcom.org/history/father-cybernetics-norbert-wieners-letter-uaw-president-walter-reuther.

  5.Flo Conway and Jim Siegelman, Dark Hero of the Information Age: In Search of Norbert Wiener, The Father of Cybernetics, Kindle ed. (New York: Basic Books, 2009), Kindle location 246.

  6.Anthony Carew, Walter Reuther (Manchester, UK: Manchester University Press, 1993).

  7.Conway, Dark Hero of the Information Age, 246.

  8.Stephen Meyer, “‘An Economic “Frankenstein”’: UAW Workers’ Response to Automation at the Ford Brook Park Plant in the 1950s,” Michigan Historical Review 28 (2002): 63–90.

  9.“Wiener Denounces Devices ‘For War’: M.I.T. Mathematician Rebuffs Bid to Harvard Symposium of Calculating Machinery,” New York Times, January 9, 1947.

  10.Norbert Wiener, “A Scientist Rebels,” Atlantic Monthly, January 1947.

  11.John Markoff, “In 1949, He Imagined an Age of Robots,” New York Times, May 20, 2013, http://www.nytimes.com/2013/05/21/science/mit-scholars-1949-essay-on-machine-age-is-found.html?pagewanted=all.

  12.Ibid.

  13.Ibid.

  14.Carew, Walter Reuther, 144.

  15.The Ad Hoc Committee on the Triple Revolution, “The Triple Revolution,” Liberation, April 1964, http://www.educationanddemocracy.org/FSCfiles/C_CC2a_TripleRevolution.htm.

  16.Mark D. Stahlman, “Wiener’s Genius Project” (invited paper, IEEE 2014 Conference on Norbert Wiener in the 21st Century, 2014).

  17.Steve J. Heims, John von Neumann and Norbert Wiener: From Mathematics to the Technologies of Life and Death (Cambridge, MA: MIT Press, 1980), 343.

  18.Norbert Wiener, God and Golem, Inc.: A Comment on Certain Points where Cybernetics Impinges on Religion (Cambridge, MA: MIT Press, 1964), 29.

  19.“Machines Smarter Than Men? Interview with Dr. Norbert Wiener, Noted Scientist,” U.S. News & World Report, February 24, 1964, http://21stcenturywiener.org/wp-content/uploads/2013/11/Machines-Smarter-Than-Man-Interview-with-Norbert-Wiener.pdf.

  20.Defense Science Board, “The Role of Autonomy in DoD Systems,” U.S. Department of Defense, July 2012, http://www.acq.osd.mil/dsb/reports/AutonomyReport.pdf.

  21.John Maynard Keynes, “Economic Possibilities for Our Grandchildren,” in Essays in Persuasion (New York: W.W. Norton & Co, 1963), 358–373.

  22.Jeremy Rifkin, The End of Work: The Decline of the Global Labor Force and the Dawn of the Post-Market Era (New York: Putnam, 1995), xvii.

  23.John Markoff, “Armies of Expensive Lawyers, Replaced by Cheaper Software,” New York Times, March 4, 2011, http://www.nytimes.com/2011/03/05/science/05legal.html?pagewanted=all.

  24.Erik Brynjolfsson and Andrew McAfee, Race Against the Machine (Lexington, MA: Digital Frontier Press, 2011).


  25.Paul Beaudry, David A. Green, and Ben Sand, “The Great Reversal in the Demand for Skill and Cognitive Tasks,” NBER Working Paper No. 18901, National Bureau of Economic Research, March 2013, http://www.economics.ubc.ca/files/2013/05/pdf_paper_paul-beaudry-great-reversal.pdf.

  26.Ibid.

  27.James Manyika, Susan Lund, Byron Auguste, and Sreenivas Ramaswamy, “Help Wanted: The Future of Work in Advanced Economies,” McKinsey Global Institute, March 2012, http://www.mckinsey.com/insights/employment_and_growth/future_of_work_in_advanced_economies.

  28.Robin Harding, “US Has Lost 2M Clerical Jobs since 2007,” Financial Times, April 1, 2013, http://www.ft.com/intl/cm/s/0/37666e6c-9ae5-11e2-b982-00144feabdc0.html#axzz3V2czZqsP.

  29.Melody Johnson, “Right-Wing Media Attack Obama for Accurate Remarks on Business’ [sic] Investment in Automated Machines,” MediaMatters for America, June 15, 2011, http://mediamatters.org/research/2011/06/15/right-wing-media-attack-obama-for-accurate-rema/180602.

  30.“Are ATMs Stealing Jobs?” Economist, June 15, 2011, http://www.economist.com/blogs/democracyinamerica/2011/06/technology-and-unemployment.

  31.Ben Sumers, “Bank Teller Case Study” (unpublished, 2012).

  32.Erik Brynjolfsson and Andrew McAfee, The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies (New York: W. W. Norton & Co., 2014), 127.