We also need interpreters of what’s going on in these systems, a bit like TV meteorologists. Near the end of Average Is Over, the economist Tyler Cowen speculates about this new breed of future interpreters. He says they “will hone their skills of seeking out, absorbing, and evaluating this information. . . . They will be translators of the truths coming out of our networks of machines. . . . At least for a while, they will be the only people left who will have a clear notion of what is going on.” These interpreters—who will likely be comfortable with simulations—can help provide us with a glimmer of intuition into complex systems. As more and more of our technology becomes impenetrable, schools and universities will need to incorporate playing with simulations and creating simplifying intuitions as necessary skills—skills in the domain of the science of complexity—equipping each of us to become our own interpreter of these systems.
Simplifying intuitions—the kind of general models developed by complexity scientists—can lay the groundwork for heuristics on how to approach these systems without getting overwhelmed by their complexity. Along with muddling through—the perspective of John Gall’s “systemantics”—these little windows into massively complex systems embody the humble approach to technology that we need to adopt.
And there is yet one more positive mind-set we can resort to when we face complex technological systems.
Fostering Naches
The Yiddish language possesses a wonderful word, naches. It means pride or joy. To shep naches, as it’s said in Yiddish, is generally to derive vicarious pride from the accomplishments of those close to you, especially your children. It is one of the purest pleasures, and one that you hear spoken of during bar mitzvahs, weddings, and graduations.
Immigrants feel it is important for the next generation to be better off, and for the generation after that to positively thrive. For parents, their offspring must always be more intelligent and more successful than they are, and the drive to make this happen is no doubt part of the reason why we have such naches. So, why should we not shep naches from the accomplishments of our technologies?
We might not understand what these machines or systems are doing. In some cases, it was even other pieces of technology that constructed them. But they are our brainchildren, our intellectual offspring, and they, too, can give us naches. Humans have valued this emotion for thousands of years, and it brings us great happiness. We just need to transfer our parental pride to the technological realm.
So what does naches mean exactly for technology? At the most basic level, the creators of these machines can shep naches from the accomplishments and powers of their technological progeny. Among all their other achievements, computer programs are now even capable of generating sophisticated artworks or musical compositions. The composer David Cope of the University of California, Santa Cruz, has developed software that generates novel musical compositions in the style of a given composer. And they sound really good. A Scott Joplin–style composition sounds like Joplin. While Cope didn’t create these songs directly, he still can take pride in their construction. His computational creations can provide him with naches. Similarly, the creators of IBM’s Watson might shep naches from the machine’s win over its human opponents on Jeopardy!
We can broaden this sense of naches still more. Many of us support a sports team and take pride in its wins, even though we had nothing to do with them. Or we become excited when a citizen of our country takes the gold in the Olympics, or makes a new discovery and is awarded a prestigious prize. So, too, should it be with our machines for all humanity: we can root for what humans have created, even if it wasn’t our own personal achievement and even if we can’t fully understand it. Many of us are grateful for technological advances, from the iPhone to the Internet, even if we don’t know how they work.
If our creations’ complexity outstrips our ability to understand them, we should not be disturbed or disappointed. When our children do something surprising and amazing that we can’t really understand, we don’t despair or worry; we are delighted and even grateful for their success. We can respond similarly to our technological creations.
But naches is also a framework for helping us recognize that we are following the same trajectory we have been tracing for thousands of years, in which fewer and fewer people are able to understand the most complex parts of the world we have created. These recent trends are not really so new or different from what has come before.
Humility refracted through the lens of naches is an optimistic perspective on the incomprehensibility to which we have given birth. Furthermore, this humility is a recognition of the balance between our unceasing desires—and in the case of experts, almost a mandate—to grasp the dimensions of what we have constructed and our limits in doing so.
Humility can also clarify a distinction between mystery and wonder.
Mystery and Wonder
A book published in the second half of the nineteenth century called The World of Wonders is a grab bag of intriguing ideas, fascinating things, and strange events. It is written in the manner of someone delighted with life and constantly curious, interspersing anthropic advances with phenomena from nature. It includes “all that is most wonderful in history and philosophy and the marvels of science, the wonders of animal life revealed by the glass of the optician or the labours of the chemist,” and much more. It is a marvel of the Victorian period and demonstrates the many ways that we can find wonder all around us. But this wasn’t wonder at a world that was unexplained. It was wonder precisely because our advances and technologies were so interesting, so powerful, and above all, understandable—not mysterious.
There are two potential extreme responses to mystery in our complex technological systems. The first is to underemphasize it, to such a degree that there are no mysteries. We tell ourselves simplifying stories about how things work, stories that, while appealing, severely underestimate the complexity of our systems. We say that we understand a technology we are using, and any issue is just a small glitch that needs to be ironed out. Many large technology companies tend toward this style, dismissing any unexpected behaviors as minor issues in the process of being eliminated.
At the other extreme are those who actively seek out mystery and the unknown, whether or not it’s actually there. These people—often laypeople—wear their mystification at a device’s or system’s inner workings almost as a badge of honor, declaring their iPhones or the ways of the electrical grid to be nearly magical. These are the kindred spirits of Calvin, friend to the philosophical tiger, who asked his father how lightbulbs and vacuum cleaners worked, and was content with the answer: “Magic.” Many of us, though, lie somewhere in the middle. We might recognize a certain amount of mystery, yet still want the world to be explainable.
Things get messier when we bring in wonder, our ability to marvel and to feel a sense of the numinous in the world around us. We may delight in the complexity or beauty of a phenomenon, but, as is clear from The World of Wonders, wonder doesn’t have to be at the expense of understanding—or vice versa. In fact, many people find it more gratifying the more they can grapple with and succeed in understanding a really complicated system. Don Norman has written of the delight in truly understanding the infield fly rule in baseball. In a complicated rule set—baseball’s rules are over 200 pages long—just the definition of an infield fly contains a thicket of exceptions and qualifiers: “An infield fly is a fair fly ball (not including a line drive nor an attempted bunt) which can be caught by an infielder with ordinary effort, when first and second, or first, second and third bases are occupied, before two are out.” Fans enjoy working hard to understand this complex system, and find wonder in it.
So how do we balance mystery and wonder when it comes to these vast systems of our own making, specifically those that are—no matter how hard we try—not fully comprehensible?
We mustn’t give in to the temptation to collapse these two senses together—to
conflate mystery and wonder. Those who do this argue that the unexplained must necessarily inspire us. The mysterious is incorrectly identified with the wondrous: something not understood should therefore be marveled at. Similarly, we must resist the temptation to abandon wonder once we understand something completely. This happens too often with technological change. Changes are happening so rapidly that we forget to marvel at how impressive our understanding of the universe—and our ability to harness it—has become. We forget how recently we gained the ability to render three-dimensional worlds on our screens, communicate instantly across the planet, or even summon decades-old television commercials with the click of a mouse. The knowledge that has made these changes possible too often fails to inspire wonder. Then we are left with “the sad inertness of a world explained and controlled,” as the philosophers Hubert Dreyfus and Sean Dorrance Kelly describe the disenchantment that threatens when wonder declines as our technological powers rise—driving some into fantasies of an enchanted past.
Instead, we must work to maintain two opposing states: mystery without wonder and wonder without mystery. The first requires that we strive to eliminate our ignorance, rather than simply reveling in it. And the second means that once we understand something, we do not take it for granted.
In essence, these two final combinations are what I think of as the scientific mind-set, the prerequisite for our ability to learn new things and solve the puzzles that confront us. When we wonder too much at the mysteries we have inadvertently created, we undercut our desire to eliminate those mysteries and understand our Entanglement.
We will never fully understand it. The kluges are too complex. We will always be left with some mystery, but that’s okay. As long as we neither fear nor revel in it, we can take the proper perspective: humility, even with a touch of awe. Because we built these systems. Humility, wonder, and even some naches might be the best that we can do. We must continue to pursue a biological mind-set toward these systems, even if we don’t gain a full understanding. We can be humble if we fail, but there are paths for our intuition, glimpses into our technologies.
• • •
When vehicles manufactured by Toyota began accelerating unexpectedly, even resulting in deaths, experts could not figure out exactly what had caused the error. One suggestion was to inform Toyota drivers that if their cars ever began to accelerate uncontrollably, a simple solution involved shifting the car into neutral. Of course, when you are a consumer entrusting your life and your family’s lives to a powerful machine that might behave in an unpredictable and frightening manner, that’s not what you want to hear. It sounds almost like a callous Band-Aid slapped onto a deeper problem.
If we know that rare failures and unexplainable glitches are by-products of the evolution of complexity, however, and that even the most vigilant engineering cannot anticipate them all, we will be better prepared to respond, should they occur. In that case, “Just shift into neutral” isn’t the worst thing to tell someone who is about to step into a fantastically complicated machine, if they accept that these messy, incomprehensible systems are the new reality. Both designers and drivers are now partners in pioneering the Entanglement and are not as far apart as they might seem, at least in one important respect: neither can fully predict the behavior of a complex system. Many of us already recognize this, albeit in a much more minor way and even if only subconsciously, smacking a machine in frustration or simply turning our computers off and then on again, in the hope that the complexity will resolve itself in our favor.
The twentieth century brought us numerous limitative theorems, statements that placed bounds upon what we could ever know and understand. Kurt Gödel showed that in mathematics there will always be statements that can never be proven as either true or false, within a given mathematical system. In computer science, Alan Turing demonstrated the limits on what any machine could ever do, no matter how fancy an algorithm one might develop. But neither of these fields died out or suffered a huge setback. Despite being bounded by limits, they flourished, in many ways far beyond what these men could have ever imagined.
In building and using complex technological systems, there are limits to what we can understand about how they work and how they fail. This does not mean an end to such creations. Rather, it simply means that as we continue to build such systems, ones that will continue to grow ever more impressive and sophisticated, we must recalibrate our expectations.
Starting from the recognition that we can’t fully understand these systems changes how we approach them. If you’re involved in their construction, you can assemble them with an awareness of our limits of comprehension. If you interact with such a system, you can recognize that bugginess and unexpectedness are common, not the exceptions, and that if you try to eliminate them, not only will you fail, but you may even make things worse.
The science fiction writer William Gibson has spoken of the “unthinkable present,” the setting for many of his stories. This phrase contains the kernel of what we are currently living inside of, this Entanglement. But in the end, these systems are not completely incomprehensible, nor will their properties and qualities remain forever ineffable. The story doesn’t end with horror at the unknowable or wonder at a completely impenetrable mystery. Humility tempered with the acknowledgment of an iterative and biological approach to understanding must guide our interactions with our overcomplicated technology. But no matter the details of our perspective, we do not have to give in to the logic of Ludwig Wittgenstein: “What we cannot speak about we must pass over in silence.”
There is much to be said about what we can’t fully understand and how to respond to it. We can live and thrive in this Entanglement.
Further Reading
This book did not emerge whole cloth. Its ideas grew out of and were drawn from a very large number of sources. In addition to what I reference in the endnotes, I’d like to highlight several books and articles that were particularly thought-provoking and helped inform some of my thinking when writing this book. Some of these references aren’t even discussed in this text but could be useful if you are interested in delving further into the topics I touch upon.
In addition, there are many topics that, while relevant, I did not discuss in detail in this book. These include such things as technological change and how that affects education, more on Big Data, our increasing partnership with our machines, automation and the future of jobs, and the many, many different types of systems that we see around us, including those in manufacturing, food, government, energy, and so much more. Some of the works below address these topics.
• • •
The Systems Bible by John Gall is a bizarre romp into how to think about large systems and how they work, or don’t. It is fascinating and much of my thinking parallels Gall’s.
Out of Control by Kevin Kelly includes some of the same points about biological thinking and how technology is becoming increasingly biological and unable to be understood, though from the perspective of emergence and biological complexity and the use of biological principles to build technologies. I also recommend What Technology Wants by the same author.
Autonomous Technology by Langdon Winner. The penultimate chapter is particularly salient and raises many of the same issues discussed in this book.
The Ingenuity Gap by Thomas Homer-Dixon is about how to deal with an increasingly complex and unpredictable world. It explores all manner of systems, many of those beyond the scope of this book, and is fascinating.
Normal Accidents by Charles Perrow is a classic text on failure and “living with high-risk technologies.”
Geek Sublime by Vikram Chandra is a beautiful meditation on the nature of programming and computers, among many fascinating topics.
Infinite in All Directions by Freeman Dyson includes some great discussions on the nature of scientific thinking and discovery.
The Mythical Man-Month by Frederick Brooks is a classic exploration of
software development and design. It focuses on the management of engineering teams but has huge amounts of interesting thoughts on the nature of software and programming more generally.
Curiosity: How Science Became Interested in Everything by Philip Ball is a great account of how scientists, especially the early ones, began exploring our incredibly diverse and detailed world and attempted to impose upon it a sense of order.
Living with Complexity by Don Norman examines the origins of (and need for) complexity, particularly from the perspective of design.
The Techno-Human Condition by Braden R. Allenby and Daniel Sarewitz is a discussion of how to grapple with coming technological change and is particularly intriguing when it discusses “wicked complexity.”
Superintelligence by Nick Bostrom explores the many issues and implications related to the development of superintelligent machines.
The Works, The Heights, and The Way to Go by Kate Ascher examine how cities, skyscrapers, and our transportation networks, respectively, actually work. Beautifully rendered and fascinating books.
The Second Machine Age by Erik Brynjolfsson and Andrew McAfee examines the rapid technological change we are experiencing and can come to expect, and how it will affect our economy, as well as how to handle this change.
The Glass Cage by Nicholas Carr is about the perils of automation and the related technological complexity around us.
Shop Class as Soulcraft by Matthew B. Crawford explores the importance of getting close to our technologies again, as part of the virtue of manual labor.
Summa Technologiae by Stanisław Lem (translated by Joanna Zylinska) is a wide-ranging exploration from the 1960s by a science fiction writer of the future of technology, with an emphasis on the limits of humanity’s powers of understanding.
Think Twice by Michael Mauboussin looks at how to think properly—and often counterintuitively—about the complex systems that are all around us.
Overcomplicated Page 13