The Age of Surveillance Capitalism

by Shoshana Zuboff

  Speaking at Facebook’s 2017 developers’ conference, Zuckerberg linked his assertion of the company’s historic role in establishing a “global community” to the standard myth of the modern utopia, assuring his followers, “In the future, technology is going to… free us up to spend more time on the things we all care about, like enjoying and interacting with each other and expressing ourselves in new ways.… A lot more of us are gonna do what today is considered the arts, and that’s gonna form the basis of a lot of our communities.15

  As Nadella and other surveillance capitalists spin their utopian dreams, the surveillance capitalists fail to mention that the magical age they envision comes at a price: Big Other must expand toward totality as it deletes all boundaries and overwhelms every source of friction in the service of its economic imperatives. All power yearns toward totality, and only authority stands in the way: democratic institutions; laws; regulations; rights and obligations; private governance rules and contracts; the normal market constraints exercised by consumers, competitors, and employees; civil society; the political authority of the people; and the moral authority of individual human beings who have their bearings.

  This point was made in Goethe’s fable of the sorcerer’s apprentice, when, in the absence of the sorcerer’s authority to guide and check the action, the apprentice transforms the broom into a demonic force of pure unrelenting power:

  Ah, the word with which the master

  Makes the broom a broom once more!

  Ah, he runs and fetches faster!

  Be a broomstick as before!

  Ever new the torrents

  That by him are fed,

  Ah, a hundred currents

  Pour upon my head!16

  III. Applied Utopistics

  Instrumentarian power, like the apprentice’s broom, has flourished in the sorcerer’s absence with little authority to check its action, and the surveillance capitalists’ appetite for totality has grown with this success. The utopian rhetoric of a magical age has been critical to this progress. The notion that Big Other will solve all of humanity’s problems while empowering each individual is usually dismissed as mere “techno utopianism,” but it would be a mistake for us to ignore this rhetoric without examining its purpose. Such discourse is no mere hogwash. It is the minesweeper that precedes the foot soldiers and the canny diplomat sent ahead to disarm the enemy and smooth the way for a quiet surrender. The promise of a magical age plays a critical strategic role, simultaneously distracting us from and legitimating surveillance capitalism’s totalistic ambitions that necessarily include “people” as a “first class object.”

  The “societal goal” articulated by the leading surveillance capitalists fits snugly into the notion of limitless technological progress that dominated utopian thought from the late eighteenth century through the late nineteenth century, culminating with Marx. Indeed, surveillance capitalists such as Nadella, Page, and Zuckerberg conform to five of the six elements with which the great scholars of utopian thought, Frank and Fritzie Manuel, define the classic profile of the most ambitious modern utopianists: (1) a tendency toward highly focused tunnel vision that simplifies the utopian challenge, (2) an earlier and more trenchant grasp of a “new state of being” than other contemporaries, (3) the obsessive pursuit and defense of an idée fixe, (4) an unshakable belief in the inevitability of one’s ideas coming to fruition, and (5) the drive for total reformation at the level of the species and the entire world system.17

  The Manuels observe a sixth characteristic of the future-oriented modern visionary, and this is where the men and the corporations examined here represent powerful exceptions to the rule: “Often a utopian foresees the later evolution and consequences of technological development already present in an embryonic state; he may have antennae sensitive to the future. His gadgets, however, rarely go beyond the mechanical potentialities of his age. Try as he may to invent something wholly new, he cannot make a world out of nothing.”18 In our time, however, surveillance capitalists can and do make such a world—a genuinely historic deviation from the norm.

  Individually and collectively, the knowledge, power, and wealth that surveillance capitalists command would be the envy of any ancient potentate, just as they are now coveted by the modern state. With 2017 balance sheets reporting $126 billion in cash and securities for Microsoft, $92 billion for Google, and about $30 billion for Facebook, and the financial markets endorsing their ever-expanding instrumentarian regimes with more than $1.6 trillion in market capitalization in mid-2017, these are the rare utopianists who can oversee the translation of their imaginations into fact without soldiers to pave the way in blood.19

  In this respect, the surveillance capitalist leaders are sui generis utopianists. Marx grasped the world with his thickly articulated theory, but with only the power of his ideas, he could not implement his vision. Long after the publication of Marx’s theories, men such as Lenin, Stalin, and Mao applied them to real life. Indeed, the Manuels describe Lenin as a specialist in “applied utopistics.”20 In contrast, the surveillance capitalists seize the world in practice. Their theories are thin—at least this is true of the thinking that they share with the public. The opposite is true of their power, which is monumental and largely unimpeded.

  When it comes to theory and practice, the usual sequence is that theory is available to inspect, interrogate, and debate before action is initiated. This allows observers an opportunity to judge a theory’s worthiness for application, to consider unanticipated consequences of application, and to evaluate an application’s fidelity to the theory in which it originates. The unavoidable gap between theory and practice creates a space for critical inquiry. For example, we can question whether a law or governmental practice is consistent with a nation’s constitution, charter of rights, and governing principles because we can inspect, interpret, and debate those documents. If the gap is too great, citizens act to close the gap by challenging the law or practice.

  The surveillance capitalists reverse the normal sequence of theory and practice. Their practices move ahead at high velocity in the absence of an explicit and contestable theory. They specialize in displays of instrumentarianism’s unique brand of shock and awe, leaving onlookers dazed, uncertain, and helpless. The absence of a clear articulation of their theory leaves the rest of us to ponder its practical effects: the vehicular monitoring system that shuts down your engine; the destination that appears with the route; the suggested purchase that flashes on your phone the moment your endorphins peak; Big Other’s continuous tracking of your location, behavior, and mood; and its cheerful herding of city dwellers toward surveillance capitalism’s customers.

  However meager and secretive the surveillance capitalists’ theories might be, the instrumentarian power they wield can make their dreams come true, or at least inflict a whirlwind of consequences as they try. The only way to grasp the theory advanced in their applied utopistics is to reverse engineer their operations and scrutinize their meaning, as we have done throughout these chapters.

  Applied utopistics are on the move at Facebook, Google, and Microsoft as the frontier of behavioral surplus extraction moves into realms of life traditionally understood as societal and elaborated under some combination of civil institutions and public leadership. Zuckerberg’s 2017 mission statement for Facebook, introduced as “building global community,” announced a new phase of applied utopistics: “Overall, it is important that the governance of our community scales with the complexity and demands of its people. We are committed to always do better, even if that involves building a worldwide voting system to give you more voice and control. Our hope is that this model provides examples of how collective decision-making may work in other aspects of the global community.”21 Later that year, Zuckerberg told an audience of developers that “we have a full roadmap of products to help build groups and community, help build a more informed society, help keep our communities safe, and we have a lot more to do here.”22

  Back on that
stage in the spring of 2017, Microsoft’s Nadella encouraged his developers: “Whether it’s precision medicine or precision agriculture, whether it’s digital media or the industrial internet, the opportunity for us as developers to have broad, deep impact on all parts of society and all parts of the economy has never been greater.”23 The vision that Nadella unveiled that day is emblematic of the wider surveillance capitalist template for our futures. Where do they think they are taking us?

  IV. Confluence as Machine Relations

  In order to decipher the true measure of an instrumentarian society, let’s set aside the razzle-dazzle of a “magic age” and focus instead on the practices of applied utopistics and the social vision they imply. Nadella provided a valuable opportunity when he unveiled a series of practical applications that imply a sweeping new vision of machine relations as the template for a new era’s social relations.

  The reveal begins with Nadella’s account of a Microsoft collaboration with a 150-year-old Swedish manufacturer of high-precision metal-cutting equipment that has reinvented itself for the twenty-first century. The project is a state-of-the-art illustration of what Nadella describes as the “fundamental change in the paradigm of the apps that we are building, a change in the worldview that we have… from… a mobile-first, cloud-first world to a new world that is going to be made up of an intelligent cloud and an intelligent edge.” Artificial intelligence, he says, “learns from information and interacts with the physical world,” thus citing the capabilities required for economies of action.24

  Nadella first describes the machines linked by telemetry in the new factory setting as they continuously stream data to the “IoT hub” in the “cloud,” where Microsoft’s analyses search out anomalies that could put the machines at risk. Each anomaly is traced back through the data stream to its cause, and machine intelligence in the hub learns to identify the causal patterns so that it can preemptively shut down a threatened piece of equipment in about two seconds, before a potentially damaging event can occur.

  Then Nadella describes the new “breakthrough capability” in which a computational actuating sensor is embedded directly in the machine, dramatically reducing the time to a preemptive shutdown: “That logic is now running locally, so there’s no cloud loop.” The “edge” knows immediately when the machine experiences an event that predicts a future anomaly, and it shuts down the equipment within 100 milliseconds of this computation, a “20X improvement.” This is celebrated as “the power of the cloud working in harmony with an intelligent edge” to anticipate and preempt variations from the norm “before they happen.”25

  The power of machine learning develops exponentially as the devices learn from one another’s experiences, feeding into and drawing upon the intelligence of the hub. In this scenario it’s not that the whole is greater than the sum of the parts; it’s more like there are no parts. The whole is everywhere, fully manifest in each device embedded in each machine. Nadella translates these facts into their practical application, observing that once you have lots of devices around, an “ad hoc data center” is created “on a factory floor, at home, or anywhere else.… You can turn any place into a safe, AI-driven place.”26

  With this statement it finally becomes clear that “safe” means “automatically anomaly-free.” In Nadella’s factory, machine knowledge instantaneously replaces ignorance, herding all machine behavior to preestablished norms. Rather than concern for the multiplication of risk and the contagion of failure should machine learning go awry, Nadella celebrates the synchrony and universality of certain outcomes, as every machine is the same machine marching to the same song.

  Just as a century ago the logic of mass production and its top-down administration provided the template for the principles of industrial society and its wider civilizational milieu, so too is Nadella’s new-age factory revealed as the proving ground for his social vision—surveillance capitalism’s vision—of an instrumentarian society enabled by a new form of collective action. Machine learning is rendered here as a collective mind—a hive mind—in which each element learns and operates in concert with every other element, a model of collective action in which all the machines in a networked system move seamlessly toward confluence, all sharing the same understanding and operating in unison with maximum efficiency to achieve the same outcomes. Confluent action means that the “freedom” of each individual machine is surrendered to the knowledge that they share. Just as the behavioral theorists Planck, Meyer, and Skinner predicted, this sacrifice amounts to an all-out war on accidents, mistakes, and randomness in general.

  Nadella takes this template of new machine relations and applies it to a more complex illustration of a human and machine system, though still in the “economic domain.” This time it’s a construction site, where human and machine behaviors are tuned to preestablished parameters determined by superiors and referred to as “policies.” Algorithmic uncontracts apply rules and substitute for social functions such as supervision, negotiation, communication, and problem solving. Each person and piece of equipment takes a place among an equivalence of objects, each one “recognizable” to “the system” through the AI devices distributed across the site.

  For example, each individual’s training, credentials, employment history, and other background information are instantly on display to the system. A “policy” might declare that “only credentialed employees can use jackhammers.” If an employee who is not accredited for jackhammer use approaches that tool, the possibility of an impending violation is triggered, and the jackhammer screams an alert, instantly disabling itself.

  Significantly, it is not only the unified action of things on the site that are mobilized in alignment with policies. Confluent human action is also mobilized, as social influence processes are triggered in the preemptive work of anomaly avoidance. In the case of the at-risk jackhammer, the humans at the site are mobilized to swarm toward the location of the AI-anticipated anomalous offense in order to “quickly resolve it.” “The intelligent edge,” Microsoft developers are told, “is the interface between the computer and the real world… you can search the real world for people, objects and activities, and apply policies to them.…”27

  Once people and their relationships are rendered as otherized, equivalent “things in the cloud,” 25 billion computational actuating devices can be mobilized to shape behavior around safe and harmonious “policy” parameters. The most “profound shift,” Nadella explained, is that “people and their relationship with other people is now a first-class thing in the cloud. It’s not just people but it’s their relationships, it’s their relationships to all of the work artifacts, their schedules, their project plans, their documents; all of that now is manifest in this Microsoft Graph.” These streams of total information are key to optimizing “the future of productivity,” Nadella exulted.28

  In Microsoft’s instrumentarian society, the factories and workplaces are like Skinner’s labs, and the machines replace his pigeons and rats. These are the settings where the architecture and velocities of instrumentarian power are readied for translation to society in a digital-age iteration of Walden Two in which machine relations are the model for social relations. Nadella’s construction site exemplifies the grand confluence in which machines and humans are united as objects in the cloud, all instrumented and orchestrated in accordance with the “policies.” The magnificence of “policies” lies precisely in the fact that they appear on the scene as guaranteed outcomes to be automatically imposed, monitored, and maintained by the “system.” They are baked into Big Other’s operations, an infinity of uncontracts detached from any of the social processes associated with private or public governance: conflict and negotiation, promise and compromise, agreement and shared values, democratic contest, legitimation, and authority.

  The result is that “policies” are functionally equivalent to plans, as Big Other directs human and machine action. It ensures that doors will be locked or unlocked, car engines will shut down or come to
life, the jackhammer will scream “no” in suicidal self-sacrifice, the worker will adhere to norms, the group will swarm to defeat anomalies. We will all be safe as each organism hums in harmony with every other organism, less a society than a population that ebbs and flows in perfect frictionless confluence, shaped by the means of behavioral modification that elude our awareness and thus can neither be mourned nor resisted.

  Just as the division of labor migrated from the economic domain to society in the twentieth century, Nadella’s construction site is the economic petri dish in which a new division of learning mutates into life, ready for translation to society. In the twentieth century the critical success factors of industrial capitalism—efficiency, productivity, standardization, interchangeability, the minute division of labor, discipline, attention, scheduling, conformity, hierarchical administration, the separation of knowing and doing, and so forth—were discovered and crafted in the workplace and then transposed to society, where they were institutionalized in schools, hospitals, family life, and personality. As generations of scholars have documented, society became more factory-like so that we might train and socialize the youngest among us to fit the new requirements of a mass production order.

  We have entered this cycle anew, but now the aim is to remake twenty-first-century society as a “first-class thing” organized in the image of the machine hive for the sake of others’ certainty. The connectedness that we once sought for personal sustenance and effectiveness is recast as the medium for a new species of power and the social confluence that translates into guaranteed outcomes.

  V. Confluence as Society

  Microsoft scientists have been working for years on how to take the same logic of automated preemptive control at the network’s edge and transpose it to social relations. As Nadella observed in 2017, if “we” can do this in a “physical place,” it can be done “everywhere” and “anywhere.” He advised his audience of applied utopianists, “You could start reasoning about people, their relationship with other people, the things in the place.…”29