To this end, Paradiso’s students invented a “ListenTree,” which emits streaming sound that “invites attention” and “points to a future where digital information might become a seamless part of the physical world.” He and his colleagues populated a 250-acre marsh with hundreds of sensors that measure and record temperature, humidity, moisture, light motion, wind, sound, tree sap flow, chemical levels, and more. They developed “inertial sensors” that track and compute complex movements and “flexible sensate fibers” to create “radically new functional substrates that can impact medicine, fashion, and apparel… and bring electronics into all things stretchable or malleable.” There are electronics that attach directly to skin in the form of tattoos and makeup, while fingernails and wrists are transformed into computational interfaces that can read finger gestures, even in the absence of hand movements. “Sensor tape” and “stickers” can adhere “to inaccessible surfaces and building materials,” where they can be “wirelessly interrogated.…”14
Paradiso and his colleagues wrestled with the paradox of, on the one hand, proliferating sensor data in nearly every environment—from smartphones to home devices to streets to cameras to cars—and, on the other hand, the difficulties involved in integrating sensor-generated data flows and producing meaningful analyses. Their answer was “DoppelLab,” a digital platform for combining and visually representing sensor data.15 The idea is to transform any physical space, from the interior of an office building to an entire city, into a “browse-able environment” where you can see and hear everything going on in that space as it flows from thousands or billions or trillions of sensors. Just as browsers like Netscape first “gave us access to the mass of data contained on the internet, so will software browsers enable us to make sense of the flood of sensor data that is on the way.”16
The aim here is a grand synthesis: the collation and fusion of every sort of sensor data from every channel and device to develop a “virtual sensor environment” in which “crawlers will constantly traverse data… calculating state and estimating other parameters derived from the data” collected from everywhere from office interiors to entire cities.
Paradiso is confident that “a proper interface to this artificial sensoria promises to produce… a digital omniscience… a pervasive everywhere augmented reality environment… that can be intuitively browsed” just as web browsers opened up the data contained on the internet. He insists that ubiquitous sensor information and computing will be “an extension of ourselves rather than an embodiment of an ‘other.’” Information will stream “directly into our eyes and ears once we enter the age of wearables… the boundaries of the individual will be very blurry in this future.”17
According to Paradiso and his coauthor, Gershon Dublon, the next great technological challenge is “context aggregation,” which means the ability to assemble rapidly expanding sensor information into new “applications.” The idea is that every physical space and every trace of behavior within that space—bees buzzing, your smile, the temperature fluctuations in my closet, their breakfast conversation, the swoosh of the trees—will be “informated” (translated into information). Spaces can be aggregated into a seamless flow of searchable information, sights, and sounds in much the same way that Google once aggregated web pages for indexing and searching: “This shift will create a seamless nervous system that covers the planet—and one of the main challenges for the computing community now is how to merge the rapidly evolving ‘omniscient’ electronic sensoria onto human perception.”18
For all their brilliance, these creative scientists appear to be unaware of the restless economic order eager to commandeer their achievements under the flag of surveillance revenues. Paradiso does not reckon with the translation of his paradise of omniscience into the realpolitik of surveillance capitalism as the prediction imperative insists on surplus culled from these new flows and surveillance capitalists fill the front seats of the classroom of digital omniscience.
IV. Surveillance Capitalism’s Realpolitik
Waning levels of government leadership and funding for “ubiquitous computing” leave the technology companies to lead in basic research and applications, each vying to become “the Google” of the new apparatus and its architectures of extraction and execution.19 Despite the radical prospects of the ubiquitous connected sensate computational apparatus and the often-repeated claim “It will change everything,” technology firms in the US have, thus far, continued their run of relative lawlessness, unimpeded by any comprehensive social or regulatory vision. As Intel’s chief strategist for the “internet of things” commented in response to concerns over privacy implications, “One thing that we absolutely believe is that though we hear the conversation around policy, we don’t want policy to get in the way of technological innovation.…”20
In place of “policy” or a “social contract,” it is capitalism, and increasingly surveillance capitalism, that shapes the action. New behavioral futures markets and “targeted applications” are eagerly awaited. As Microsoft’s director of the machine intelligence platform for integrating and analyzing data from the “internet of things” says, “The part that’s equally cool and creepy is what happens after everybody and their competitor gets on board with smart devices: a big secondary market for data… a secondary revenue source.” These markets, he explains, are “just like Google or Facebook’s” markets for targeted advertising.”21 An IBM report concurs: “Thanks to the internet of things, physical assets are turning into participants in real-time global digital markets. The countless types of assets around us will become as easily indexed, searched and traded as any online commodity.… We call this the ‘liquification of the physical world.’”22
In an ominous parallel to the rhetoric of “data exhaust” as the prelude to dispossession, this second phase of expropriation also requires new rhetoric that can simultaneously legitimate and distract from the real action unleashed by the prediction imperative. A new euphemism, “dark data,” plays this role. For example, Harriet Green directed IBM’s $3 billion investment in the “internet of things,” a resource commitment that aimed to make the company a serious contender to become “the Google” of ubiquitous computing. Green says that digital omniscience is impeded by the fact that most of the data collected by companies are “unstructured,” making them difficult to “datafy” and code.23 IBM’s customers are plagued by the question “What can we do with this [unstructured] data to make us more efficient or to create new products and services that we can sell to optimize what we’re doing or create new things for clients?”24
Unstructured data cannot merge and flow in the new circuits of liquefied assets bought and sold. They are friction. Green fixes on the declarative term that simultaneously names the problem and justifies its solution: dark data. The message we saw honed in the online world—“If you’re not in the system, you don’t exist”—is refined for this new phase of dispossession. Because the apparatus of connected things is intended to be everything, any behavior of human or thing absent from this push for universal inclusion is dark: menacing, untamed, rebellious, rogue, out of control. The stubborn expanse of dark data is framed as the enemy of IBM’s and its customers’ ambitions. Note the echoes of MacKay here, with his determination to penetrate the secrets of unrestrained animals and inaccessible regions. The tension is that no thing counts until it is rendered as behavior, translated into electronic data flows, and channeled into the light as observable data. Everything must be illuminated for counting and herding.
In this way the notion of “dark data” handily becomes the “data exhaust” of ubiquitous computing. It provides the moral, technical, commercial, and legal rationale for powerful systems of machine intelligence that can capture and analyze behaviors and conditions never intended for a public life. For those who seek surveillance revenues, dark data represent lucrative and necessary territories in the dynamic universal jigsaw constituted by surveillance capitalism’s urge toward scale, scope, and action. T
hus, the technology community casts dark data as the intolerable “unknown unknown” that threatens the financial promise of the “internet of things.”25
It is therefore understandable that Green portrays machine intelligence—and specifically IBM’s anthropomorphized artificial intelligence system called “Watson”—as the authoritative savior of an apparatus threatened by waste and incomprehensibility. Machine intelligence is referred to as “cognitive computing” at IBM, presumably to avoid the uneasy connotations of inscrutable power associated with words like machine and artificial.
Under the leadership of CEO Ginni Rometty, the corporation invested heavily in “Watson,” heralded by the company as “the brains of the ‘internet of things.’” Rometty wants IBM to dominate the machine learning functions that will translate ubiquitous data into ubiquitous knowledge and action. “The first discussion is around how much dark data you have that only Watson and cognitive can really interrogate,” Green says. “You know the amount of data being created on a daily basis—much of which will go to waste unless it is utilized. This so-called dark data represents a phenomenal opportunity… the ability to use sensors for everything in the world to basically be a computer, whether it’s your contact lens, your hospital bed, or a railway track.”26 The message is that surveillance capitalism’s new instruments will render the entire world’s actions and conditions as behavioral flows. Each rendered bit is liberated from its life in the social, no longer inconveniently encumbered by moral reasoning, politics, social norms, rights, values, relationships, feelings, contexts, and situations. In the flatness of this flow, data are data, and behavior is behavior. The body is simply a set of coordinates in time and space where sensation and action are translated as data. All things animate and inanimate share the same existential status in this blended confection, each reborn as an objective and measurable, indexable, browsable, searchable “it.”
From the vantage point of surveillance capitalism and its economic imperatives, world, self, and body are reduced to the permanent status of objects as they disappear into the bloodstream of a titanic new conception of markets. His washing machine, her car’s accelerator, and your intestinal flora are collapsed into a single dimension of equivalency as information assets that can be disaggregated, reconstituted, indexed, browsed, manipulated, analyzed, reaggregated, predicted, productized, bought, and sold: anywhere, anytime.
The worldview elaborated by scientists such as Paradiso and business leaders such as Green has been swept into action on many fronts where digital omniscience is eagerly welcomed as the recipe for certainty in the service of certain profits. The next section is an opportunity to see this worldview in action, in a business sector far from the pioneers of surveillance capitalism: automobile insurance. Extraction and prediction become the hallmarks of a new logic of accumulation as insurers and their consultants plot their approach to surveillance revenues. In the plans and practices of these new actors, we witness both the determination to institutionalize economies of scope and action and the drift toward a dark new world in which the automatic and closely targeted means of behavioral modification are understood as the path to profit.
V. Certainty for Profit
In Chapter 3 we met Google’s Hal Varian, and now once again he lights the way, exposing the significance and specific requirements of the prediction imperative. Recall that Varian identified four new “uses” of the computer mediation of transactions.27 The first of these was “data extraction and analysis,” from which we deduced the extraction imperative as one of the foundational mechanisms of surveillance capitalism. Varian says that the other three new uses—“new contractual forms due to better monitoring,” “personalization and customization,” and “continuous experiments”—“will, in time, become even more important than the first.”28 That time has come.
“Because transactions are now computer-mediated we can observe behavior that was previously unobservable and write contracts on it,” Varian says. “This enables transactions that were simply not feasible before.” He gravitates to the example of “vehicular monitoring systems,” recognizing their paradigmatic power. Varian says that if someone stops making monthly car payments, “Nowadays it’s a lot easier just to instruct the vehicular monitoring system not to allow the car to be started and to signal the location where it can be picked up.”29 Insurance companies, he notes, can also rely on these monitoring systems to check if customers are driving safely and thus determine whether to maintain the insurance policy, vary the cost of premiums, and decide whether to pay a claim.
Varian’s new uses of computer mediation in this insurance realm are entirely dependent upon internet-enabled devices that know and do. In fact, they are impossible to imagine without the material means of extraction and execution architectures planted in and permeating the real world. The vehicular monitoring system that he prescribes, for example, provides economies of scope and action. It knows and intervenes in the state of play, monitoring data and acting on programmed instructions to shut off the car’s engine, thus allowing the repo man to locate the disabled automobile and its vanquished driver.
As the prediction imperative pulls supply operations into the real world, product or service providers in established sectors far from Silicon Valley are enthralled by the prospects of surveillance revenues. For example, the CEO of Allstate Insurance wants to be like Google: “There are lots of people who are monetizing data today. You get on Google, and it seems like it’s free. It’s not free. You’re giving them information; they sell your information. Could we, should we, sell this information we get from people driving around to various people and capture some additional profit source…? It’s a long-term game.”30 Automobile insurers appear to be especially eager to implement Varian’s vision and MacKay’s telematics. The fight for your car, it turns out, is an illustration of the intensity of purpose with which companies great and small now pursue behavioral surplus.
Auto insurers have long known that risk is highly correlated with driver behavior and personality, but there was little that they could do about it.31 Now the remote sensate monitoring systems of modern telematics can provide a continuous stream of data about where we are, where we’re going, the details of our driving behavior, and the conditions of our vehicle. App-based telematics can also calculate how we are feeling and what we are saying by integrating dashboard and even smartphone information.
Auto insurers are besieged by consultants and would-be technology partners who proffer surveillance capitalistic strategies that promise a new chapter of commercial success. “Uncertainty will be strongly reduced,” intones a McKinsey report on the future of the insurance industry. “This leads to demutualization and a focus on predicting and managing individual risks rather than communities.”32 A report by Deloitte’s Center for Financial Services counsels “risk minimization”—a euphemism for guaranteed outcomes—through monitoring and enforcing policyholder behavior in real time, an approach called “behavioral underwriting.” “Insurers can monitor policyholder behavior directly,” Deloitte advises, by “recording the times, locations, and road conditions when they drive, whether they rapidly accelerate or drive at high or even excessive speeds, how hard they brake, as well as how rapidly they make turns and whether they use their turn signals.”33 Telematics produce continuous data flows, so real-time behavioral surplus can replace the traditional “proxy factors,” such as demographic information, that had previously been used to calculate risk. This means that surplus must be both plentiful (economies of scale) and varied (economies of scope) in both range and depth.
Even smaller underwriters that cannot afford extensive capital outlays for telematics are advised that they can accomplish most of these aims with a smartphone application, eliminating costly hardware and data-transmission expenses: “These insurers may also benefit because a mobile app gathers first-hand data on the behavior and performance of the driver carrying the smartphone… yielding a 360-degree view of the total exposure being underwrit
ten.…”34
As certainty replaces uncertainty, premiums that once reflected the necessary unknowns of everyday life can now rise and fall from millisecond to millisecond, informed by the precise knowledge of how fast you drive to work after an unexpectedly hectic early morning caring for a sick child or if you perform wheelies in the parking lot behind the supermarket. “We know that 16-year old drivers have a whole lot of accidents… but not every 16-year old is a lousy driver,” observes one insurance industry telematics expert. Rates based on actual behavior are “a big advantage in being able to price appropriately.”35 This kind of certainty means that insurance contracts designed to mitigate risk now give way to machine processes that respond “almost immediately” to nuanced infractions of prescribed behavioral parameters and thus substantially decrease risk or eliminate it entirely.36
Telematics are not intended merely to know but also to do (economies of action). They are hammers; they are muscular; they enforce. Behavioral underwriting promises to reduce risk through machine processes designed to modify behavior in the direction of maximum profitability. Behavioral surplus is used to trigger punishments, such as real-time rate hikes, financial penalties, curfews, and engine lockdowns, or rewards, such as rate discounts, coupons, and gold stars to redeem for future benefits. The consultancy firm AT Kearney anticipates “IoT enriched relationships” to connect “more holistically” with customers “to influence their behaviors.”37
Varian’s blithe statement that “it’s a lot easier” to instruct a vehicular monitoring system to shut off a car when a payment is late is not hyperbole. For example, Spireon, which describes itself as “the largest aftermarket vehicle telematics company” and specializes in tracking and monitoring vehicles and drivers for a variety of customers such as lenders, insurers, and fleet owners, offers a system akin to Varian’s ideal.38 Its “LoanPlus Collateral Management System” pushes alerts to drivers when they have fallen behind in their payments, remotely disables the vehicle when delinquency exceeds a predetermined period, and locates the vehicle for the repo man to recover.
The Age of Surveillance Capitalism Page 26