The Age of Surveillance Capitalism

by Shoshana Zuboff

  Though pledging reform to the public, the corporation was simultaneously forced to adapt to governmental demands in a range of countries—including Australia, Belgium, Canada, France, Holland, Hong Kong, Ireland, Israel, Italy, New Zealand, Poland, Spain, South Korea, the UK, and the US—where Street View was subjected to litigation, fines, and/or regulation. In Japan, homeowners complained of Street View cameras that peered above privacy fencing to record private homes. Google agreed to government demands to lower its cameras, reshoot all images, and blur identifiable facial images and license plates. In Germany, Google allowed residents to request that their homes be blurred in any Street View images. Nearly 250,000 households made opt-out requests in 2009–2010, requiring Google to hire 200 temporary programmers to meet the demand.52 Google was fined 145,000 euros by the Hamburg data-protection supervisor who had first discovered the Street View illicit data gathering, just short of the 150,000 euro fee he could have imposed.53 It was the largest fine ever to have been levied by European regulators for privacy concerns. The discount reflected Google’s assurances that it would swiftly and thoroughly delete the payload data. In 2011 Google ended its Street View program in Germany, continuing to support but no longer update the images it had already collected.54

  Other countries imposed bans on Street View operations. Switzerland initially banned the service in 2009, insisting that Google remove all imagery it had posted of Swiss towns and cities. Eventually, the ban was lifted, but the Swiss Federal Administrative Court imposed a series of strict guidelines, including blurring faces, instituting opt-out measures, and reducing camera heights. By 2016, Google’s service remained confined to outdoor tourist sites.55 The corporation also faced Street View bans in Austria, the Czech Republic, Greece, India, and Lithuania. By the summer of 2017, though, Street View data were available from at least some regions of each of these countries.56

  Stage Four: Redirection

  What Google did not say in its mea culpa blog post, the one thing that it could not say, was that it would abandon its fundamental logic of accumulation: the principles of surveillance capitalism that had brought the behemoth into existence and sustained its growth. The message of Street View’s redirection campaign was that Google would not exempt anything from the grid. Everything must be corralled for conversion into raw material. Short of institutional suicide, there is little that Google can say or do to ensure “user privacy.” This helps to explain why, as one 2015 article celebrating the history of Google Maps observes, “Google Maps was attracting all sorts of privacy controversies… people were freaked out.… But that doesn’t mean Street View has been tamped down as a project. It’s now available in 65 of Google Maps’ 200-some countries.”57

  Alma Whitten’s job was to repair Google’s privacy reputation, not to dismantle the extraction imperative and its relentless demand for economies of scale in the supply function. This is to say that her job was a logical impossibility. That she may have nevertheless taken it seriously is suggested by the fact that just two-and-a-half years after her appointment as privacy czar, she announced her retirement from Google in April 2013. Indeed, it is painful to watch Whitten testify about Google’s practices to an early-2013 congressional hearing. She was under questioning from Congress, and one sees the effort required as she hunts for the words to convey an answer without conveying the truth.58 The time had come to regroup and redirect the global mapping project, not to end it.

  That nothing much had changed or would change was immediately suggested by the fate of Google’s mystery engineer in the two years that followed the scandal. Within days of the FCC report in April 2012, a former state investigator who had been assigned to the Street View inquiry identified Google’s “rogue” actor as Marius Milner, a celebrated hacker and wardriving specialist. It had been two years since the supposedly irreparable damage that he inflicted on Google and his “clear violation” of policy, yet he continued to be employed at the firm in its YouTube operations. Later that year, he would be one of six inventors on a team led by John Hanke to patent “A System and Method for Transporting Virtual Objects in a Parallel Reality Game.”59

  The invention in which Milner participated was related to a virtual reality game called Ingress, also developed by Hanke and his team at Google. (Hanke would eventually establish his own shop, Niantic Labs, within Google’s new Alphabet holding company.) Ingress became a test bed for many of the foundational concepts that reappeared in another “game,” Pokémon Go, a prototype of a second phase of surveillance capitalist expansion that we examine closely in Part II. In this next phase, Google’s maps are a critical resource for the expansion of digital dispossession from the virtual world to the one that we call “real.” In light of those plans, Street View could not be allowed to die or even to be constrained. The corporation’s senior product manager for Google Maps framed it succinctly in September 2012, just four months after the FCC investigation: “If you look at the offline world, the real world in which we live, that information is not entirely online. Increasingly as we go about our lives, we are trying to bridge that gap between what we see in the real world and [the online world], and Maps really plays that part.”60

  Google’s closely guarded “Ground Truth” project, initiated in 2008 but only publicly revealed just four months after the FCC report in 2012, exemplifies the point. Ground Truth is the “deep map” that contains the detailed “logic of places”: walking paths, goldfish ponds, highway on-ramps, traffic conditions, ferry lines, parks, campuses, neighborhoods, buildings, and more.61 Getting these details right is a source of competitive advantage in the contest for behavioral surplus accrued from mobile devices. The construction of the deep map draws on public resources such as geographic databases from the US Census Bureau and the US Geological Survey,62 but what distinguishes Google’s maps from all others is the integration of its exclusive proprietary data from Street View. In other words, data compiled through public investments are augmented with data taken from a unilateral transfer of surplus behavior and decision rights. The composite results are then reclassified as private assets.

  One of the first journalists invited to see demonstrations of Ground Truth in 2012, Alexis Madrigal, observed that “the Maps team, largely driven by Street View, is publishing more imagery data every two weeks than Google possessed in total in 2006.… Google is up to five million miles driven now.” Street View cars are likened to Google Search’s early web crawlers, which quietly commandeered web pages for indexing and access in the corporation’s original act of dispossession. By 2012, Street View data also provided street signs and addresses. Soon, Madrigal wrote, “any word that is visible from a road will become a part of Google’s index of the physical world” thanks to Street View. Madrigal’s look at the Ground Truth operation concludes, “The geographic data Google has assembled is not likely to be matched by any other company.… They’ve built this whole playground as an elaborate lure for you.”63

  As one project leader put it, “The challenge of deciding you’re going to map the world is that you can’t ever stop.”64 So it was that by 2016, Google’s Street View website celebrated its successful evolution by stating, “We’ve come a long way since our initial U.S. launch in 2007; today we’ve expanded our 360-degree panoramic views to include locations on all seven continents.” Street View’s fleet of surveillance-gathering tools had been augmented to include a wearable backpack, a three-wheeled pedicab, a snowmobile, and a trolley, all of which were designed to capture places that Street View cars could not traverse. Tourist boards and nonprofits were offered the use of the company’s Trekker equipment (the backpack camera) to “collect views of remote and unique places” that were, literally and figuratively, “off the grid.”65

  What Google couldn’t build, it bought. In 2013 the corporation won a reported bidding war with Facebook for Israeli social mapping startup Waze, a firm that pioneered community-sourced real-time traffic information. In 2014 it acquired real-time satellite imaging startup Skybox ju
st as the US Department of Commerce lifted restrictions on high-resolution satellite imagery. As an expert explained,

  If you imagine a satellite above your office then the old resolution could probably make out your desk. The new imagery—where each pixel measures around 31 cm—can now make out what’s on your desk. When you reach this sort of frequency you can begin to add in what we call “pattern of life” analysis. This means looking at activity in terms of movement—not just identification.66

  In this context one appreciates the significance of another aspect of Google’s redirection campaign: a 2011 announcement that the corporation had breached “a new frontier” with the introduction of an “indoor positioning system” that enabled it to locate and follow people “when you’re inside an airport, shopping mall, or retail store.” Eventually, these new capabilities would include sensors and embedded cameras that let users map and navigate interior spaces.67 In a September 2014 blog, Google Maps’ dynamic new capabilities were showcased to the public as your new “co-pilot for deciding everything from turn-by-turn directions, to discovering new restaurants, to deciding which hiking trails to climb.” The post credits Street View with providing these wondrous new capabilities and announces the expansion of the whole incursion with the introduction of a mobile mapping tool dubbed “Cartographer,” worn as a backpack and able to map the interior of buildings.68 Cartographer’s information could be added to the growing navigational database of interior spaces, amplifying Google’s ability to locate people and devices as they moved between outdoor and indoor spaces.

  Building interiors had eluded Street View and the extraction imperative; few homeowners were likely to invite those cameras indoors. Instead, Cartographer’s capabilities were bundled into the larger Street View redirection campaign and pitched to businesses as a way to enhance consumer trust, allay anxiety, and substantially increase revenues. Google exhorted consumer-facing businesses to “invite customers inside.” With its “Business View,” consumers would be able to see inside thousands of hotels, restaurants, and other destinations. Search listings would feature the new Street View content. Hotel listings would offer a virtual tour of the properties. “Give them the confidence they’re seeking,” Google told its business marketplace, by allowing consumers “to experience your location before they arrive.” Google asserted that virtual tours “double bookings,” and it instated a certification program that enabled businesses to hire a Google-approved freelance photographer to produce images for Street View. These extraordinary new redirection tactics aimed to flip the old pattern. They reframed Street View from an edgy incursion circumventing resistance through stealth to an opulent VIP tent where businesses scrambled for an entry pass.

  Street View’s redirection and elaboration announced a critical shift in the orientation and ambition of the surveillance program: it would no longer be only about routes, but about routing. We will examine this new episode of dispossession in the chapters that follow. For now, suffice to say that Street View and the larger project of Google Maps illustrate the new and even more ambitious goals toward which this cycle of dispossession would soon point: the migration from an online data source to a real-world monitor to an advisor to an active shepherd—from knowledge to influence to control. Ultimately, Street View’s elaborate data would become the basis for another complex of spectacular Google incursions: the self-driving car and “Google City,” which we learn more about in Chapter 7. Those programs aim to take surplus capture to new levels while opening up substantial new frontiers for the establishment of behavioral futures markets in the real world of goods and services. It is important to understand that each level of innovation builds on the one before and that all are united in one aim, the extraction of behavioral surplus at scale.

  In this progression, Google perceives an opportunity that it hopes its customers will come to appreciate: its ability to influence actual behavior as it occurs in the real spaces of everyday life. In 2016, for example, the corporation introduced a new Maps app feature called “Driving Mode” that suggests destinations and travel times without users even selecting where they want to go. If you searched for a hammer online, then “Driving Mode” can send you to a hardware store when you buckle up your seat belt. “Google is integrating this ‘push’ technology into its main mobile search app,” reported the Wall Street Journal.69

  With this app, Google the “copilot” prompts an individual to turn left and right on a path defined by its continuously accruing knowledge of the person and the context. Predictions about where and why a person might spend money are derived from Google’s exclusive access to behavior surplus and its equally exclusive analytic capabilities: “Eat here.” “Buy this.” Google’s surplus analysis can predict that you are likely to buy an expensive woolen suit, and its real-time location data can trigger the proprietor or advertiser’s real-time prompt, matched to your profile and delivered at the very moment that you are within sight of the flannels, tweeds, and cashmeres. Push and pull, suggest, nudge, cajole, shame, seduce: Google wants to be your copilot for life itself. Each human response to each commercial prompt yields more data to refine into better prediction products. The prompts themselves are bought and paid for in a novel iteration of Google’s online ad markets: real-time, real-world trading in behavioral futures. Your future.

  The stakes are high in this market frontier, where unpredictable behavior is the equivalent of lost revenue. Google cannot leave anything to chance.70 In September 2016 the tech newsletter the Register revealed that the Google Play app preinstalled in the latest Android phone continuously checks a user’s location, sending that information to your third-party apps as well as to Google’s own servers. One security researcher was shocked when his Android phone prompted him to download the McDonald’s app at the very moment that he crossed the threshold of the fast-food restaurant. He later discovered that Google Play had monitored his location thousands of times. Similarly, Google Maps “doesn’t give you a decent option of turning it off.” If you do, the operating system warns, “basic features of your device may no longer function as intended.”71 Google’s insistence reflects the authoritarian politics of the extraction imperative as well as the corporation’s own enslavement to the implacable demands of its economics.

  The historic point for us to consider here is that the once-spurned Street View found new life in its contribution to the decisive expansion of behavioral futures markets both online and in the real world. Once dedicated to targeted online advertising, these markets now grow to encompass predictions about what human beings will do now, soon, and later, whether they make their way online, on sidewalks and roads, or through rooms, halls, shops, lobbies, and corridors. These ambitious goals foreshadow fresh incursions and dispossessions as resistance is neutralized and populations fall into dulled submission.

  Google discovered by chance or intention the source of every mapmaker’s power. The great historian of cartography, John B. Harley, said it succinctly: “Maps created empire.” They are essential for the effective “pacification, civilization, and exploitation” of territories imagined or claimed but not yet seized in practice. Places and people must be known in order to be controlled. “The very lines on the map,” wrote Harley, were a language of conquest in which “the invaders parcel the continent among themselves in designs reflective of their own complex rivalries and relative power.” The first US rectangular land survey captured this language perfectly in its slogan: “Order upon the Land.”72 The cartographer is the instrument of power as the author of that order, reducing reality to only two conditions: the map and oblivion. The cartographer’s truth crystallizes the message that Google and all surveillance capitalists must impress upon all humans: if you are not on our map, you do not exist.

  IV. The Dogs of Audacity

  Projects such as Street View taught Google that it could assume the role of arbiter of the future and get away with it. It learned to sustain even the most-contested dispossession efforts when they are necessar
y to secure vital new supply lines. For example, while Street View protests erupted around the world and just months before Germany’s announcement that Street View was secretly capturing personal information from unprotected Wi-Fi networks, Google introduced Buzz—a platform intended to float Google’s nets in the path of the coveted behavioral surplus that streamed from social networks. The invasive practices introduced with Buzz—it commandeered users’ private information to establish their social networks by fiat—set off a fresh round of the dispossession cycle and its dramatic contests.

  As Google learned to successfully redirect supply routes, evading and nullifying opposition, it became even more emboldened to let slip the dogs of audacity and direct them toward havoc. Among many examples, Google Glass neatly illustrates the tenacity of the extraction imperative and its translation into commercial practice. Google Glass combined computation, communication, photography, GPS tracking, data retrieval, and audio and video recording capabilities in a wearable format patterned on eyeglasses. The data it gathered—location, audio, video, photos, and other personal information—moved from the device to Google’s servers, merging with other supply routes to join the titanic one-way flow of behavioral surplus.

  The project was seen as a precursor to more flexible and less overt forms of wearable computation and surplus capture. John Hanke described its familiar shape in the form of eyewear as suitable for “the early adoption phases” of wearable technology in much the same way that the first automobiles resembled horse-drawn buggies. In other words, the “glasses” were intended to disguise what was in fact unprecedented: “Ultimately we will want these technologies, wherever they are on your body, to be totally optimized based on the job they’re doing, not on what is more socially acceptable at that first moment of creation, just because it reminds people of something they’ve seen in the past.”73 Introduced with great flair in the spring of 2012 as fashion-forward futurism, it wasn’t long before the public registered fresh horror at this bizarre invasion. Those who wore the device were recast as “glassholes,” and some businesses banned the glasses from their premises.74