by Andrew Keen
What Jeremy Bentham called a “simple idea in Architecture” reflected his brother’s interest in disciplining the serfs on Potemkin’s Krichev estate. Borrowing from the Greek myth of Panoptes, a giant with a hundred eyes, the Panopticon—intended to house a large institution like a prison, a school, or a hospital—was a circular structure designed to house a single watchman to observe everyone in the building. This threat of being watched, Jeremy Bentham believed, represented “a new mode of obtaining power of mind over mind.” The Panopticon was a “vividly imaginative” fusion of architectural form with social purpose,” the architectural historian Robin Evans explains. And this purpose was discipline. The more we imagined we were being watched, Jeremy and Samuel Bentham imagined, the harder we would work and the fewer rules we would break. Michel Foucault thus described the Panopticon as a “cruel, ingenious cage.” It was “a microcosm of Benthamite society,” according to one historian, and “an existential realization of Philosophical Radicalism,” according to another.39
As the founder of Philosophical Radicalism, a philosophical school better known today as utilitarianism, Jeremy Bentham saw human beings as calculating machines driven by measurable pleasure and pain. Society could be best managed, Bentham believed, by aggregating all these pleasures and pains in order to determine the greatest collective happiness. In the words of the British legal philosopher H. L. A. Hart, Bentham was a “cost-benefit expert on the grand scale.”40 And the nineteenth-century Scottish thinker Thomas Carlyle criticized Bentham as a philosopher focused on “counting up and estimating men’s motives.” Half a century before his compatriot Charles Babbage invented the first programmable computer, Bentham was already thinking about human beings as calculating machines. And the Panopticon—which he spent much of his life futilely trying to build—is a “simple idea in Architecture” that enables everything and everyone to be watched and measured.
The establishment of a Bentham-style electronic panopticon, fused with his utilitarian faith in the quantification of society, is what is so terrifying about twenty-first-century networked society. We are drifting into a Benthamite world in which everything—from our fitness to what we eat to our driving habits to how long and how hard we work—can be profitably quantified by companies like Google’s smart home device manufacturer Nest, which is already building a lucrative business managing the electricity consumption of consumers on behalf of energy utilities.41 And with its Gross National Happiness Index and its secret experiments to control our moods, Facebook is even resurrecting Bentham’s attempt to quantify our pleasure and pain.
In an electronic panopticon of 50 billion intelligent devices, a networked world where privacy has become a privilege of the wealthy, it won’t just be our televisions, our smartphones, or our cars that will be watching us. This is John Lanchester’s “new kind of human society,” a place where everything we do and every place we go can be watched and turned into personal data—a commodity that EU consumer commissioner Meglena Kuneva describe as the “new oil of the Internet and the new currency of the digital world.”42
“Is the Internet now just one big human experiment?”43 asked the Guardian’s Dan Gillmor in response to July 2014 revelations about emotion-manipulation experiments conducted by Facebook and by the dating site OkCupid. In the future, I’m afraid, the answer to Gillmor’s question will be yes. As the sociologist Zeynep Tufekci warns about this infinitely creepy networked world, big data companies like Facebook and OkCupid “now have new tools and stealth methods to quietly model our personality, our vulnerabilities, identify our networks, and effectively nudge and shape our ideas, desires and dreams.”44
Such nudging and shaping—particularly for dating—isn’t necessarily new, argues the Financial Times’ Christopher Caldwell. But in the pre-Internet past, he notes, this has been done by outside authorities—particularly parents, communities, and religious bodies. “The difference,” Caldwell notes, between OkCupid’s experiment and parent and religious groups, “is that these groups actually loved the young people they were counselling, had a stake in ensuring things did not go wrong, would help as best they could if things did, and were not using the young lovers strictly as a means of making money.”45
We will be observed by every unloving institution of the new digital surveillance state—from Silicon Valley’s big data companies and the government to insurance companies, health-care providers, the police, and ruthlessly Benthamite employers like Jeff Bezos’s Amazon, with its scientifically managed fulfillment centers where the company watches over its nonunionized workforce. Big data companies will know what we did yesterday, today, and, with the help of increasingly accurate predictive technology, what we will do tomorrow. And—as in what Christopher Caldwell calls OkCupid’s “venal” experiment—the goal of these big data companies will be strictly to make money from our personal data rather than use it as a public service.
Our Crystal Future
Imaginary dystopias about a future dominated by monstrously powerful technology companies tend to be presented as updated versions of the Orwellian totalitarian state. One example is Ridley Scott’s Prometheus, a 2012 movie about a future in which a tech company called the Weyland Corporation has become so powerful that its CEO is able to boast, “We are the gods now.” The assumption is these companies will replace the government. That they will become Big Brother.
Such a dramatic scenario works well in movies, but is an overly Manichaean take on the future. In our libertarian age of hostility to the state, Google doesn’t really need to actually become the government to give it more power over us. So Eric Schmidt, when asked if Google wanted to operate like a government, said that his company didn’t want to have the responsibilities of being a country. “We’re not becoming a state,” he explained. “We don’t want to be because states have a lot of complicated problems.”46 But, of course, Google doesn’t need to become an old-fashioned state—with all those “complicated problems” of tax collection and welfare and educational policy—to increase its power and wealth. Google can, instead, partner with the government to create a more efficient and profitable surveillance society.
“Who should we fear more with our data: the government or companies?” asks Guardian columnist Ana Marie Cox.47 Unfortunately, however, it’s not an either/or question. In today’s networked world, we should fear both the government and private big data companies like Facebook and Google.
We got a preview of this terrifying new world in the summer of 2013 with the National Security Agency data-mining Prism scandal revealed by the former NSA analyst Edward Snowden. “If Big Brother came back, he’d be a public-private partnership,” explained the British historian Timothy Garton Ash. And it’s exactly this kind of partnership between big data companies like Google and the NSA—both the government and private companies—that we should most fear.
According to a June 2013 report in the New York Times, the Prism program “grew out of the National Security Agency’s desire several years ago to begin addressing the agency’s need to keep up with the explosive growth of social media.”48 Prism showed the backdoor access to the data of their customers that Microsoft, Yahoo, Google, Facebook, PalTalk, AOL, Skype, YouTube, and Apple all gave—or were legally required to give, according to these companies—to the government. As the Internet historian John Naughton notes, Prism uncovered the “hidden wiring of our networked world”49 and revealed the fact “that Google, Facebook, Yahoo, Amazon, Apple and Microsoft are all integral components of the US cyber-surveillance system.”50
The Prism scandal reveals what the New York Times’ James Risen and Nick Wingfield call the “complex reality” of data mining as “an industry and a crucial intelligence tool” that “binds N.S.A and Silicon Valley leaders.”51 The Atlantic’s Michael Hirsh argues that “the government’s massive data collection and surveillance system was largely built not by professional spies or Washington bureaucrats but by Silicon Valley and private defense contractors.”52 As the New York Times�
�� Claire Cain Miller adds, some Internet companies, notably Twitter, “declined to make it easy for the government” to collect personal data. But most were compliant and “many cooperated at least a bit”53 with the government. Google, for example, complied with government requests during the second half of 2012 for information 88% of the time.54
Unfortunately, the NSA Prism scandal is not the only example of data collusion between Internet companies and the US government. One of the creepiest online data companies is Acxiom, an information broker that, according to the technology writer Sue Halpern, has “profiles of 75% of all Americans, each around five thousand data points that can be constructed and deconstructed” to find supposedly suspicious people. “It should come as no surprise,” Halpern says, “that the NSA and the Departments of Defense and Homeland Security buy this material from Acxiom.”55
Competing with Acxiom as the Internet’s creepiest public-private partnership is Peter Thiel’s 2004 data intelligence startup Palantir, “the go-to company mining massive data sets for intelligence and law enforcement applications.”56 Funded in part by a $2 million investment from the CIA’s venture arm Q-Tel, with the CIA being its only client between 2005 and 2008, Palantir now boasts a client list that includes the FBI, CIA, Army, Marines, Air Force, and Defense Department and was valued at $9 billion when the private company raised $107.5 million in 2013. According to Mark Bowden, the author of a popular book about the killing of Osama bin Laden, Palantir “actually deserves the popular designation Killer App.”57 One Special Forces member based in Afghanistan who has extensively used Palantir compares its intelligence to a God-like force. “It’s like plugging into the Matrix,” he told Bloomberg’s Ashlee Vance and Brad Stone. “The first time I saw it, I was like, ‘Holy crap. Holy crap. Holy crap.’”58
Since the Snowden leak, Internet companies have scrambled to distance themselves from the NSA and the US government or any association with creepy data collectors like Acxiom. There have also been calls from both Google and Twitter for more encryption of Web traffic59 and the publication of an open letter to Congress and Barack Obama for the American government to “take the lead” and put an end to digital surveillance.60 But as a December 2013 editorial in Bloomberg argued, the effort of Silicon Valley’s retrospective critique of the NSA is “richly hypocritical” because “collecting, packaging and selling personal information, often without users’ full knowledge and sometimes without their informed consent, is generally what these companies do for a living.”61
The Bloomberg editorial is right. “The primary business model of the Internet is built on mass surveillance,” notes Bruce Schneier, a leading computer security expert, “and our government’s intelligence-gathering agencies have become addicted to this data.”62
So rather than an aberration, Silicon Valley’s involvement with the NSA’s Prism surveillance program conforms with the Internet’s core identity. Data, as the EU’s Meglena Kuneva reminds us, is the new oil of the digital economy. So whether it’s Google’s attempt to embed tiny cameras in networked contact lenses63 or the networked home with its detailed knowledge of our comings and goings,64 or smart cities that track everything from our driving to our shopping habits,65 surveillance remains the Internet’s main business model.
“Cities are our paradises of anonymity, a place for self-erasure and self-invention,” the veteran technology reporter Quentin Hardy reminds us.66 So what becomes of self-erasure and self-invention in today’s digital panopticon, with products like the aptly named Panono ball camera that films everything it sees? What is the fate of privacy in an Internet of Everything and Everyone?
Today’s “simple idea of Architecture,” as Jeremy Bentham put it, is an electronic network in which everything we do is recorded and remembered. Bentham’s eighteenth-century Panopticon has been upgraded to a twenty-first-century instrument of mass surveillance. Like Vannevar Bush’s Memex, its trails never fade; like Ted Nelson’s hypertext, there is no “concept of deletion”; like Erich Mielke’s Stasi, its appetite for our personal data is insatiable. The Internet has, indeed, become a crystal republic for crystal man.
We shape our architecture; and thereafter it shapes us.
CHAPTER EIGHT
EPIC FAIL
FailCon
Big Brother might be dead, but one department of the old totalitarian state remains in robust health. Orwell’s Ministry of Truth—in fact, of course, the Ministry of Propaganda—was supposed to have gone out of business in 1989 with the fall of the Berlin Wall. But, like other failed twentieth-century institutions, the ministry has relocated its operations to the west coast of America. It has moved to the epicenter of twenty-first-century innovation—to Silicon Valley, a place so radically disruptive that it is even reinventing failure as the new model of success.
On the list of all-time greatest lies, the idea that FAILURE IS SUCCESS doesn’t quite match the Orwellian trinity of WAR IS PEACE, FREEDOM IS SLAVERY, or IGNORANCE IS STRENGTH, but it’s still an astonishing perfidy, worthy of the best Ministry of Truth propagandist. And yet, in Silicon Valley the “failure is success” lie has become such an accepted truth that there is now even a San Francisco event called FailCon, dedicated to its dissemination.
Along with several hundred other aspiring disruptors, I’d gone to FailCon to learn why, in the Valley at least, failure is considered to be desirable. Held at San Francisco’s luxury Kabuki hotel a couple of miles west of the Battery, FailCon was part countercultural remix of the old Protestant work ethic, part classic Californian self-help therapy, and—like most technology events in Silicon Valley—wholly divorced from reality. It’s as if Orwell’s Ministry of Truth had, to borrow another fashionable Valley word, “pivoted” into the conference business. “Stop being afraid of failure and start embracing it,”1 the event instructed its audience. And to help us overcome the fear, to make it feel good to fail, FailCon invited some of Big Tech’s greatest innovators to outfail each other with tales of their losses.
At FailCon, the F-word was ubiquitous among illustrious Silicon Valley speakers like Airbnb cofounder Joe Gebbia, the billionaire venture capitalist Vinod Khosla, and Eric Ries, the author of a bestselling handbook for Internet success called The Lean Startup. Indeed, the more uncannily prescient the investor, the more moneyed the startup entrepreneur, the bigger the influencer, the more boastfully they broadcasted their litany of failures. At FailCon, we heard about failure as the most valuable kind of education, failure as a necessity of innovation, failure as a version of enlightenment, and, most ironically, given the event’s self-congratulatory tenor, failure as a lesson in humility.
But the award for the most successful and least humble of FailCon failures went to Travis Kalanick, the cofounder and CEO of the transportation network Uber, whose prematurely graying hair and hyperkinetic manner suggested a life of perpetual radical disruption. Both his appearance and his business “innovations” personified Schumpeter’s “perennial hurricane of creative destruction.” This self-styled “badass,” a pinup for our libertarian age who identifies himself as one of the violent criminals in Quentin Tarantino’s movie Pulp Fiction,2 certainly isn’t too shy to present himself as a historic risk taker. On Twitter, @travisk even once borrowed the cover of The Fountainhead, Ayn Rand’s extreme libertarian celebration of free-market capitalism, as his profile photo.3
Kalanick’s $18 billion venture is certainly a badass company, with customers accusing its drivers of every imaginable crime from kidnapping4 to sexual harassment.5 Since its creation, the unregulated Uber has not only been in a constant legal fight with New York City, San Francisco, Chicago, and federal regulators, but has been picketed by its own nonunionized drivers demanding collective bargaining rights and health-care benefits.6 Things aren’t any better overseas. In France, opposition to the networked transportation startup has been so intense that, in early 2014, there were driver strikes and even a series of violent attacks on Uber cars in Paris.7 While in September 2014, a Frankfurt court bann
ed Uber’s budget price UberPop product entirely from the German market, claiming that the massively financed startup unfairly competed with local taxi companies.8
Uber drivers don’t seem to like Kalanick’s anti-union company any more than regulators do. In August 2013, Uber drivers sued the company for failing to remit tips and in September 2014 around a thousand Uber drivers in New York City organized a strike against the company’s unfair working conditions. “There’s no union. There’s no community of drivers,” one sixty-five-year-old driver who has been working for Uber for two years complained to the New York Times in 2014. “And the only people getting rich are the investors and executives.”9
The fabulously wealthy Silicon Valley investors, who will ride the startup till its inevitable IPO, love Uber, of course. “Uber is software [that] eats taxis. . . . It’s a killer experience,” you’ll remember Marc Andreessen enthused.10 Tragically, that’s all too true. On New Year’s Eve 2013, an Uber driver accidentally ran over and killed a six-year-old girl on the streets of San Francisco. Uber immediately deactivated what they call their “partner’s” account, saying that he “was not providing service on the Uber system during the time of the accident.”11
How generous. And happy 2014 to all our partners, Uber might have added.
So much for shared responsibility in the sharing economy. No wonder Kalanick’s own drivers, whom he calls “transportation entrepreneurs,” are picketing Uber. And no wonder that the parents of Sofia Liu, the San Francisco girl killed by the Uber driver, are suing Uber itself in a wrongful-death lawsuit.
It’s not just drivers and pedestrians who are being killed by Uber. If you don’t like it, walk, Uber tells its customers, with Kalanickian tact, about a service that uses “surge” pricing—a euphemism for price gouging—which has resulted in fares being 700–800% above normal on holidays or in bad weather.12 During a particularly ferocious December 2013 snowstorm in New York City, one unfortunate Uber rider paid $94 for a trip of less than two miles that took just eleven minutes.13 Even the rich and famous are being outrageously ripped off by the unregulated Uber service, with Jessica Seinfeld, Jerry’s wife, being charged $415 during that same December storm to take her kid across Manhattan.14