The stage was set for the discovery of your Facebook profile’s easy pillage in 2010, when a collaboration of German and US scholars came to the unexpected conclusion that Facebook profiles are not idealized self-portraits, as many had assumed. Instead, they found that the information on Facebook reflects the user’s actual personality, as independently assessed by the well-validated protocols of the five-factor personality model and as compared to study participants’ own descriptions of their “ideal self.”49
There is compelling evidence to suggest that the unique dynamics of the Facebook milieu eventually complicated this picture of “real personality,” as we shall explore in Chapter 16, but in 2011 these early findings encouraged three University of Maryland researchers to take the next logical step. They developed a method that relies on sophisticated analytics and machine intelligence to accurately predict a user’s personality from publicly available information in his or her Facebook profile.50
In the course of this research, the team came to appreciate the magic of behavioral surplus, discovering, for example, that a person’s disclosure of specific personal information such as religion or political affiliation contributes less to a robust personality analysis than the fact that the individual is willing to share such information in the first place. This insight alerted the team to a new genre of powerful behavioral metrics. Instead of analyzing the content of user lists, such as favorite TV shows, activities, and music, they learned that simple “meta-data”—such as the amount of information shared—“turned out to be much more useful and predictive than the original raw data.” The computations produced on the strength of these behavioral metrics, when combined with automated linguistic analysis and internal Facebook statistics, led the research team to conclude that “we can predict a user’s score for a personality trait to within just more than one-tenth of its actual value.”51 The University of Maryland team began what would become a multiyear journey toward the instrumentalization of data from the depths for the purposes of a highly intentional program of manipulation and behavioral modification. Although they could not see very far down that road, they nevertheless anticipated the utility of their findings for an eager audience of surveillance capitalists:
With the ability to infer a user’s personality, social media websites, e-commerce retailers, and even ad servers can be tailored to reflect the user’s personality traits and present information such that users will be most receptive to it.… The presentation of Facebook ads could be adjusted based on the personality of the user.… Product reviews from authors with personality traits similar to the user could be highlighted to increase trust and perceived usefulness.… 52
The new capabilities also proved robust when applied to other sources of social media meta-data. Later that year, the Maryland team published findings that used publicly available Twitter data to predict scores on each of the five personality dimensions to within 11–18 percent of their actual value. Similar research findings would become central to the progress of rendering Facebook profiles as behavior for new caches of surplus from the depths.53
In the UK a team of researchers, including Cambridge University’s Michal Kosinski and the deputy director of Cambridge’s Psychometrics Centre, David Stillwell, built on this line of research.54 Stillwell had already developed the myPersonality database, a “third-party” Facebook application that allows users to take psychometric tests, like those based on the five-factor model, and receive feedback on their results. Launched in 2007 and hosted at the Psychometrics Centre, by 2016 the database contained more than six million personality profiles complemented by four million individual Facebook profiles. Once regarded as a unique, if offbeat, source of psychological data, myPersonality had become the database of choice for the scoping, standardization, and validation of the new models capable of predicting personality values from ever-smaller samples of Facebook data and meta-data. Later, it would become the model for the work of a small consultancy called Cambridge Analytica, which used the new caches of behavioral surplus for an onslaught of politically inspired behavioral micro-targeting.
In a 2012 paper Kosinski and Stillwell concluded that “user personality can be easily and effectively predicted from public data” and warned that social media users are dangerously unaware of the vulnerabilities that follow their innocent but voluminous personal disclosures. Their discussion specifically cited Facebook CEO Mark Zuckerberg’s unilateral upending of established privacy norms in 2010, when he famously announced that Facebook users no longer have an expectation of privacy. Zuckerberg had described the corporation’s decision to unilaterally release users’ personal information, declaring, “We decided that these would be the social norms now, and we just went for it.”55
Despite their misgivings, the authors went on to suggest the relevance of their findings for “marketing,” “user interface design,” and recommender systems.56 In 2013 another provocative study by Kosinski, Stillwell, and Microsoft’s Thore Graepel revealed that Facebook “likes” could “automatically and accurately estimate a wide range of personal attributes that people would typically assume to be private,” including sexual orientation, ethnicity, religious and political views, personality traits, intelligence, happiness, use of addictive substances, parental separation, age, and gender.57
The authors appeared increasingly ambivalent about the social implications of their work. On the one hand, they announced that these new predictive capabilities could be used to “improve numerous products and services.” They concluded that online businesses can adjust their behavior to match each user’s personality, with marketing and product recommendations psychologically tailored to each individual. But the authors also warned that automated prediction engines run by companies, governments, or Facebook itself can compute millions of profiles without individual consent or awareness, discovering facts “that an individual may not have intended to share.” The researchers cautioned that “one can imagine situations in which such predictions, even if incorrect, could pose a threat to an individual’s well-being, freedom, or even life.”58
Despite these ethical quandaries, by 2015 Kosinski had moved to Stanford University (first to the Computer Science Department and then to the Graduate School of Business), where his research quickly attracted funding from the likes of Microsoft, Boeing, Google, the National Science Foundation, and the Defense Advanced Research Projects Agency (DARPA).59 Kosinksi and a variety of collaborators, often including Stillwell, went on to publish a succession of articles that elaborated and extended the capabilities demonstrated in the early papers, refining procedures that “quickly and cheaply assess large groups of participants with minimal burden.”60
A paper published in 2015 broke fresh ground again by announcing that the accuracy of the team’s computer predictions had equaled or outpaced that of human judges, both in the use of Facebook “likes” to assess personality traits based on the five-factor model and to predict “life outcomes” such as “life satisfaction,” “substance use,” or “depression.”61 The study made clear that the real breakthrough of the Facebook prediction research was the achievement of economies in the exploitation of these most-intimate behavioral depths with “automated, accurate, and cheap personality assessment tools” that effectively target a new class of “objects” once known as your “personality.”62 That these economies can be achieved outside the awareness of unrestrained animals makes them even more appealing; as one research team emphasizes, “The traditional method for personality evaluation is extremely costly in terms of time and labor, and it cannot acquire customer personality information without their awareness.…”63
Personality analysis for commercial advantage is built on behavioral surplus—the so-called meta-data or mid-level metrics—honed and tested by researchers and destined to foil anyone who thinks that she is in control of the “amount” of personal information that she reveals in social media. In the name of, for example, affordable car insurance, we must be coded as conscientious, agr
eeable, and open. This is not easily faked because the surplus retrieved for analysis is necessarily opaque to us. We are not scrutinized for substance but for form. The price you are offered does not derive from what you write about but how you write it. It is not what is in your sentences but in their length and complexity, not what you list but that you list, not the picture but the choice of filter and degree of saturation, not what you disclose but how you share or fail to, not where you make plans to see your friends but how you do so: a casual “later” or a precise time and place? Exclamation marks and adverb choices operate as revelatory and potentially damaging signals of your self.
That the “personality” insights themselves are banal should not distract us from the fact that the volume and depth of the new surplus supplies enabled by these extraction operations are unprecedented; nothing like this has ever been conceivable.64 As Kosinski told an interviewer in 2015, few people understand that companies such as “Facebook, Snapchat, Microsoft, Google and others have access to data that scientists would not ever be able to collect.”65 Data scientists have successfully predicted traits on the five-factor personality model with surplus culled from Twitter profile pictures (color, composition, image type, demographic information, facial presentation, and expression…), selfies (color, photographic styles, visual texture…), and Instagram photos (hue, brightness, saturation…). Others have tested alternative algorithmic models and personality constructs. Another research team demonstrated the ability to predict “satisfaction with life” from Facebook messages.66 This new world has no manager on his or her hands and knees scissoring pages of computer conferencing messages into thematically organized piles of paper scraps. It’s not the office floor that is crawled. It is you.
In his 2015 interview, Kosinski observed that “all of our interactions are being mediated through digital products and services which basically means that everything is being recorded.” He even characterized his own work as “pretty creepy”: “I actually want to stress that I think that many of the things that… one can do should certainly not be done by corporations or governments without users’ consent.” Recognizing the woefully asymmetric division of learning, he lamented the refusals of Facebook and the other internet firms to share their data with the “general public,” concluding that “it’s not because they’re evil, but because the general public is bloody stupid… as a society we lost the ability to convince large companies that have enormous budgets and enormous access to data to share this goodness with us.… We should basically grow up finally and stop it.”67
In capitalism, though, latent demand summons suppliers and supplies. Surveillance capitalism is no different. The prediction imperative unleashes the surveillance hounds to stalk behavior from the depths, and well-intentioned researchers unwittingly oblige, leaving a trail of cheap, push-button raw meat for surveillance capitalists to hunt and devour. It did not take long. By early 2015, IBM announced that its Watson Personality Service was open for business.68 The corporation’s machine intelligence tools are even more complex and invasive than those used in most academic studies. In addition to the five-factor personality model, IBM assesses each individual across twelve categories of “needs,” including “Excitement, Harmony, Curiosity, Ideal, Closeness, Self-expression, Liberty, Love, Practicality, Stability, Challenge, and Structure.” It then identifies “values,” defined as “motivating factors which influence a person’s decision-making across five dimensions: Self-transcendence/Helping others, Conservation/Tradition, Hedonism/Taking pleasure in life, Self-enhancement/Achieving success, and Open to change/Excitement.”69
IBM promises “limitless” applications of its new surplus supplies and “deeper portraits of individual customers.” As we would expect, these operations are tested among captive employees who, once habituated, can become docile members of a behaviorally purified society. “Personality correlates” can now be identified that predict the precise ways in which each customer will react to marketing efforts. Who will redeem a coupon? Who will purchase which product? The corporation says that “social media content and behavior” can be used to “capitalize on targeted revenue-generating opportunities” with “mapping rules from personality to behavior.” The messaging and approach of customer service agents, insurance agents, travel agents, real estate agents, investment brokers, and so on can be “matched” to the “personality” of the customer, with those psychological data displayed to the agent at the precise moment of contact.70 IBM’s research demonstrates that agents who express personality traits associated with “agreeableness” and “conscientiousness” produce significantly higher levels of customer satisfaction. It is common sense, except that now these interactions are measured and monitored in real time and at scale, with a view to rewarding or extinguishing behavior according to its market effect.71
Thanks to rendition, a handful of now-measurable personal characteristics, including the “need for love,” predict the likelihood of “liking a brand.”72 In a Twitter targeted-ad experiment, IBM found that it could significantly increase click-through rates and “follow” rates by targeting individuals with high “openness” and low “neuroticism” scores on the five-factor personality analysis. In another study, IBM rendered behavioral data from 2,000 Twitter users to establish metrics such as response rates, activity levels, and elapsed time between tweets, in addition to psycholinguistic analyses of tweets and five-factor personality analysis. IBM “trained” its predictive model by asking the 2,000 users either location-related or product-related questions. The findings showed that personality information predicted the likelihood of responses. People whom the machines rated as moral, trusting, friendly, extroverted, and agreeable tended to respond, compared to low response rates from people rated as cautious and anxious. Many of the characteristics that we try to teach our children and model in our own behavior are simply repurposed as dispossession opportunities for hidden machine processes of rendition. In this new world, paranoia and anxiety function as sources of protection from machine invasion for profit. Must we teach our children to be anxious and suspicious?
IBM is not alone, of course. An innovative breed of personality mercenaries quickly set to work institutionalizing the new supply operations. Their efforts suggest how quickly we lose our bearings as institutionalization first establishes a sense of normalcy and social acceptance and then gradually produces the numbness that accompanies habituation. This process begins with business plans and marketing messages, new products and services, and journalistic representations that appear to accept the new facts as given.73
Among this new cohort of mercenaries was Cambridge Analytica, the UK consulting firm owned by the reclusive billionaire and Donald Trump backer Robert Mercer. The firm’s CEO, Alexander Nix, boasted of its application of personality-based “micro-behavioral targeting” in support of the “Leave” and the Trump campaigns during the ramp-up to the 2016 Brexit vote and the US presidential election.74 Nix claimed to have data resolved “to an individual level where we have somewhere close to four or five thousand data points on every adult in the United States.”75 While scholars and journalists tried to determine the truth of these assertions and the role that these techniques might have played in both 2016 election upsets, the firm’s new chief revenue officer quietly announced the firm’s less glamorous but more lucrative postelection strategy: “After this election, it’ll be full-tilt into the commercial business.” Writing in a magazine for car dealers just after the US election, he tells them that his new analytic methods reveal “how a customer wants to be sold to, what their personality type is, and which methods of persuasion are most effective.… What it does is change people’s behavior through carefully crafted messaging that resonates with them.… It only takes small improvements in conversion rates for a dealership to see a dramatic shift in revenue.”76
A leaked Facebook document acquired in 2018 by the Intercept illustrates the significance of data drawn from the depths in the fabrication of Facebook’s predi
ction products, confirms the company’s primary orientation to its behavioral futures markets, and reveals the degree to which Cambridge Analytica’s controversial practices reflected standard operating procedures at Facebook.77 The confidential document cites Facebook’s unparalleled “machine learning expertise” aimed at meeting its customers’ “core business challenges.” To this end it describes Facebook’s ability to use its unrivaled and highly intimate data stores “to predict future behavior,” targeting individuals on the basis of how they will behave, purchase, and think: now, soon, and later. The document links prediction, intervention, and modification. For example, a Facebook service called “loyalty prediction” is touted for its ability to analyze behavioral surplus in order to predict individuals who are “at risk” of shifting their brand allegiance. The idea is that these predictions can trigger advertisers to intervene promptly, targeting aggressive messages to stabilize loyalty and thus achieve guaranteed outcomes by altering the course of the future.
Facebook’s “prediction engine” is built on a new machine intelligence platform called “FBLearner Flow,” which the company describes as its new “AI backbone” and the key to “personalized experiences” that deliver “the most relevant content.” The machine learning system “ingests trillions of data points every day, trains thousands of models—either offline or in real time—and then deploys them to the server fleet for live predictions.” The company explains that “since its inception, more than a million models have been trained, and our prediction service has grown to make more than 6 million predictions per second.”78
As we have already seen, “personalization” derives from prediction, and prediction derives from ever richer sources of behavioral surplus and therefore ever more ruthless rendition operations. Indeed, the confidential document cites some of the key raw materials fed into this high-velocity, high-volume, and deeply scoped manufacturing operation, including not only location, Wi-Fi network details, and device information but also data from videos, analyses of affinities, details of friendships, and similarities with friends.
The Age of Surveillance Capitalism Page 34