Dataclysm: Who We Are (When We Think No One's Looking)

by Christian Rudder

  We’ve had a look so far at the ways two people come together in the first blush of attraction. I’m not sure a computer will ever capture their path to full togetherness, but we do have a picture of their lives once they get there. That pattern of a couple together, the enmeshing of what’s come to be called their “social graphs,” is now well documented.

  I have 384 friends on Facebook, and here they are. I’m the dot in the middle; my wife, Reshma, is in black at about three o’clock. Everyone’s connections to everyone else are shown by the gray lines:

  Though the groups of my friends are nicely clustered, this plot wasn’t arranged by hand—my able research assistant, James Dowdell, wrote special software to create it. The dots come together based on their number of shared connections. Think of them as little bits of iron dust magnetized by the POWER OF FRIENDSHIP, and then dropped on a tabletop to settle into place. Even though I don’t use Facebook for much of anything besides the highly circular task of accepting Facebook friend requests, you can see all the sides of my life in there. My very tight-knit set of in-laws, as near to overlapping as the software would allow, is A; the people I went to high school with are B; my coworkers are C; my gamer friends, D. You can even read my once and future career as a musician in the graph. I spent years touring in a band, and those singleton dots all along the left perimeter are primarily people I met on the road. Their bond to one another is our music, invisible to algorithms.

  Let me expand the graph to include Reshma’s connections as well, to show the scope of our network as a couple. The connections we share, our mutual friends, are in dark red.

  Though this might seem like a dry abstraction of a couple’s life together, a mutual plot like this tells you a tremendous amount about the bond between the two people it’s built around. From just the plot, the image alone, we can calculate that Reshma and I are much less likely than other couples to break up.

  Network analysis, the study of dots and lines just like the patterns above, has been a science for almost three hundred years, and you can see something of the rise of data (from trickle to cataclysm) in its progress. The first network problem was a kind of rustic brainteaser, really an Enlightenment-era urban legend, that it was impossible to walk through the Prussian city of Königsberg by crossing each of its seven bridges once and only once. In 1735, Leonhard Euler, as geniuses will do, came along and reduced what had been a colloquial question of neighborhoods and footpaths to an abstraction of dots and lines (formally: nodes and edges), and in doing so, he proved with rigor that the legend was true. He expressed the town as a network, and a discipline was founded.

  Euler’s insight was that because you’re only supposed to cross each bridge once, to enter a new neighborhood you need a pair of bridges—one to get you in, another to get you out. So the solution is as simple as looking at the network plot and asking whether each point along your path, other than your beginning and end, has an even number of lines (a pair of bridges) attached. In Königsberg, none of them do, so the problem was solved. That from such homely origins can come an enduring and flourishing science, one that’s only now finding its full expression, is, I think, the best possible case for the human spirit.1 Euler’s concept of nodes and edges, which at first unraveled nothing more than a day’s walk, has since helped us understand disease and its vectors, trucks and their routes, genes and their bindings, and of course, people and their relationships. And in just the last few decades, network theory’s application to these last have exploded—because the networks themselves have exploded.

  Forty years ago, Stanley Milgram was mailing out parcels (kits with instructions and postage-paid envelopes) to a hundred people in Omaha, working on his “six degrees of separation,” hoping maybe a few dozen adventuresome souls would participate. His quaint methods—ingenious though they were—would give him the famous theory, but not quite its proof. In 2011, the unprecedented and overwhelming scale of Facebook allowed us to see that he was indeed right: 99.6 percent of the 721 million accounts at the time were connected by six steps or fewer.

  Today, network theory, working on data sets enabled by technology, shows how people can find new jobs, sort information from nonsense, and even make better movies. When they built their headquarters, Pixar famously put the only bathrooms in the building inside the central atrium to force interdepartmental small talk, knowing that innovation often comes from the serendipitous collision of ideas. Theirs was an application of “the strength of weak ties,” a concept postulated in the 1970s with samples in the dozens, but since amplified on new, robust network data: it tells us that it’s the people you don’t know very well in your life who help ideas, especially new ones, spread.2

  Another long-held idea in network theory is “embeddedness.” One of its expressions is the amount of overlap in a pair of social graphs—Reshma’s and my embeddedness is simply how large the red portion of our graph is compared with the whole. Research using a variety of sources (e-mail, IM, telephone) has shown that the more mutual friends two people share, the stronger their relationship. More connections imply more time together, more common interests, and more stability. But unlike, say, telephone records, or even e-mail, online social networks attach rich data to a graph’s edges and nodes (not unlike how dating sites have taken the timeless ritual of courtship and added age and beauty as variables to study) and of course Facebook is the richest such network ever created. The effects of that richness are just being felt.

  Social-graph analysis began as, and largely remains, a matter of “who knows who.” The scope of Facebook data—you can go many degrees deep with practically no added effort—is starting to turn that on its head. For relationships, and romantic relationships specifically, this data has recently enabled a new, powerful measure of how strong a bond between two people is. It turns out your lives should not just be intertwined but intertwined in a specific way. And, rare among network analysis metrics, who doesn’t know who is the important quantity.

  Two scientists, Lars Backstrom and Jon Kleinberg, working through 1.3 million couples from Facebook, established the idea in a 2013 paper. Their measure was based on counting the number of times a person and her spouse functioned as the bridge between disjointed parts of their network as a couple. Here’s what I mean: the graph on the left below is a hunky-dory scene, more or less everybody knows one another; it is very highly embedded. But the stronger marriage is on the right. There, the couple, A and B, are the sole connectors for the two cliques around them:

  This probably feels a little strange—why would you want your network to be more fractious but for you and your spouse? But like the best ideas, it plays out intuitively in real life. For example, going back to my own story, Reshma’s cousin Sheel is highly embedded in her life. The two of them grew up together, and he, like she does, has connections to virtually every member of their large extended family, including many people I don’t even know. They’ve known each other their entire lives, whereas Reshma and I have been married for only seven years. Sheel and Reshma’s relationship as a central pair would function much like my left-hand example above. However, Sheel doesn’t know Reshma’s coworkers. He doesn’t know the members of Reshma’s dance troupe. He doesn’t know Reshma’s friends from college. I know them all, and what’s more, I am the only other person in her life these three distinct groups have in common, at least directly. For these groups, we embody the ideal on the right. It’s worth noting that if, for example, Reshma and I worked together, or she didn’t dance, or we went to the same college, we could not play the role we do in each other’s networks.

  Backstrom and Kleinberg call the level to which a relationship fulfills this ideal its “dispersion” because it shows how disconnected your graph would be without you—that is, how utterly your social circle would fly to the winds if you and your spouse were somehow ripped from the center (by, say, having a second child). I prefer “assimilation” because I think that better captures the upshot: assimilated people have a unique role as
a couple within their mutual network. Highly assimilated couples function—the two people together—as the bond between many otherwise unconnected cliques. They are the special glue in a given spread of dots, and furthermore, they’re a glue like epoxy: it takes both ingredients to make the thing hold together.

  The power of assimilation comes from the fact that your spouse is one of the few people (if not the only person) you introduce into the far corners of your life. She is there at work parties, there at reunions, and there when your gamer friends come over for that all-day Magic: the Gathering blowout you look forward to all year. (Or she’s not there, if she can help it, but you get the idea.) Meanwhile, these coworkers, these classmates, and these gamers, though all densely intraconnected groups themselves, are unrelated to one another but for you and your spouse.

  And here’s why it matters: For married people on Facebook, their spouse is the most assimilated member of their network an astounding 75 percent of the time. And, even more important for assimilation as a metric of relationship strength, the young couples for whom that’s not the case are 50 percent more likely to break up. In the most stable relationships, the two people play this unique role in each other’s lives. Considering alternate graphs of a nonassimilated couple, it makes a certain sense why—in an overly embedded one, like the left-hand example before, you and your spouse end up competing with everyone else for time and attention. There’s too much leveling, no specialness. Too many girls’ nights. Or in a cliquey network without assimilation, “leading separate lives” can very quickly become “leading secret lives,” which might look something like this:

  Against assimilation, Backstrom and Kleinberg tested many other ways to evaluate a relationship, and there was one detail in their paper, presented almost as an aside, that I found particularly wry. Early on, the best predictor of a relationship doesn’t depend on the couple’s social graph at all; for the first year or so of dating, the optimal method is how often they view each other’s profile. Only over time, as the page views go down and their mutual network fills out, does assimilation come to dominate the calculus. In other words, the curiosity, discovery, and (visual) stimulation of falling for someone is eventually replaced by the graph-theory equivalent of nesting.

  There’s this idea in computer science that you should be your own customer—that you should at least have enough confidence in the website or software you’re foisting on the world to use it yourself. Just like Jonas Salk injecting himself with his brand-new polio vaccine, you want to prove what you’re doing is good. Programmers call it dogfooding, as in “Eat your own dog food,” because as a group they make bad decisions at mealtimes. At some companies, dogfooding is mandatory. Have a meeting with Microsoft people, and they’ll roll up with their Windows phones and Surface tablets, dutiful hounds chewing tough bits of tendon.

  You and I don’t have those kinds of orders from on high here, of course. But I purposefully led this chapter with my own data because, first, I needed to work the abstract concepts upon a clear example. But also I wanted to show that, in a book that picks apart so many other people’s highly personal data, I’m willing to apply the same analysis to myself.

  I offer you the same opportunity. To let you test your own marriage, partnership, or unhealthy codependent friendship against the principles discussed in this chapter, I have implemented the Backstrom/Kleinberg algorithm at:

  dataclysm.org/relationshiptest

  Give it a pair of Facebook credentials, and it will not only depict your mutual graph and your embeddedness but also rank your most assimilated relationships. The world has now arrived at a place where we can do something with our own data—we don’t have to wait for a Milgram, let alone an Euler, to teach us about ourselves. In the same way a service like Facebook or Twitter exposes our data to academic scrutiny, it reflects it back at us, for scrutiny of our own. Weak tools to capture and analyze our own physical activity are already here, and better ones are not long off. When you see people in middle management dickering with their Fitbits in the elevator, you know the Quantified Self movement is here to stay. The above is my very small attempt to add to the possibilities.

  If you use my app with someone else, here’s hoping you’re at the top of each other’s lists, and remember: a little creative defriending can give your assimilation score the necessary boost. Because self-measurement is all well and good until some ex-girlfriend comes in ahead of your wife.

  1 Evidence against: of the seven bridges so famous in Euler’s time, four have since been destroyed. Two by bombs and two by a superhighway.

  2 The original paper has been cited more than 20,000 times.

  5.

  There’s No Success Like Failure

  There’s a great Tumblr called “Clients from Hell,” where anyone can submit their service-industry horror stories. There are all kinds of cluelessness and oblivion on display, and new posts go up every few hours. Here’s a typical submission, from someone doing a photo spread:

  CLIENT: Can we have a heading on the photo as well?

  DESIGNER: Well, it already has a caption.

  CLIENT: If the reader misses the caption, then they will still see the heading.

  DESIGNER: It would be quite unusual to have both a heading and a caption on a photo.

  CLIENT: That makes sense. Just put a heading next to the caption, then.

  My favorite client quote on the site right now is: “I don’t like the dinosaur in this graphic. It looks too fake. Use a real photo of a dinosaur instead.” The blog mostly gets submissions from graphic designers, but Clients from Hell’s popularity speaks to a universal truth. People hate their customers.

  I don’t mean hate on an individual level but, en masse, customers, like any rabble, are to be feared. Anyone who tells you otherwise, from the cupcake-shop owner down the street to the CEO in the boardroom, is lying. Part of it is the “… is always right” thing—nobody likes a person with that much power. But by far the biggest cause of frustration is that people don’t understand and can’t articulate what they actually need. As Steve Jobs said, “People don’t know what they want until you show it to them.” What he didn’t say is that showing them, especially in tech, means playing a game of Pin the Tail on the Donkey with several million people shouting advice.

  If you are, say, a car company and people don’t like some part of your product, they mostly tell you indirectly, by not buying it. There’s historically been no open channel between Ford and the folks who want the cup holders to be green or who think it would be better if the steering wheel were a square, because, you know, most turns are 90 degrees. That’s why traditional companies spend so much on market research—they have to stay way ahead of these kinds of things, because by the time a company like Ford would naturally hear about a problem, via Accounts Receivable, it’s way too late.

  A website is different: if people have a cockamamie idea, someone at the company is just an e-mail away. And if people don’t use something, the site notices immediately. Measurements are tracked in real time, down to the finest grain, everywhere. Whenever you see something new on your favorite site—Google, Facebook, LinkedIn, YouTube, or anywhere—and you click it, know that someone, probably wearing headphones and eating Doritos, just saw a little counter go up by 1. That’s when the richness of data can drive a person crazy: one of Google’s best designers, the person who in fact built their visual design team, Douglas Bowman, eventually quit because the process had become too microscopic. For one button, the company couldn’t decide between two shades of blue, so they launched all forty-one shades in between to see which performed better. Know thyself: It was etched into a footstone of the Temple of Apollo at Delphi. But like the rest of the best wisdom that time has to offer, it goes right out the window as soon as anyone turns on a computer.

  Not knowing what customers need from a car, or even from a particular website interface—those are matters for a business school or a design workshop. It’s when people don’t understand their o
wn hearts that I get interested. People saying one thing and doing another is pretty much par for the course in social science, but I had a rare opportunity to see people acting in two contradictory ways. And it all happened because I didn’t know what they wanted either.

  On January 15, 2013, OkCupid declared “Love Is Blind Day” and removed everyone’s profile photos from the site for a few hours. The idea was to do something different and get a little attention for a new service we were launching at the same time. The programmers “flipped the switch” at nine a.m.:

  It was a bona fide pit of despair—rare in the wild! The new service OkCupid was trying to promote was a mobile app called Crazy Blind Date. With a couple taps on the screen, it would pair you with a person and select a place nearby and a time in the near future for the two of you to meet. The app provided an interface to let both parties confirm, but there was no way for anyone to directly communicate before the date. The only information it gave you about the other person was a first name and a scrambled thumbnail, like the one below. You were just supposed to show up and hope for the best.

  a CBD-style scramble of a stock photo

  You’ve probably already noticed that I’m speaking of Crazy Blind Date in the past tense. Even after a quarter million downloads, it failed, because in the end people insist on seeing what they’re getting into. The app was one of those ideas that looks great on a whiteboard and miserable in the full color of creation—it was like one long “Love Is Blind Day,” and with no way to flip the switch back to normal. A few months after launch, we shut the service down, but before Crazy Blind Date went off to the great app store in the sky (little-known fact: there are no bugs in heaven, just sweet features), about 10,000 people used it to share a beer or a cup of coffee with someone they’d never seen or spoken to before.