The Inevitable

by Kevin Kelly

  I wore VR goggles all through high school. These lightweight frames give a much more vivid image than glassless AR. In class I’d watch all kinds of simulations, especially how-to rehearsals. I preferred the “ghost” mode in maker classes, like cooking or electrical hacking. That is how I learned how to weld. In AR I slipped my hands into the position of the teacher’s ghostly virtual guide hands in order to correctly grip the virtual welding rod held against the virtual steel tube. I tried to move my hands to follow the ghost hands. My virtual welds were only as good as my actions. For sports I wore a full helmet display. I rehearsed my moves with 360-degree motion on a real field, shadowing a model shadow body. I also spend a lot of time practicing plays in VR in a room. A couple of sports, like broadswording, we played entirely inside VR.

  At my “office” I wear an AR visor on my forehead. The visor is a curved band about hand width wide that is held a few inches away from my eyes for extra comfort during daylong use. The powerful visor throws up virtual screens all around me. I have about 12 virtual screens of all sizes and large data sets I can wrestle with my hands. The visor provides enough resolution and speed that most of my day I am communicating with virtual colleagues. But I see them in a real room, so I am fully present in reality as well. Their photorealistic 3-D avatar captures their life-size likeness accurately. My coworkers and I usually sit at a virtual table in a real room while we work independently, but we can walk around each other’s avatar. We converse and overhear each other just as if we are in the same room. It is so convenient to pop up an avatar that even if my real coworker is on the other side of the real room, we’ll just meet in the AR rather than walk across the gap.

  When I want to get really serious about augmented reality, I’ll wear an AR roaming system. I put on special contact lenses that give me full 360-degree views and impeccable fictional apparitions. With the contacts on, it is very difficult to visually ascertain if what I see is fake—except that one part of my brain is aware that a seven-meter-tall Godzilla stalking the street is absolute fantasy. I wear a ring on one finger of each hand to track my gestures. Tiny lenses in my shirt and headband track my body orientation. And GPS in my pocket device tracks my location to within a few millimeters. I can thus wander through my hometown as if it were an alternative world or a game platform. When I rush through the real streets, ordinary objects and spaces are transformed into extraordinary objects and spaces. A real newspaper rack on the real sidewalk becomes an elaborate 22nd-century antigravity transponder in an AR game.

  The most intense VR experience of all requires a full-body VR rig. It’s a lot of trouble so I suit up only occasionally. I have an amateur rig at home that includes a standing harness to prevent me from falling while I flail about. It gives me a full cardio workout while chasing dragons. In fact, VR harnesses have replaced exercise equipment in most basements. But once or twice a month I join some friends at the local realie theater to get access to state-of-the-art VR technology. Wearing my own silk underwear suit for hygienic purposes, I slip into an inflatable exoskeleton that closes around my limbs. This generates amazing haptic feedback. When I grasp a virtual object with my virtual hand, I feel its weight—the pressure against my hand—because the inflatable is squeezing my hand just the right amount. If I bump my shin against a rock in the virtual world, the sheath on my leg will “bump” my shin just so, making a totally believable sensation. A reclining seat holds my torso, giving me the option of doing genuinely felt jumps, flips, and dashes. And the accuracy of the super-hi-res helmet, with binaural sound and even real-time smells, creates a totally convincing presence. Within two minutes of entering, I usually forget where my real body is; I am elsewhere. The best part of a realie theater is that with zero latency 250 other people are sharing my world with equal verisimilitude. With them I can do real things in a fantasy world.

  * * *

  • • •

  VR technology offers one more benefit to users. The strong presence generated by VR amplifies two paradoxically opposing traits. It enhances realness, so we might regard a fake world as real—the goal of many games and movies. And it encourages unrealness, fakery to the nth degree. For instance, it is easy to tweak the physics in VR to, say, remove gravity or friction, or to model fictional environments simulating alien planets—say, an underwater civilization. We can also alter our avatars to become other genders, other colors, or other species. For 25 years Jaron Lanier has talked about his desire to use VR to turn himself into a walking lobster. The software would swap his arms for claws, his ears for antennae, and his feet for a tail, not just visually, but kinetically. Recently at the Stanford VR lab Lanier’s dream came true. VR creation software is now agile and robust enough to quickly model such personal fantasies. Using the Stanford VR rig, I too got to modify my avatar. In the experiment, once I was in VR, my arms would become my feet, and my feet my arms. That is, to kick with my virtual foot I had to punch with my real arm. To test how well this inversion worked, I had to burst floating virtual balloons with my arms/feet and feet/arms. The first seconds were awkward and embarrassing. But amazingly, within a few minutes I could kick with my arms and punch with my feet. Jeremy Bailenson, the Stanford professor who devised this experiment and uses VR as the ultimate sociological lab, discovered that it usually took a person only four minutes to completely rewire the feet/arm circuits in their brain. Our identities are far more fluid than we think.

  That’s becoming a problem. It’s very difficult to determine how real someone online is. Outward appearances are easily manipulated. Someone may present himself as a lobster, but in reality he is a dreadlocked computer engineer. Formerly you could check their friends to ascertain realness. If a person online did not have any friends on social networks, they probably weren’t who they claimed to be. But now hackers/criminals/rebels can create puppet accounts, with imaginary friends and imaginary friends of friends, working for bogus companies with bogus Wikipedia entries. The most valuable asset that Facebook owns is not its software platform but the fact that it controls the “true name” identities of a billion people, which are verified from references of the true identities of friends and colleagues. That monopoly of a persistent identity is the real engine of Facebook’s remarkable success. And it is fragile. The normal tests we used to prove who we are in digital worlds, such as passwords and captchas, no long work very well. A captcha is a visual puzzle that was easy for humans to solve, but hard for computers. Now humans have trouble solving them, while machines find it easier. Passwords are easily hacked or stolen. So what is the better solution than passwords? You, yourself.

  Your body is your password. Your digital identity is you. All the tools that VR is exploiting, all the ways it needs to capture your movements, to follow your eyes, to decipher your emotions, to encapsulate you as much as possible so you can be transported into another realm and believe you were there—all these interactions will be unique to you, and therefore proof of you. One of the recurring surprises in the field of biometrics—the science behind the sensors that track your body—is that almost everything that we can measure has a personally unique fingerprint. Your heartbeat is unique. Your gait when you walk is unique. Your typing rhythm on a keyboard is distinctive. What words you use most frequently. How you sit. Your blinks. Of course, your voice. When these are combined, they fuse into a metapattern that almost can’t be faked. Indeed, that’s how we identify people in the real world. If I were to meet you and was asked if we had met before, my subconscious mind would churn through a spectrum of subtle attributes—voice, face, body, style, mannerisms, bearing—before aggregating them into a recognition or not. In the technological world, we’ll come to inspect a person with nearly the same spectrum of metrics. The system will check out a candidate’s attributes. Do the pulse, breathing, heart rate, voice, face, iris, expressions, and dozens of other imperceptible biological signatures match who (or what) they claim? Our interactions will become our password.

  Degrees of
interaction are rising, and will continue to increase. Yet simple noninteractive things, such as a wooden-handled hammer, will endure. Still, anything that can interact, including a smart hammer, will become more valuable in our interactive society. But high interactivity comes at a cost. Interacting demands skills, coordination, experience, and education. Embedded into our technology and cultivated in ourselves. All the more so because we have only begun to invent novel ways to interact. The future of technology resides, in large part, in the discovery of new interactions. In the coming 30 years, anything that is not intensely interactive will be considered broken.

  10

  TRACKING

  We are opaque to ourselves and need all the help we can get to decipher who we are. One modern aid is self-measurement. But the noble pursuit of unmasking our hidden nature with self-measurement has a short history. Until recently it took an especially dedicated person to find a way to measure themselves without fooling themselves. Scientific self-tracking was expensive, troublesome, and limited. But in the last few years extremely tiny digital sensors that cost a few pennies have made recording parameters so easy (just click a button), and the varieties of parameters so vast, that almost anyone can now measure a thousand different aspects of themselves. Already these self-experiments have started to change our ideas of medicine, health, and human behavior.

  Digital magic has shrunk devices such as thermometers, heart rate monitors, motion trackers, brain wave detectors, and hundreds of other complex medical appliances to the size of words on this page. A few are shrinking to the size of the period following this sentence. These macroscopic measurers can be inserted into watches, clothes, spectacles, or phones, or inexpensively dispersed in our rooms, cars, offices, and public spaces.

  In the spring of 2007 I was hiking with Alan Greene, a doctor friend of mine, in the overgrown hills behind my house in northern California. As we slowly climbed up the dirt path to the summit, we discussed a recent innovation: a tiny electronic pedometer that slipped into the laces of a shoe to record each step, then saved the data to an iPod for later analysis. We could use this tiny device to count the calories as we climbed and to track our exercise patterns over time. We began to catalog other available ways to measure our activities. A week later, I took the same hike with Gary Wolf, a writer for Wired magazine, who was curious about the social implications of these emerging self-tracking devices. There were only a dozen existing ones, but we both could see clearly that tracking technology would explode as sensors steadily got smaller. What to call this cultural drift? Gary pointed out that by relying on numbers instead of words we were constructing a “quantified self.” So in June 2007 Gary and I announced on the internets that we would host a “Quantified Self” Meetup, open to absolutely anyone who thought they were quantifying themselves. We left the definition wide open to see who would show up. More than 20 people arrived at my studio in Pacifica, California, for this first event.

  The diversity of what they were tracking astounded us: They measured their diet, fitness, sleep patterns, moods, blood factors, genes, location, and so on in quantifiable units. Some were making their own devices. One guy had been self-tracking for five years in order to maximize his strength, stamina, concentration, and productivity. He was using self-tracking in ways we had not imagined. Today there are 200 Quantified Self Meetup groups around the world, with 50,000 members. And every month, without fail, for eight years, someone at a Quantified Self meeting has demo’d an ingenious new way to track an aspect of their life that seemed unlikely or impossible a moment before. A few individuals stand out for their extreme habits. But what seems extreme today will soon become the new normal.

  Computer scientist Larry Smarr tracks about a hundred health parameters on a daily basis, including his skin temperature and galvanic skin response. Every month he sequences the microbial makeup of his excrement, which mirrors the makeup of his gut microfauna, which is fast becoming one of the most promising frontiers in medicine. Equipped with this flow of data, and with a massive amount of amateur medical sleuthing, Smarr self-diagnosed the onset of Crohn’s disease, an inflammatory bowel disease, in his own body, before he or his doctors noticed any symptoms. Surgery later confirmed his self-tracking.

  Stephen Wolfram is the genius behind Mathematica, a clever software app that is a math processor (instead of a word processor). Being a numbers guy, Wolfram applied his numeracy to the 1.7 million files he archived about his life. He processed all his outgoing and incoming email for 25 years. He captured every keystroke for 13 years, logged all his phone calls, his steps, his room-to-room motion in his home/office, and his GPS location outside his house. He tracked how many edits he made while writing his books and papers. Using his own Mathematica program, he turned his self-tracking into a “personal analytics” engine, which illuminated patterns in his routines over several decades. Some patterns were subtle enough, such as the hours when he is most productive, that he had not detected them until he analyzed his own data.

  Nicholas Felton is a designer who has also tracked and analyzed all of his emails, messages, Facebook and Twitter postings, phone calls, and travel for the past five years. Every year he generates an annual report in which he visualizes the previous year’s data findings. In 2013 he concluded that he was productive on average 49 percent of the time, but most productive on Wednesdays, when he was 57 percent productive. At any given moment there is a 43 percent chance he is alone. He spent a third of his life (32 percent) sleeping. He used this quantitative review to help him “do a better job,” including remembering the names of people he met.

  At Quantified Self meetings we’ve heard from people who track their habitual tardiness, or the amount of coffee they drink, their alertness, or the number of times they sneeze. I can honestly say that anything that can be tracked is being tracked by someone somewhere. At a recent international Quantified Self conference, I made this challenge: Let’s think of the most unlikely metric we can come up with and see if someone is tracking it. So I asked a group of 500 self-trackers: Is anyone tracking their fingernail growth? That seemed pretty absurd. One person raised their hand.

  Shrinking chips, stronger batteries, and cloud connectivity has encouraged some self-trackers to attempt very long-term tracking. Particularly of one’s health. Most people are lucky to see a doctor once a year to get some aspect of their health measured. But instead of once a year, imagine that every day, all day, invisible sensors measured and recorded your heart rate, blood pressure, temperature, glucose, blood serum, sleep patterns, body fat, activity levels, mood, EKG brain functions, and so on. You would have hundreds of thousands of data points for each of these traits. You would have evidence while at both rest and at full stress, while sick and healthy, in all seasons, all conditions. Over the years you would gain a very accurate measurement of your normal—the narrow range your levels meander in. It turns out that, in medicine, normal is a fictional average. Your normal is not my normal and vice versa. The average normal is not very useful to you specifically. But with long-term self-tracking, you’d arrive at a very personal baseline—your normal—which becomes invaluable when you are not feeling well, or when you want to experiment.

  The achievable dream in the near future is to use this very personal database of your body’s record (including your full sequence of genes) to construct personal treatments and personalized medicines. Science would use your life’s log to generate treatments specifically for you. For instance, a smart personalized pill-making machine in your home (described in Chapter 7) would compound medicines in the exact proportions for your current bodily need. If the treatment in the morning eased the symptoms, the dosage in the evening would be adjusted by the system.

  The standard way of doing medical research today is to run experiments on as many subjects as one possibly can. The higher the number (N) of subjects, the better. An N of 100,000 random people would be the most accurate way to extrapolate results to the entire population of the
country because the inevitable oddballs within the test population would average out and disappear from the results. In fact, the majority of medical trials are conducted with 500 or fewer participants for economic reasons. But a scientific study where N=500, if done with care, can be good enough for an FDA drug approval.

  A quantified-self experiment, on the other hand, is just N=1. The subject is yourself. At first it may seem that an N=1 experiment is not scientifically valid, but it turns out that it is extremely valid to you. In many ways it is the ideal experiment because you are testing the variable X against the very particular subject that is your body and mind at one point in time. Who cares whether the treatment works on anyone else? What you want to know is, How does it affect me? An N=1 provides that laser-focused result.

  The problem with an N=1 experiment (which was once standard procedure for all medicine before the age of science) is not that the results aren’t useful (they are), but that it is very easy to fool yourself. We all have hunches and expectations about our bodies, or about things we eat, or ideas of how the world works (such as the theory of vapors, or vibrations, or germs), that can seriously blind us to what is really happening. We suspect malaria is due to bad air, so we move to higher ground, and that helps, a little. We suspect gluten is giving us bloat, and so we tend to find evidence in our lives that it is the culprit and then we ignore contrary evidence that it doesn’t matter. We are particularly susceptible to bias when we are hurting or desperate. An N=1 experiment can work only if we can separate the ordinary expectations of the experimenter from those of the subject, but since one person plays both roles, this is extremely hard. This kind of inbred prejudice is exactly what large randomized double-blind trials were invented to overcome. The subject is unaware of the parameters of the test and therefore cannot be biased. What helps overcome some of our self-fooling in an N=1 experiment in the new era of self-tracking is automatic instrumentation (having a sensor make the measurement many times for long periods so it is “forgotten” by the subject) and being able to track many variables at once to distract the subject, and then using statistical means later to try to unravel any patterns.