The more I thought about it, the more excited I got. There are applications for this technology in every business and every home. Imagine educational software that lets students shrink to the size of an atom and study chemical reactions up close, then fly into space and witness the first steps on the moon. Virtual clothing stores where customers can see how an outfit looks on a perfect simulation of their own body. Design tools that let engineers sit inside and drive a new car prototype before it’s been built. Real estate open houses where buyers tour a new home without leaving their own home. Or wars fought entirely by robots, each one controlled by a soldier wearing a VR rig in a command center thousands of miles away.
I was getting ahead of myself. At that moment, I needed to stick to the facts and take careful notes about the Rift demo. I took out a notebook and started writing—what the hardware looked like, how it felt when I put it on my head, what the graphics looked like, and the plot of each short scene. But I kept having to stop because I wasn’t sure how to describe what had happened. Was I merely recording observations of a piece of media, like a movie reviewer recounting the plot of a film? Or was I making a memoir of an event I’d personally experienced?
As a reporter who writes about video games, I was used to describing participatory entertainment. Even though we as players experience the action through a fictional protagonist, we say, “I defeated Bowser and saved the princess,” not “I pressed a button that caused a collision detection function to trigger an animation indicating the game was over.” But the Rift had pushed the idea that gamers become their character to a whole new level. There was no hero character acting as an intermediary between me and the creatures I met in VR, and no TV screen or monitor physically separating me from their world. My whole life I’d been playing games that required me to ignore what I could see and feel in the real world and imagine I was somewhere else. But the Rift had completely flipped the script. VR doesn’t need you to ignore your senses—it needs you to embrace them. If anything, the challenge is to remember that what you’re experiencing isn’t real.
For all practical purposes, I had visited a city of skyscrapers. I had waved at an alien. A dinosaur had roared at me, and my body reacted like it was actually there. What did that mean? It surely didn’t feel like a game. And this was a first-generation product, comparatively primitive. What would happen when VR inevitably improved to a point where its simulations were indistinguishable from the real world? If I visit a virtual Paris that looks, sounds, smells, and feels exactly like Paris, then how is my experience any different from that of a person who physically travels there?
I’d been dreaming about virtual reality since I was a kid, but I wasn’t prepared for the reality of VR. My expectations were based on limited experience. I’d interacted with computers through graphical user interfaces that employ the metaphor of a desktop with files and folders, so that’s the way I thought of VR—an amazing way to access information, but basically just Microsoft Windows in immersive, realistic 3-D. It never occurred to me that a VR experience might trick my brain into thinking it was real, because in the thousands of hours I’d sat in front of a screen, I’d never believed my computer’s desktop was an actual physical table.
But there I was, my brow still damp with “oh no, a dinosaur is going to eat me” sweat, drying off in the California sun. And that’s when I started thinking that VR isn’t just going to be a new tool for business or a new medium for 3-D movies and video games. It’s going to change what it means to be human. This technology could allow us to escape the bodies we’re born in and the geographies that confine us. It could allow us to experience the impossible, to do the unthinkable. What happens to humanity when you can experience at the push of a button what it’s like to kill a person, or to have sex with them? Will we still feel the need to explore when the bottom of the ocean or the surface of Mars is just a click away?
The more I thought about virtual reality, the more I felt like one of the primitive apes at the beginning of Stanley Kubrick’s 2001: A Space Odyssey—the monolith has appeared in our midst, and all I could think to do was throw handfuls of dirt at it. I wanted to know how VR got here, how it worked, and where it was going; I wanted to understand how it might change our culture, our communication, and our consciousness.
* * *
—
So this book is an attempt to reach out and touch the monolith. It chronicles some of the things I learned and experienced in just over two years during the most explosive period of growth in the history of virtual reality—a time when we went from zero commercially viable products to millions of units sold by some of the most important companies in the tech industry. It documents the origins of virtual reality, from its birth in military labs, through the booms and busts that nearly doomed the technology, to the twenty-first-century inventors who learned from history and finally brought VR to the masses. And it also looks forward, examining how this remarkable tool might change different industries, pastimes, and the way we all relate to one another.
This book has also been an excuse for me to indulge those old childhood dreams of being a console cowboy, and try to get more people to join me in the virtual world. By writing it, I’m hoping to share all the cool stuff that’s happening right now, but I also want to get people excited. I want more hands making better hardware, smarter interfaces, and lots of great VR content. And I also want more people to get involved in the conversation. Right now, a small group of people are building a place where we’re all going to spend a lot of time in the future. They’re amazing people, but that’s too big a job to leave in the hands of a lucky few. We need everyone involved: women and men, adults and children, rich and poor, able and disabled. We need computer scientists collaborating with philosophers, ethicists, and artists. We’ve got the chance to build a new world, so let’s make it for everyone.
The more I’ve learned about VR, the more I’ve become aware of a gulf that’s dividing our present and future. Most of us stand on the near side and wonder what’s across the water. Maybe they haven’t had the time, money, or opportunity to experience virtual reality. Or maybe they’ve tried it and still remain skeptical. But as the tech gets cheaper and better, every day more people will come across the water. First by the thousands, soon by the millions, and before you know it, virtual reality will be a part of daily life for billions of people.
I hope this book will be a message in a bottle encouraging you to take the journey.
Chapter 1
PYGMALION’S SPECTACLES
There is a unicorn in a cavern under an ancient forest in France. I’ve seen it myself, even though it wasn’t really there; I’ve stood in front of it, even though its home is a place where no one can go.
The unicorn is painted on the wall of Lascaux Cave, a subterranean complex in the Vézère Valley, about two hours’ drive east of Bordeaux. Seventeen thousand years ago, some of the first modern humans in Europe decorated the caverns with hundreds of images of animals, symbols, and abstract shapes. In 1940, it was rediscovered by a group of teenagers and became one of the most famous archaeological sites in the world.
For decades following the discovery, countless tourists flocked to Lascaux. Henri Breuil, a French priest and archaeologist who was one of the first researchers to study the caves, called it the Sistine Chapel of prehistory; after artist Pablo Picasso saw the paintings, he lamented that his entire generation of revolutionary artists had “invented nothing new.” But the ancient drawings were fragile, and the presence of so many visitors took a toll. In 1963, the French government banned the public and locked the doors.
Fortunately, technology allows a modern tourist to sneak inside. Lascaux comes to life in 360-degree videos and virtual reality tours; I’d seen photos of the caves and studied their paintings in college, but it took seeing that unicorn in VR to really understand the place. Even though the French government has built detailed replicas of the caves for people to visit, t
he virtual version makes for a better trip—it’s appropriately claustrophobic and free from the distracting presence of other tourists.
It’s also a particularly fitting way to visit, because Lascaux may be the oldest example in human history of an attempt to create a virtual world. You’d never know it from looking at photographs, but when you stand inside the caves—virtually or in person—the intention is clear. Just like VR headsets block the real world from view, the caverns separate a visitor from the forest above. Instead of drawing with pixels on a screen, the cave artists used pigment on rock walls. Lascaux’s creators used the topography of the caverns to create immersion: a chamber decorated with sketches of wild horses isn’t just a jumble of drawings, but a herd that surrounds the viewer. They used perspective tricks to make the illustrations seem three-dimensional: the body of an ox is presented in profile, but its head is turned to face the viewer. And they used the shape of the rock to give their art depth and form; for example, a twist in a wall makes a deer appear to turn away as it dashes around a corner.
No one knows why the people who created the caves went to so much trouble. Perhaps the goal was to enthrall the viewer in order to teach them something, like an Upper Paleolithic version of a flashy how-to video on YouTube: As writer and technology expert Howard Rheingold suggests, “subterranean cyberspaces” like Lascaux may have served to “imprint information on the minds of the first technologists.” Or maybe the caves were immersive entertainment, an attempt to tell a story in the most realistic way possible, to convey the excitement of hunting without the risk of getting gored or trampled.
In the millennia after the cavemen left their caves, increasingly advanced civilizations created increasingly sophisticated immersive entertainment. Around 2,500 years ago, the ancient Greeks built their own VR viewers out of wood, stone, and marble—the word theater comes from the Greek verb theasthai, or literally, “to view.” These technological marvels exploited then-cutting-edge science in order to trick audiences, including acoustics (semicircular amphitheaters enhanced sound waves, allowing large audiences to hear voices at a distance) and mechanics (a wooden crane called a mechane lifted actors into the air and made them appear to be flying; it’s the device that made possible the deus ex machina). Elaborate costumes, handheld props, and painted scenery panels helped complete the illusion, transforming the stage into a fantastic environment.
Other cultures created virtual worlds through visual arts like sculpture and painting. Qin Shi Huang, the first emperor of China, started building a city-sized mausoleum around 246 BC, filling it with terra-cotta sculptures of his soldiers, pets, musicians, and palace officials—a kind of simulated palace for him to occupy in the afterlife. The Romans used frescoes to transform their homes from dingy concrete into more pleasant realities, covering entire walls with illusionistic images. Buildings in the city of Pompeii, preserved after the eruption of Mount Vesuvius in AD 79, reveal interior rooms painted so that they appear to have balconies and windows that open up onto lush imaginary landscapes.
Chinese artists created immersive art using paintings on long scrolls of paper. Gu Hongzhong’s eleven-foot-wide The Night Revels of Han Xizai is a record of an actual night in the life of a tenth-century government minister. According to legend, it was created so emperor Li Yu could witness Han Xizai’s debauchery, including listening to music and watching women dancing. In the twelfth century, the artist Zhang Zeduan created a ten-by-seventeen-foot depiction of the city of Kaifeng, Along the River During the Qingming Festival, which includes accurate images of buildings, animals, and more than eight hundred people. By virtue of their size, real-life subjects, and narrative content, these panoramic paintings amount to another kind of virtual reality that transports the viewer back to dynastic China.
In 1787, the Irish-born artist Robert Barker took that idea a huge step further. Inspired by the sweeping 360-degree view from Calton Hill, a spot in central Edinburgh, he painted a massive seventy-foot-wide watercolor landscape of the city and hung it on the inside of a cylindrical surface. Viewers stood on a platform in the middle, surrounded by the painting, so that it simulated the experience of actually being in Scotland. He patented the technique as the Panorama, coining the now common term from the Greek words pan and horama—“all that which is seen.”
After first exhibiting the work in Edinburgh and Glasgow, Barker took it to London. Reviews were good, but audiences were limited, so Barker created another, bigger panorama, this time of the London skyline. English viewers liked that better, and the artist earned a small fortune selling tickets to see it. In 1793, Barker went even bigger, constructing a building in Leicester Square that could display two immense panoramas, one of them stretching floor to ceiling in a three-story-tall rotunda. The new venue became an immensely popular attraction and showed a variety of scenes, including foreign cities and historical military battles. By the early nineteenth century, panoramas were a worldwide phenomenon, with similar sites constructed in cities across the globe. The immersive experiences were even touted as a high-tech alternative to actual travel.
In 1822, the French artists Charles-Marie Bouton and Louis Daguerre combined the panorama’s massive imagery with elements of theater to create an attraction they called the Diorama—scenes made of multiple paintings on linen curtains over on a stage, constructed in such a way that when bright lights were shone on different spots, the images would appear to move and change. The shows were a big hit and were also copied around the world; later versions added sound effects, stage props, and even live actors to enhance the illusion. (Fueled in part by his success with the Diorama, a few decades later Daguerre pioneered another way to render realistic images—the early photographic process known as the daguerreotype.)
* * *
—
In a way, all art strives to create a virtual reality, to transport an audience and immerse them in a subject or story. But theater and paintings are only distant relatives of today’s high-tech 3-D headsets—the direct ancestors of modern VR came out of the study of optics.
Humans are able to view the world in three dimensions because our brains combine a number of different visual cues to create the perception of depth. Monocular cues require only one eye and provide information largely based on context, including the relative size of objects (a car that appears to be tiny is probably farther away than one that appears to be huge), occlusion (if one car blocks the view of another car, you can tell which one is closer), and motion parallax (when you’re driving in a car, objects on the side of the road move out of your field of view faster than objects that are far away).
Then there are binocular cues, which work because we’ve got two eyes spaced apart laterally on the front of our head. Each eye views the world from a slightly different perspective. You can see this by closing your right eye and looking through your left, then closing your left eye and looking through your right. Do that rapidly a few times and you’ll notice a difference in the apparent position of objects. That’s called parallax, and when your brain notices it, it can compare and contrast the images to calculate distance. The result is one of our most powerful binocular cues, stereopsis, and it’s the foundation of the simulated depth in virtual reality and 3-D movies. (Another binocular cue, convergence, is drawn from the inward movement of your eyes when they look at something up close; to test it, touch your finger to your nose and focus on it, and then hold it at arm’s length and look again. That movement you felt helps your brain figure out the relative distance of nearby objects, and it’s actually the cause of some problems in modern VR, since your distance to the screen never changes.)
The English scientist Sir Charles Wheatstone first identified stereopsis in an 1838 paper called “Contributions to the Physiology of Vision,” which described how the eyes capture two perspectives that the brain fuses into one three-dimensional image. It also outlined a device he’d invented that could “enable any person to observe all the phenomena in questi
on with the greatest ease and certainty”—an instrument he called a stereoscope.
It had a simple design: two mirrors attached at a common edge to form a right angle, and two wood panels, one facing each mirror at a 45-degree angle (viewed from above, it would look like the letters I V I). The viewer would position himself with his nose against the outside corner joint of the mirror so that his left eye saw the reflection of the left-hand panel, and his right eye saw the right-hand one. Each panel would hold a slightly different illustration of an object or scene. The viewer would look straight ahead and see two perspectives of what appeared to be the same object, and his brain would combine them into a single three-dimensional image.
In his paper, Wheatstone focused on how a few simple line drawings—cubes, cones, and pyramids—could be made to appear as if they were three-dimensional objects. But he also understood that with better graphics, stereoscopic devices might present convincing virtual realities. “Careful attention would enable an artist to draw and paint the two component pictures, so as to present to the mind of the observer . . . perfect identity with the object represented,” he wrote. “Flowers, crystals, busts, vases, instruments of various kinds . . . might thus be represented so as not to be distinguished by sight from the real objects themselves.”
After Wheatstone demonstrated his invention at an annual meeting of the British Association for the Advancement of Science, one of his rivals picked up the idea and ran with it. The Scottish physicist Sir David Brewster was already well known for inventing the kaleidoscope, an optical toy that had become phenomenally popular, and he spotted a similar opportunity in 3-D imaging.
Defying Reality Page 2