It hadn’t occurred to me that the real power of VR cinema is to completely obliterate the line between audience and performer. At the simplest level, an audience member can be a passive participant, a friend in the car, a person other characters talk to instead of just an unacknowledged viewer. At a more interactive level, their observations can actually cause the action—an actor could wait to speak until the viewer looks at him, or a scene won’t start to unfold until the viewer walks into the right room. And then there are VR films that could play out like Real Virtuality, where the viewer is actually the star of the film; exploring a set, interacting with other characters, and overcoming obstacles. It’s the difference between inserting the viewer in the Temple of Doom alongside Indiana Jones, and putting the viewer in Indy’s actual place. The future of filmmaking isn’t just the most immersive 3-D movies ever—it’s a blend of cinema, video games, immersive theater, and fantasy role-playing. In The Matrix, we all are Neo.
And with that thought, I pulled my parka back on and headed back out into the snowy streets of Park City.
* * *
—
“VR movies—this is our mantra, this is our word, and this is going to be a landmark year,” the man said, raising a hand to shade his eyes from the stage lights and peer out at the crowd. “For the first time ever, the general public will be able to watch VR movies in their own homes. This is going to be a game changer.”
Colum Slevin’s title was Head of Experiences for Oculus VR, but on that night his job was master of ceremonies for a party at the St. Regis Deer Valley hotel. Oculus had enticed Sundance attendees with the promise of “cocktails, light bites, good company and great content,” and a strange brew of Hollywood insiders, visual artists, and virtual reality nerds had shown up. After about an hour of awkward mingling, Slevin kicked off a few presentations with a short speech and a promise that “the bar’s gonna stay open.”
Oculus threw the party to promote its cinematic VR division, Oculus Story Studio. It had been founded in 2014, after Brendan Iribe ran a tech demo of the Rift headset for a major Hollywood director. “He took [the headset] off at one point and said, ‘Brendan, let’s make a movie. How do we do this?’” Iribe said. “But I didn’t have an answer for him. We really wouldn’t know how to do that. So we said, let’s go back and study this, and figure out how to use the Rift for cinema.”
To do that, Oculus hired Saschka Unseld and Max Planck, two veteran directors from Pixar Animation Studios, and put them in charge of a ten-person team that included alumni from other Hollywood powerhouses including DreamWorks and Industrial Light & Magic. Their mission was to develop a computer-animated film that could show off the Rift’s storytelling capabilities, and hopefully inspire other filmmakers to start working in virtual reality.
Story Studio’s first effort, a film called Lost, depicted a first-person encounter with a friendly robot in a forest and had debuted a year earlier at the 2015 Sundance festival. I’d seen it earlier and I’d been impressed. The short was only about five minutes long, but the virtual reality environment allowed it to be studied according to the viewer’s own interests and pace.
When I was immersed in the movie, I could stop and examine areas of particular interest, and the environment would respond. I remember looking at a swarm of fireflies, and then they flew over to check me out.
In 2016, Oculus had returned to Sundance to show off another animated film, Dear Angelica, which I’d watched a few hours before the Story Studio party. The movie told the story of a young woman writing a letter to her mother, and it was like nothing I’d ever seen before. As I stood in her bedroom watching her work, her memories sprung to life all around me as painted, animated works of art. It was touching, beautiful, and surprisingly intimate: I felt like an intruder in both her bedroom and her thoughts.
At the cocktail party, the film’s director, Saschka Unseld, and art director, Wesley Allsbrook, took the stage and showed how they painted the animations entirely in VR using a program called Quill—Oculus’s answer to the painting program I’d tried out on the Vive, back at GDC.
“In Dear Angelica, when you walk closer to a memory, it will unfold and wind around you exactly the way Wesley drew it,” Unseld said. “It would be impossible to create this if she had drawn on a flat page . . . we needed to create this inside of VR for it to truly be different, to truly be something that’s unique.”
Unlike the other films at Sundance, Oculus hadn’t brought Dear Angelica to Park City in order to sell it to a distributor. Rather, the company was looking to sell filmmakers on the idea of VR, to show off the tools and to demonstrate how the medium enabled new ways to tell stories. The elevator pitch was simple: totally immersive movies that the viewer can interact with and explore.
To help make the case to Hollywood, Oculus enlisted a celebrity guest to close out its party. Joseph Gordon-Levitt—the actor and director known for roles in films including Looper, Inception, and The Dark Knight Rises—took the stage after Unseld and addressed the crowd.
I’m not particularly experienced in VR, but I’ve had a chance here at Sundance to see some incredible work . . . and it’s just so inspiring.
There’s one particular storytelling device that I am personally quite interested in, being an actor, and that’s the device of the protagonist. You find a protagonist, of course, in all these different mediums . . . in the oral tradition, in the written word, in theater, or in movies. But protagonists function differently depending on technology. In a novel, the protagonist can go on and on for pages, just articulating their own internal thought process. You can’t really do that in theater or in a movie.
So what’s gonna be the protagonist’s function in VR? From what I’ve gathered so far, it seems like there’s a fork in the road. In some VR experiences, I am the viewer, I watch the protagonist. And then in other VR experiences, I am the protagonist. When you are the protagonist, that’s completely different than watching the protagonist. I feel like that’s where VR really kind of becomes its own self . . . it’s doing something completely new.
Whenever a new technology emerges, usually the first stuff that people try are like the old ways. The first movies were sort of like moving photographs or plays. But as new generations came and inherited the technology, the art of moviemaking developed its own language and its own devices. And that took, like, a hundred years. A hundred years ago, there were people that were saying, “Movies? That’s a novelty. That’s not art.”
The question now is, what are we gonna do in VR? I think the only way we’re gonna figure it out is to just try stuff. That’s why it’s so cool just to see you guys making things and trying things, and seeing how it feels to experience them. It inspires the hell out of me.
After the speeches were over, I went to get a drink and ran into Gordon-Levitt on my way back from the bar. I complimented him for his recent appearance on the TV show The Muppets (he sang “Fly Me to the Moon” with Miss Piggy) and then remembered we were supposed to be talking about virtual reality filmmaking.
“In some of the demos I’ve done, I notice an intimate connection with the performance that you don’t get in a movie theater,” I told him.
He nodded in agreement. “I feel sometimes that it’s more like theater than movies,” he said. “There’s such a diversity to VR, and that’s the fun part. There aren’t any conventions. If you go to a multiplex, all of the movies are basically following all the same rules, but I’ve watched so many VR experiences here at Sundance, and they all had different rules. It’s such an exciting time.”
I asked if there was a particular VR movie that really excited him.
“Chris Milk’s Evolution of Verse . . . the one with the train,” he said. “I knew I was standing there with a headset, but I absolutely physically felt scared that the train was gonna bowl me over. That’s how people in movie theaters felt a hundred years ago, when the Lumière brothers put a camera righ
t by the train tracks. Now you see people doing VR here at Sundance, and they’re screaming and crying and falling over.”
He laughed. “It’s just so exciting. Most generations don’t get to inherit this kind of open sandbox, where the rules aren’t set, and it’s time to play and figure it out.”
* * *
—
The following morning, I shook off my open-bar headache and trudged through the snow to check out the other side of the virtual moviemaking business. The Finnish consumer electronics company Nokia Corporation had rented a ski lodge on the outskirts of Park City to show off a new piece of hardware they billed as the first-ever virtual reality camera made for professional filmmakers.
Until recently, recording any kind of video for immersive cinema had required new inventions and unconventional hardware. Early attempts at immersive filmmaking, like the Sensorama booth or Cinerama theaters, used unique wide-angle lenses and curved screens to draw in the viewer. Decades later, 360-degree videos (like the documentaries The New York Times created for its Google Cardboard project) were filmed with hacked-together rigs of eight or more separate digital cameras, each one pointed in a different direction and recording video that had to be stitched together on a computer, requiring massive amounts of slow and costly postproduction.
Nokia’s Ozo camera was a simple—but costly—solution to that problem. The camera was a single orb of plastic, the size of a melon, studded at regular distances with eight embedded camera lenses. Each lens had a 195-degree point of view, so together they could see in every direction surrounding the camera. And since each lens was connected and run by the same software, the system could capture 360-degree video in real time, automatically stitch the images together, and then immediately record it to a computer’s hard drive or stream it over a network. Eight microphones dotted around the exterior also captured stereo audio synced to the video capture.
When Nokia officially debuted the device two months earlier, they’d recorded a live band performing on top of the iconic Capitol Records Building in Hollywood, and streamed the video to attendees watching in VR at a party across town in Los Angeles. They’d also announced the list price at a staggering $60,000.
At Nokia’s ski lodge, I took a few minutes to politely watch a VR recording of the concert, and then buttonholed Ramzi Haidamus, at the time the president of Nokia Technologies, to ask him if there was really a market for such an expensive product.
“Yeah, absolutely,” he insisted. “This is the only product on the market today that is purpose-built for high-quality VR capture in 3-D, audio, and video.”
“But there are enough people who want to do that and who can afford to spend $60,000?”
“The first phone that we made cost $4,000, and back then that was a lot of money,” said Haidamus. “You make your first products to learn, to get feedback, to see what works and what doesn’t work . . . This is going to be the flagship, and then I imagine going into a second line of products that’s more for the average consumer.” Nokia, he argued, was already in talks with every major Hollywood movie studio, including 20th Century Fox, where the company’s in-house futurist, Ted Schilowitz, had been a key partner in the development of the product.
Historically, Nokia was best known for developing mobile phones, I pointed out, and companies like Samsung and Google clearly saw the future of VR as happening first on mobile. So why didn’t Nokia develop a cheap mobile VR viewer first, instead of a high-end camera?
“Initially, we did work on a viewer,” he said. “We had an interesting experiment going on. But I felt that the market was too crowded, and it’s gonna be so competitive. And I wasn’t sure how much room there will be to differentiate, because some of the viewers are going to become standardized, just like televisions today.” Every headset needs content, he said, but consumers don’t need every headset.
As we chatted, I mentioned how I’d been at the Oculus Story Studio party the night before, and how excited the young filmmakers at the event seemed to be about making movies in virtual reality. “Do you see VR entertainment becoming a dominant form of media?” I asked him. “Are VR films going to put flat movies out of business?”
“I look at VR as a much broader opportunity,” he said. “The biggest opportunities are going to be in commerce. How many times do house hunters see a photograph of a house they’re interested in, but it turns out the photograph was of the best part of the house, and eighty percent of the house was left unseen until they go there in person? Imagine if the real estate agent had walked Ozo around there. You cannot hide anything from it. It sees everything in 3-D. It’s gonna be huge in real estate. Imagine the opportunities in education, where kids are taken around the world on trips. You don’t even have to leave your desk, you just sit in the comfort of the classroom, and you show them dangerous things. I see this as a huge market.
“But going back to your question about movies—I see VR as a complement to the current form, not a substitute. I love movies. I would love to see Star Wars made in VR, to visit another planet, to go to Mars. But people love to go sit in front of the big rectangular screen. This is gonna be a really interesting way to complement that.”
* * *
—
Just a few hours later, at another Sundance cocktail party, the movie studio 20th Century Fox promoted a big-screen blockbuster by offering their guests a virtual visit to another planet. The event was held in the VR Bar, a converted conference space in a Park City business center, and when I arrived it was dimly lit, filled with dance music, and crowded with fashionable people who weren’t talking to one another. They sat on leather divans, wearing Gear VR headsets and bright red over-ear headphones, or perched in front of computers wearing Rift headsets and holding Touch controllers. Their heads cocked at strange angles, staring at nothing, observing a world inside the headset and not the party around them. Their bodies twisted and turned in response to virtual obstacles, sometimes bumping into real people, spilling their drinks, or jostling trays of canapés. Other attendees stood in line waiting for their turn, watching this strange silent dance with a mix of bemusement and incredulity.
Inside their headsets, the partygoers were taking a tour of Mars via The Martian VR Experience, a virtual reality short based on the sci-fi film The Martian. The interactive adventure allowed viewers to experience the perspective of the film’s protagonist, astronaut Mark Watney, as he tried to survive alone on the surface of the red planet. The project was produced by the film’s director, Ridley Scott, and was largely shot during the production of the movie, using the same actors and sets that would appear on-screen in theaters.
“It’s a brand extension of the movie,” said Mike Dunn, president of 20th Century Fox Home Entertainment. “It’s really leveraging that content, going out during the theatrical window while you have a consumer’s interest. They can get into [The Martian] either through VR or through theaters. I think that if you watch our VR experience, there’s no way you won’t go see the movie later, and if you see the movie and then find out there’s a VR experience, you’ll want to go to Mars and experience it for yourself.”
The Martian VR Experience was only the latest VR film to emerge from the Fox Innovation Lab, a cutting-edge think tank run by the movie studio and dedicated to creating “true next generation home entertainment experiences.” In 2016, the lab premiered a VR experience called Wild, tied into the Fox Searchlight film of the same name. The three-minute, 360-degree video starred Reese Witherspoon as her film character, Cheryl, in an original scene tied into the movie. I’d tried it a few months earlier, and was impressed when it drew me in using methods that would be impossible in a movie theater. In the short, Cheryl hikes up a trail through a sun-dappled forest, toward the stationary viewer, and when she gets close, she sits down on a large rock to catch her breath. The immersive video made the scene unusually intimate, and seeing Cheryl in 3-D in front of me made her feel like a real person. But
Wild got real when Cheryl turned and looked directly at me; we shared a connection for a minute, until I realized she wasn’t looking at me but through me.
I turned around, and discovered another character had appeared in the forest: the ghost of Cheryl’s mother, played by the actor Laura Dern. Since the VR application knew which direction my head was facing, it was able to insert her into the scene when I wasn’t looking, and wait until I was actually facing her to have her start talking. When she finished, she sat there until I turned back to Cheryl, and then she disappeared again while I wasn’t looking.
It was a simple trick, but one that illustrated an exciting aspect of virtual reality: the idea that an experience can be passively interactive. Unlike video games, which require users to actively navigate through a conflict or narrative, VR films can allow the viewer to relax and watch but still control the action. It opens up interesting new ways to tell a story—like dialogue that changes depending on which character you’re looking at, or alternate scenes that you can only see if you move your virtual self into a different room.
Most of the studio-produced VR to date had been immersive but static, essentially just 360-degree videos with no interactive elements. But increasingly, experiments like Wild and The Martian VR Experience included segments where users could take control. At the Sundance party, guests using a Rift headset and Touch controllers could drive Watney’s rover across the Martian landscape, for instance. It wasn’t the most audacious use of virtual reality, but the audience seemed to enjoy it. “This is so freaking dope!” a young woman wearing a headset and a backless knit dress told me right before she unknowingly grabbed my thigh, unable to see that I wasn’t her boyfriend.
Defying Reality Page 16