Here is an example that illustrates some of the steps in the process of putting scenes together. These are bare-bones sketches, really, as the animation cannot be conveyed through stills.
This set of stills comes from the sequence in which Pi and Richard Parker are caught in a swarm of flying fish.
the vfx assembly line: who does what, or why end credits go on and on
As a computer-generated tiger, Richard Parker’s central nervous system comprised a far-flung network of communication, computing, and creativity, stretching from New York to Los Angeles, by way of Mumbai and Hyderabad, Pi’s homeland, where Rhythm & Hues also had animators working—whole teams of people working on every yawn, stretch, and leap, take after take, for weeks on end.
roaring water, flowing tiger: the animation pipeline
Rhythm & Hues created a specialized workflow to deal with the animation needs of Life of Pi, a film that operated under two particular conditions: 1) the fact that much of the film was shot on real water, which needed to be blended in varying degrees with CG water (except in the storm sequences, where it was entirely replaced); and 2) the fact that Richard Parker was both a real tiger and a computer-generated one, and that the cutting back and forth between the two had to be absolutely seamless.
The Shot Production Team
The shot production team made sure that the elements in the 3-D shots (right eye/left eye, color) were properly aligned and matched, and they made adjustments accordingly. Tracking took the live production footage and all the other data gathered on set (including camera movements, lenses, and so on) and matched those with the camera in the virtual world. Although the animators could move their characters in a “normal” virtual space from any perspective they chose, tracking ensured that the end result could then be re-matched with what was shot in the first place. The camera department did the virtual camerawork, working closely with the FX section of the water and sky team to create a feeling of ocean movement and drift—particularly when the waves as shot in the tank had to be augmented, or when a scene was originally shot on a gimbal, whose mechanisms could never quite replicate the smooth ebb and flow of actual water. Final comp, or compositing, took every element produced for a shot by the other departments—ocean, sky, animated characters—and blended them with the live footage, ensuring that every component matched perfectly in order to produce the image that would end up on the movie theater screen.
The Water and Sky Team
The CG ocean section was in charge of water surface simulation, including parameters of height, basic lighting, and look. The art department handled the sky, taking images from HDRI (high dynamic range imagery) cameras that captured 360-degree views of the sky, and making any necessary adjustments and additions, such as cloud layers and movements. Special effects added spray, foam, and bubbles—all the details that give water its texture—and also oversaw the interaction of the CG water with the other elements such as the boat, the characters, and so on. Ocean lighting came in last, adding glints and highlights and bringing together all the ocean and sky special effects elements in the shot.
The Character Animation Team
Animation layout prepared the environment for the animator—that is, the spaces, surfaces, and objects with which the animator’s characters (tiger, hyena, or other creatures) would interact. Animation was responsible for the actual choreography and physical movements of the tigers and other animal characters, working with relatively rudimentary, stripped-down versions. These would then be filled in by tech animation, who were responsible for simulating the muscle, skin, and fur that would give the characters the detail and texture of life. The respective roles of these two types of animators are analogous to those of the CG ocean and FX divisions of the water and sky team; and similarly, at the end of the line, character lighting lit all the characters in the animated shot to match the lighting that was used in the actual shoot.
Photograph: Erik-Jan de Boer.
The mighty reference.
reference, reference, reference
In the past, many of the other creatures that fell under Bill Westenhofer’s supervision at Rhythm & Hues, like Aslan the lion in The Chronicles of Narnia: The Lion, the Witch and the Wardrobe, were required by their roles to smile or scowl, speak in the stentorian voice of Liam Neeson, and generally take on human or superhuman attributes. In Life of Pi, Richard Parker is required to do something that’s in many ways more challenging in the context of Hollywood filmmaking: he must simply remain a tiger, true to his animal nature, just as King did on the set.
“The main challenge is to not overanimate,” says animation supervisor Erik-Jan de Boer. “In terms of that feline-like motion, it is finding the proper balance. And again, we just use reference, reference, reference for all our stuff.”
“It’s to fight our own subconscious tendency to anthropomorphize things,” says Bill Westenhofer. “By sticking doggedly to reference, we are keeping nuances and keeping subtleties of animalism in the performance.” (“Bill is a real filmmaker,” Lee says admiringly about Westenhofer’s attention to detail.)
Illustrations: Rhythm & Hues.
An outtake from a project for using a person as a stand-in for the tiger—an idea that was scrapped in favor of stuffies.
“The absolute ultimate goal of mine and the whole effects team is to make it look like we did nothing at all.”
—BILL WESTENHOFER , VISUAL EFFECTS SUPERVISOR
By “reference,” de Boer and Westenhofer mean the hundreds of hours of video and thousands of photos that their team took of tigers, particularly of King. At the end of 2010, the two men began with a visit to Thierry Le Portier’s compound, where they grabbed some initial still and video footage of King and Vlad, the hyena. De Boer returned to Los Angeles and started putting together a group of some fortyfive animators—with teams in Los Angeles, Mumbai, and Hyderabad—to begin modeling the tiger.
Modeling means making sure the proportions and details of the real tiger are reflected in the digital model (skin and fur attributes come later, once the basic geometry has been established). “The tricky part is, of course, you’re dealing with an animal that will not do what you’re telling it to,” says de Boer. “It won’t sit still.” (It will sometimes lunge at you, he forgets to add.)
De Boer went to Taiwan to capture more video reference and stills for the final, detailed modeling of the CG tiger. “Basically, I shot lots of close-ups of the nose for breathing patterns. Yawning and snarling and hissing and eating, drinking, grooming, marking. Sleeping, pissing. How does a paw change shape when it takes the weight? And how does it change shape when the weight rolls over it for a step? How do the nails protract and retract? Along with regular footage, De Boer stuck a single-lens reflex camera inside the boat and set it to snap away automatically while the tigers ran through their routines, enabling him to get some amazing close-ups without getting mauled.
All of these images and videos of King—as well as other tigers from a variety of outside sources—were then catalogued and broken down by category. There were pages of just snarls, tongue flicks, ear twitches, and so forth: any given shot might use references from several different sources. “And body wise, we tried to find the closest mechanical action we can, so we always have that reality check, that backup from reference, to find out if that is realistic, possible. Is this something that the animal can do?”
But the Richard Parker that appears on screen—the CG version—is not merely a Frankenstein patchwork of disparately sourced behaviors and gestures. About a third of the Richard Parker in Life of Pi is bona fide, flesh-and-fur tiger. And in many other shots, even when Richard Parker is fully computer generated, what you see is not based on what King might do, but what he (or Themis, or Minh, or Jonas) actually did. In those cases, the real tiger footage might have worked reasonably well, but needed some adjusting—either for a technical reason or for an aesthetic one—to heighten a subjective emotion, for example: “There are moments where Ang wants viewers
to feel cinematically what they’d feel emotionally when they’re facing a tiger,” says Westenhofer, “so it was important to make sure the tiger’s eye connected to the camera. A real tiger’s never going to look at a camera in the same way it looks at the person it’s facing off at.” There isn’t any very effective way to manipulate a shot of the real tiger so that it’s looking in a different direction (for one thing, tigers turn their entire heads, not their eyes—to do the latter would seem, in Westenhofer’s words, “really weird”). “So it’ll be a CG tiger, and we’re going to base our performance off of footage of a real tiger and make adjustments so the animal is looking at the lens.” Since there’s no way of switching in midstream, every shot is either strictly real or strictly digital, and the transition from one to the other is made in the cut.
In such cases, animators would often use virtual motion capture to translate a live tiger’s performance into computergenerated imagery, matching every frame of a real shot with the CG tiger performing the same action, making whatever small changes that were necessary along the way (such as the adjustment of the gaze mentioned by Westenhofer), but otherwise staying as true as possible to all the movements and the physicality of the live animal footage.
fearful symmetry: king and richard parker
This is the story of two tigers.
One of them we know as King—the regal star of the tiger compound. King is flesh and blood.
The other tiger is called Richard Parker. Originally the star of a Man Booker Prize–winning novel, in which author Yann Martel sketched him in with a few brief descriptions (leaving the rest of him to take shape in the reader’s imagination), Richard Parker underwent the long and arduous transformation from word to image, to be reborn as a computer-generated (CG) tiger. And he got his physical appearance—every last stripe of it—from King.
King was unaware of this. But there was a purpose to things they made King do, because pictures were being taken, measurements made, and teams of effects specialists halfway across the globe were already hard at work during the same time. A process had begun, at the end of which the real King would have an exact double in the virtual world; and without ever having met, the two tigers became as one in the film Life of Pi.
The first step in creating Richard Parker was modeling. To get King’s basic proportions and details into the digital model of the tiger, the animators from Rhythm & Hues worked from the measurements and the reference photos and videos that de Boer and Westenhofer had taken of King at trainer Le Portier’s compound.
Once his overall appearance was roughed out, two different teams of effects specialists worked on the realization of Richard Parker, who was built from the inside out. First in line were the animators, who constructed a skeleton, on which they added a layer of textured skin—with stripes, but no fur—to produce a plausible-looking tiger figure that they could move around for blocking out and choreographing each shot.
As Richard Parker’s basic performance was being created by the animators (who were directed by Ang Lee), the second group, the technical animators, came in and added the subsequent layers to the computer-generated tiger: simulations of tiger muscle, skin, fur, and markings—all of which imparted texture, weight, and presence to Richard Parker.
Animation: Setting Richard Parker in Motion
Although a lot of research on tiger anatomy went into the building of the model, the virtual skeleton is not an anatomically correct tiger skeleton. It’s a simplified structure, used to determine where the joint and pivot points would be for movement, and which parts would be visible through the layers of muscle and skin. “We used it as a structure to hang our other muscles and fat and skin jiggle controllers on,” says de Boer.
Through the course of their teams’ work, animation directors will often return to the skeleton as a point of reference. If there is any uncertainty about what an individual animator is doing, the shot can be done with just the skeleton as a way of double-checking that the motions are something that the animal would be capable of.
The next layer in building the digital tiger was textured skin animation. King’s basic outline and markings were now in place as the computer-generated Richard Parker, but in a smooth, simplified form. “It looks like a tiger that’s been shaved, basically,” says Westenhofer. This was the figure that is used for blocking, the initial choreography that maps out where the animal is over the span of the shot—the rough action was then discussed and developed with the director.
Now came the work of refining the tiger’s movements and placing him in the external world. In the case of Richard Parker, it meant creating his physical reactions to the environment around him, such as nausea in response to certain movements of the boat or a sense of instability when walking across the tarp. Behavioral gestures were put into place as well—for example, Richard Parker’s tic, mentioned earlier, of tapping against the water bucket before taking a first drink. “We continue just finessing these details with the animators,” Westenhofer says. “We talk to them about weight shifting, about where the weight is, about accelerations, about collisions and impacts, friction. There are facial details, too. Any overt physicality—that’s what we’re dealing with.”
How to build a CG tiger, from the inside out:
Bones and skin animation.
Textured skin animation.
Muscle and bones simulation.
Simulated skin.
Simulated skin with grid tech.
Simulated fur.
Tiger build VFX stills: Rhythm & Hues.
Technical Animation: Giving Richard Parker His Stripes
While the skeleton-and-textured-skin version of Richard Parker was being run through his paces by the animation team and critiqued by Ang Lee, the groundwork was being prepared for the technical animators, who would start their work once the basic blocking of a shot or scene had been determined.
Technical animators specialize in the finer details of animation: muscle twitches and skin movement, the effect of wind on fur, gravity and motion on the flaps of loose skin that swing around the belly, the pressing of paws, and the protracting of claws. They also handle the interaction between the animal’s volume and that of external objects—the impact of a collision with the sides of the lifeboat for example, or the press of the animal’s weight as it walks across the tarp. As the animators set the tiger in motion, the tech animators give it presence, and the muscles constitute the core layer in this process. The movement of the muscles is what determines the behavior of the CG animal’s outer, visible layers—the surface of the skin (the twitches, contractions, and so on) and, on top of that, the fur.
King (left) and Richard Parker in his final stage.
Tiger build VFX stills: Rhythm & Hues.
Like the skeleton used by the animators, the CG tiger’s muscles reflect not so much correct tiger anatomy as what’s needed to achieve the moving, lifelike surface textures necessary for the CG King to match the real-life King as exactly as possible.
In the skin simulation, the layer that wraps over the muscles, the tiger’s surface is marked not with stripes but with a fine grid, which helps the technical animators to evaluate what the skin is doing—where the surface is stretching, contracting, or wrinkling, depending on position and movement.
A slightly different epidermal view is provided by the simulated skin with grid tech—with the letter-filled squares, Richard Parker looks like some kind of walking newspaper puzzle, but the purpose of these labeled areas is actually to help tech animators to map out the tiger’s color and stripe patterns.
Richard Parker finally got to put on a nice, warm layer in the fur simulation, but something was still not right: he was literally a white tiger, without so much as a single stripe. But this layer is purely about the texture of the tiger’s coat, and how it moves under the influence both of inner (muscular contraction, twitching skin, and so on) and outer forces (wind, water, gravity, and so on).
Finally, with the lighting comes the colors and patt
erns, the glorious orange-and-black coat that makes the CG tiger look like a real tiger. Wearing King’s clothes, Richard Parker was now ready for his close-up. And here is King once again: two tigers, one solid, one made of bytes. Which one is which?
Pi’s Ark
The other animals on “Pi’s ark,” as Pi calls it, were all constructed using more or less the same technology as was used to build up the tiger.
With Hari the hyena, as with Richard Parker, the animators had a well-documented real-life referent to both work off and try to blend seamlessly with—Thierry Le Portier’s hyena Vlad. A professional animator who appreciates a good kinesthetic challenge, de Boer says of hyenas: “In terms of their motion, they’re very stiff-spined. They have strange proportion, with a heavy head on a long neck and those tiny little hind legs that are just very powerful. But they’re sort of built to be always in a very balanced and secure frame on the ground. And that little seesawing acting that you get, with the head coming up and down, unweighting the hips so they can replant, you would almost want to compare it to how a giraffe moves. It’s been a real fun challenge to try and get that into our animated hyena. It’s very skittish, very nervous.”
Photograph: Susan MacLeod.
Mother and child orangutans from the Taipei Zoo, photographed as models for OJ.
The Making of Life of Pi Page 12