Fringe takes an animated trip

A recent episode of Fringe took a unique turn by featuring the show’s characters in an animated dream-state. Series visual effects supervisor Jay Worth looked to Zoic Studios to create 240 shots of stylized hand-drawn and 3D animation for the sequence. We talk to both Worth and Zoic’s creative director Andrew Orloff.

The actors delivering dialogue as reference
Final animated shot
In the episode, entitled ‘Lysergic Acid Diethylamide’, scientist Dr. Walter Bishop (John Noble) and his son Peter (Joshua Jackson) enter the conscience of FBI agent Olivia Dunham (Anna Torv) using LSD, in the hope of rescuing her from the mind of fellow scientist William Bell (Leonard Nimoy). Inside they encounter a number of Olivia’s visions, including her step-father, visit some spaced-out locations like the World Trade Center and even travel on a zeppelin.

Fringe already contains significant live-action visual effects, so Worth turned to frequent collaborator Zoic for the animation and environments. “It was initially going to be 12 minutes,” he says, “with six environments and all locked-off cameras. In the end it went up to about 16 minutes with eight or nine locations and we moved the camera in almost every scene to make everything require a 3D environment!”

For the actual look of the animation, Worth and the show’s producers felt that a hand-drawn style with an obvious texture was the right approach. The film Waltz with Bashir, graphic novels, Max Fleischer and Hanna-Barbera cartoons were all used as reference. “We had to choose a methodology that would stay with the vibe of the show,” adds Worth. “We do a lot of stuff with long lenses and hand-held cameras. The camera is almost always floating and moving and we always have nice depth of field. I wanted it to really look like our show and not another world.”

The process started with the real actors, and a stand-in for Leonard Nemoy, recording their lines. For non-action scenes, the actors were also filmed from the proposed correct camera angles. “When we first pitched it,” says Worth, “we were going to plan the whole thing with storyboards and not have any live-action. The more I thought about, I thought our actors do such a great job, why don’t we get hints of their performance? We’re not doing Avatar – we’re not doing full-on performance capture – I just wanted to get their nuances, their head-turns and reactions.”

Storyboards then helped flesh out the action scenes, such as shots of the zeppelin. A rooftop fight scene came together based on a fight staged by the stunt team during filming for Fringe on its Vancouver studio. And another scene involving a Humvee chase utilized a more somewhat low-key, albeit effective, methodology. “We didn’t have a storyboard artist with us for that,” explains Worth, “but we had a 5D, so I called my wife and asked her to bring over my kids’ toys and action figures and we literally shot little animatics of the different frames.”

– Above: Watch a making of video for the episode (from Fox Broadcasting).

At Zoic, artists worked on character designs and the backgrounds. To enable both the preferred graphic look and a smooth workflow for the 240 shots, the final elements would be separated out into up to 40 passes for each shot. “The shading of the characters and the lines of the characters and the backgrounds were all different pieces,” explains Zoic’s Andrew Orloff. “In fact, the outside lines of the characters were dynamically generated using Paint Effects strokes and mental ray contour lines. And the insides were generated by actually having the artist doing the character sheets create orthographic versions of the model and draw the details in, then projecting them on top and rendering them out separately.”

“When we got close up to the characters,” continues Orloff, “the level of detail on the line work of the character didn’t quite look consistent in every environment. So we developed a system in Nuke where we were bringing in the MBD files, the actual animated models. We had a 2D artist paint on top of the line work, and then projected that back down.”

Toy cars and figures as reference
Final shot
To animate the characters, Zoic used the reference from the actor read-throughs as a template for the performances. “We weren’t specifically rotoscoping over the top of the footage,” says Orloff. “Instead, we had a 3D model with a facial rig, and we were looking at the actors and translating their facial performance into keyframes. And then they’d add animation on top of that where necessary.”

The various backgrounds, sets and environments seen in the sequence had to reflect previous Fringe locations. Zoic created 2D concept art for approval, which was then finaled and mapped onto simple 3D geometry to accommodate the moving camera. “There was a layout process laying out the cameras with some basic 3D geometry,” says Orloff. “For example, for the Jacksonville street corner where we see all the businesses, they were all matte-painted storefronts mapped onto 3D geometry in Nuke.”

The final touches came in terms of adopting familiar cinematic techniques. Jay Worth looked to existing Fringe footage and camera and lens specs as reference for the animation, including for matching depth of field and lens flares, as well as to replicate the show’s natural camera float. “We actually over-rendered every scene slightly to get that signature Fringe feeling that the camera was never locked-off,” says Orloff. “It was Jay’s idea to go into previous shots and track the background and apply that to the animated piece, which we could do because we rendered everything slightly larger.”

By the end, the animated portion of ‘Lysergic Acid Diethylamide’ had been produced in only six weeks, a testament to Worth’s close collaboration with Zoic and a desire to fit the piece into the Fringe universe, despite its stylized look. “For me,” says Worth, “when there’s a gimmick like this done in a show, it can feel like a gimmick – but I think this served the story really well. It almost did us in sometimes; out of the 240 shots, I think we finaled 120 of them in the last 16 to 20 hours. We had all the environments nailed down in the last three days, but everything came together really well.”

U.S. residents can watch the entire episode via Hulu here.

All images © 2011 WBEI. All rights reserved.