We talk to several members of the Sony Pictures Imageworks team responsible for bringing many of Green Lantern's visual effects to the screen.
As principal vendor on Green Lantern, Imageworks delivered more than 1,000 shots for the film, which sees test pilot Hal Jordan (Ryan Reynolds) before suddenly granted superpowers from a mysterious ring and finding himself among the intergalactic Green Lantern Corps. Despite the high shot count - made up of CG suits, CG characters, environments and the deadly Parallax creature - Imageworks' visual effects had to fit into the real world inhabited by Hal Jordan, as envisaged by director Martin Campbell.
"Martin comes from a very physically-based filmmaking school," says Imageworks visual effects supervisor Jim Berney. "A lot of this film wasn't just trying to get a spectacle out there on the screen. The thing Martin wanted was that it had to be physically-based and look real. That posed a challenge for us because for the Green Lantern - it's all about his energy. It's about green effects and his constructs and environment."
CG suits and characters
Members of the Green Lantern Corps appear in luminescent and somewhat translucent suits. Imageworks created all of the suits as entirely computer generated effects, both for fully digital characters such as Kilowog and Tomar-Re and as body replacements for Hal Jordan, Sinestro and Abin Sur. The studio referred to concepts from production designer Grant Major and lead creature designer Neville Page for the character and suit looks, as well as previs from Pixel Liberation Front (PLF) for many of the shots in the film.
"In terms of the suit," explains Berney, "the idea was that when somebody puts this ring on and they become a Lantern and it incorporates into them and changes their mythology. We see Ryan Reynolds in this suit and we see depth to it. We see the muscle movement and the outer layer sliding against this inner layer. It gave us this ability to have this energy that not only crawled on the outside of the surface but also coming from within."
On set, Ryan Reynolds and the other actors would wear gray suits with tracking dots around their limbs. "We had two to five witness cameras," says Berney, "and we had other vendors track the camera and the body movement. There was no hard line to blend the neck and suit - no collar - all the deformations of our CG suit had to match the deformations of the practical neck."
Since Ryan Reynolds would at times be seen sporting a green mask - also an entirely CG creation - the actor had dots painted on his face and neck for a rough performance capture. "The vendors tracked the dots and gave us those dots in 3D space," says Berney. "We had different neck shapes based on a Mova session that we did, and used a lot of hand-rigging. Those were applied semi-procedurally. That got us about 99 per cent of the way there. We had to do a little bit of 3D warping to get it to line up to the head, and then it all had to work in stereo as well."
Interestingly, a bluescreen was used when shooting scenes of Hal Jordan flying or others requiring CG backgrounds. "There's probably no greater occasion for filmmaking where blue would be preferable for keying than green," jokes Berney. "But actually if we did it again I think I would have gone with green. They actors were not actually in costume when we filmed them and I would have much rather had green spill than blue spill. There was so much interactive green light played during the scenes as well that it might have been better to have that as green screen."
For animation of the CG suits, artists took a multi-stage approach. "First we start with core animation," says Imageworks animation supervisor David Schaub. "We have a tracking process where we align the hips all the way through the spine, through the shoulders, up through the neck and into the head. Not until that's absolutely right do we move on to the limbs."
"We have a system of deformers to really lock down the connection between the digital skin and really get the silhouette to line up precisely," adds Schaub. "In the world of animation, that's the equivalent to our primary animation task. Then we move on to our secondary animation - the squishy feel - bringing it away from the plastic mannequin feel and to a world of organic where you really do feel the musculature activating under the skin."
In the case of Hal Jordan, the final suit look, rendered through Arnold, consisted of an outer layer that was essentially skin sliding over the top of a muscle layer. Within that, the VFX team aimed to show the muscle volumes and shapes moving and working in order to demonstrate the Green Lantern energy emanating from the suit.
The energy itself turned into a lengthy design process for the Imageworks crew, based on significant concept art. "There's nothing more subjective than energy," admits Jim Berney. "We wanted it to be powerful and tough - but not electrical, and not fire - and it had to be grounded in reality somehow. And the energy patterns had to change as Hal learned about his powers or got better at flying."
"Some of that energy was done with shading," says Imageworks digital effects supervisor David Smith. "On the surface there were curves or geometry that we would generate or shade as a green plasma. We could bring the energy up and down depending on the physiology or attitude of the character, such as when he's just hovering or about to attack."
Jordan's green mask, also added by Imageworks, had to stand up to particularly close scrutiny. "The important thing there was maintaining his performance," notes Smith. "People are always looking right at the eyes. It often took the animators going in by hand and matching the performance captured data perfectly to get a mask that behaved as well as Ryan."
Of course, in addition to the hybrid suited characters, Imageworks produced animation for numerous other characters of the Green Lantern Corps, ranging from bipedal aliens to squid-like creatures. Thirty-five main ones were featured, then replicated and modified for various shots showing thousands of Corps members. Many of the featured characters were realized almost purely as key-frame animation and only a small amount of motion capture.
"For example," says Berney, "Tomar-Re (voiced by Geoffrey Rush) has a beak and also a 'goat leg' and that can be very difficult if not impossible to do as mo-cap. So we made a decision before we were shooting that we weren't going to try and motion capture the characters on set. We had stand-in eyelines and people to act against, but all the animation for those characters was hand-crafted after the fact, with some mo-cap for basic movement."
Kilowog (Michael Clarke Duncan), on the other hand, had to be animated to reflect his immense weight and volume. "He's like the drill sergeant and teaches Hal the ways of the Green Lantern Corps," notes David Schaub. "So the challenge was to make Kilowog really feel like he has weight. After all, he's eight and a half feet tall compared to Hal Jordan who stands at six feet."
Hal Jordan as Green Lantern soon discovers he is able to 'will' or conjure up almost anything he can think of - known as constructs. These ranged from shields to machine guns and other weapons. "The challenge with the constructs was figuring out where they came from," recalls David Smith. "We all know it comes from his mind, but physically where does it come from? We came up with the idea of a reverse explosion - this way we could deconstruct the object and bring in all kinds of matter as it came together to form the construct."
During filming Ryan Reynolds mimicked holding or operating each construct, such as a Gatling gun that he fires. "We had to get him to feel like he was firing that thing," says David Smith, "so we added a ton of jiggle of him responding to the action of the gun as it went off."
For each construct, Imageworks modeled and textured the object as if it was a normal CG creation being added to the scene. "But since it's made of the energy of will it all had to be green," notes Jim Berney, "so we tinted it green and that would be our base-line of what a construct was. Then we had to make it a little transparent with the green energy. So usually at the root or core of it we had this moving energy around the inside. That meant we didn't just model the outside, we had to model all the inside workings of the object as well. As you move towards the business end of the object it became more rigid and solid and more real."
The constructs were then pieced together in many layers via compositing. "Each construct was different with different materials," says Berney. "It wasn't just A over B over C compositing - it was almost hand-made and every little highlight having to be treated differently to deal with that."
Hal Jordan's nemesis in Green Lantern is Parallax, an organic mass of stolen souls which grows ever-larger and larger as it takes over planets and finds its way to Earth. "The idea behind Parallax was that he was made up of a bunch of skeleton souls with connecting tissue," explains Berney. "It's not just a complete avalanche heading down the streets. We wanted to give it a purpose, like a predator. He had to feel scary, and not just because it has a bunch of gaping mouths."
The Parallax shots - around 200 in total - included both the creature itself and large environment extensions and city re-creations. Animators provided a gross movement in Maya by firstly moving Parallax's 'goo pods' - tendril-like structures - by hand to perform large actions like grabbing buildings and propelling itself forward. The motion of the skeleton souls were then driven by Massive. "We collected cycles of animation that we turned into Massive simulations of enormous numbers of these souls that were then filled into the pods and tendrils," says David Smith. "Then we connected all those together with tendons and veins and wrapped it all with a diaphanous tissue that goes over the whole thing and keeps the creature as one body of being with all the souls lingering inside."
For Imageworks, it was important that Parallax be realized not just by simulations, but also artist input. "If we had just used simulations, it would have just looked like a waterfall or something along those lines, says digital effects supervisor Peter Nofz. "That was not what the director wanted. Our sims were layered on top of it which made it feel like an organism that thinks but also has weight in the real world."
Parallax's hero face, actually the villain Krona, was rendered to be able to deliver a performance against Ryan Reynolds, as well as integrate into the multitude of souls. "The head of Parallax also changed over time," says Smith. "He had to go from being a more speaking part to an angry creature. We created a model that had a simple speaking version with Krona's head and then one that was much more scary and dangerous looking as he morphed into a soul sucking version of himself."
Imageworks relied on its proprietary volumetric and modeling rendering tool SVEA to create smoke and fluid layers for Parallax. "Some of the renders took up to 80 hours per frame," notes Nofz. "We had a massive renderfarm at our disposal - up to 18,000 cores to make this character go through the pipe." The final Parallax shots were often made up of more than 20 layers, composited in Imageworks' Katana package.
Hand-crafted and out of this world
If the suits, constructs, digital characters and Parallax effects were not enough, Imageworks also contributed significant CG environment work for the Green Lantern Corps planet Oa. In addition, a number of their shots - almost 700 - were dimensionalized by the studio in-house. For Jim Berney, overseeing such a large volume of shots at Imageworks presented the opportunity to deliver both hand-crafted and diverse effects. "Green Lantern offered a lot more aspects than say other traditional superhero movies. The Green Lantern Corps are around the universe and that gave us the chance to go outside one single city or one planet. It offered the ability to create incredible characters as well as numerous amazing environments and spectacular effects."
All images copyright © 2011 Warner Bros. Pictures.
Produced in partnership with The Daily, NY.
For more film news check out The Daily on the Apple iPad from the App Store.
We've been a free service since 1999 and now rely on the generous contributions of readers like you. If you'd like to help support our work, please join the hundreds of others and become an fxinsider member.