Raja Gosnell’s The Smurfs breathes real life into the famous miniature blue cartoon characters, who find themselves chased from peaceful village to scary New York by the evil wizard Gargamel. We talk to Sony Pictures Imageworks about the creation of the Smurfs and their world, and to Tippett Studio who realized shots of Gargamel’s sidekick cat Azrael.
‘Three apples high’ – Imageworks builds the Smurfs
One of the initial considerations for tiny Smurfs was just how to make seven-inch tall – officially ‘three apples high’ – blue-skinned characters read in a live-action film, and one that would also be stereo-converted. “The attitude towards the design process was to be true to the original as much as possible,” says Imageworks visual effects supervisor Richard Hoover. “For the blue skin, we used a lot of humanistic reference with pores and peach fuzz and wrinkles around their eyes and various levels of translucency. We built their anatomy and their facial controls based on what we expected from a human performance, albeit with an view towards the unique things about Smurfs – large eyes and ears and a larger head.”
The script also called for the Smurfs to do a lot of running around and exploring in the human world, and to be chased by humans, including Gargamel who makes his way into the New York world. “We played a lot with how to do that and pull that off,” notes Hoover. “We ended up with a pretty springy run cycle in order for them to reach stride to carry and outrun a human.”
“It was a balancing act,” adds Imageworks digital effects supervisor Daniel Kramer. “Initially we started by directly copying the Smurf from the cartoon, but it just didn’t match up to something that could interact with a human. We found that using the original eyes, which had a lot of sclera, the white part of the eyes, wasn’t very expressive when put into a 3D character. But if we put too much detail into them they didn’t look like a Smurf anymore, so the right balance of skin texture and proportions was important.”
The digital characters were modeled and animated relatively traditionally in Maya, using Sony’s proprietary character set-up. Effects were completed in Houdini. However, The Smurfs represented a major foray by Imageworks into its physically-based lighting approach, something it had also carried out to a significant degree for The Green Lantern. In particular, the studio used a Spheron camera on set to take 360 degree 26-stop HDRIs. “We had the ability to do the same scan twice at an exact distance apart of 30 inches,” explains Hoover. “That allowed us to triangulate distances to surfaces and lights and calculate things like the wattage and color of the lights in reproducing our lighting model.”
In addition, a Trimble laser scanner was utilized on locations to gather point cloud geometry of the set and help with scene reconstruction. Along with the Spheron images, this all fed neatly into Imageworks’ lighting pipeline, which includes the Arnold ray-trace renderer and the studio’s own proprietary Katana set-up (now sold by The Foundry UK). “In Katana,” says Kramer, “we can bring all these things together – the HDRI, the stereo pair image which captures scene lighting and scene geometry information. We take all that and build a crude version of the set. Using the HDRI image, we apply the amount of energy that’s coming off each surface on that set. So if it’s a white counter top that’s got a lot of bounce light coming off of it, a CG Smurf walking by will receive the correct amount of light. All of that information is loaded into Katana, where artists are free to manipulate that further – adding lights or subtracting lights, for example.”
Animation of the Smurfs began with previs, R&D and on-set reference. “Our animation director, Troy Saliba, worked out how fast they could run and whether they could climb over human-type things like couches and chairs and tables,” says Hoover. “On set I had these ropes I could lay down in the scene and say, ‘This represents one second of time to get from A to B’. Because of course the ropes are flexible you could lay them over a curved couch or table.”
“Then we made macquettes,” continues Hoover, “both hard and soft versions of the key characters that we could put into the scene so the actors could see and interact with something. We would also use those as blocking devices to puppeteer their actions. Everyone got involved in that. Raja the director was on his hands and knees a lot moving these and posing them and talking to the actors at the same time about the scene.”
Two full-time voice actors were also on hand during shooting, and assisted with the puppeteering and reading characters in a number of voices against the live-action performers. “In addition to all that, we would also use old-school techniques,” notes Hoover. “We’d put little tiny emery dots on the floor for markers so the actors knew their eyelines. And I used baling wire to make circular parts that would stand up straight to the height of the Smurfs’ eyes, and I’d position those around on the set. They were so thin and black they were easy to paint out later.”
The Smurfs’ village was realized entirely in CG by Imageworks in one of the studio’s largest ever digital environments. “The idea behind it,” says Hoover, “was that if you were a human and got down on your hands and knees and crawled around on your belly you would feel like you were there. It was all made out of natural materials, with the mushroom houses constructed of twigs and mud and straw. We didn’t want anything unreal or surreal about it, even though it’s in a magical world.”
To create the enormous environment – which featured trees with thousands of leaves and millions of blades of glass, not to mention scores of Smurf characters – Imageworks relied on deferring the geometry build until render time. “We have these amazing computers that are eight or 16 cores with four to eight gigs of memory devoted to each core,” says Kramer, “but the scenes were taking 20 to 30 gigabytes of memory just to load. So we would load as little as we can into the lighter’s scene, and once we felt satisfied we would turn back on the bells and whistles and let the renderfarm at it.”
The Smurfs enter our world through a magical portal. For the Central Park side of the gateway, Imageworks augmented a real waterfall environment with several water simulations. “That was an interesting physical exercise to make it both believable in the physics, but also supernatural in that it’s a time portal device,” comments Hoover. “I started with the idea that if there was a cylinder in there, in a waterfall, what would the water do? I also looked for a lot of reference of natural whirlpools – things that occur in the ocean naturally, how it behaved.”
Hoover gave the director a few illustrations of his ideas for the portal, which would ultimately be achieved by compositing two separate sims with differing energies together. “We also ended up adding a lot of water to the waterfall for the Central Park side of the portal,” says Kramer. “There was an actual waterfall there in the plate photography but we had to augment it quite a bit and add a whole CG layer of water there even when it wasn’t being a portal, which gave us our own curtain that we could part when the portal existed. Again, we used a Trimble scan of the area and our Spheron set-up to make sure it was based in reality.”Imageworks breaks down the portal layers.
Having and in fact, starting, with something real in just about all of the shots was a significant part of the film’s visual effects, as observed by Imageworks CG supervisor Dan Lobl: “I think the interesting part of the project that made it distinct from what we had done before, technologically, was definitely the process we used for capturing the set and using it to directly light the CG. We can see things that are very accurate very quickly, and that freed up the artists to spend more time doing artistic work rather than technical work.”
“It also felt more like we were part of the filmmaking process,” adds Lobl, “in that we were using the same technology and techniques as they would on set – say incorporating bounce cards for fill. We can do that now virtually. And with a renderer like Arnold and having all the HDRI information – it felt very analogous to working on set.”
Tippett Studio and the Smurfs’ feline foe
fxg: Your main work was Azrael – how was that character approached in terms of both live action and digital work?
Liedtka: That was probably the hardest thing – striking that balance between a photorealistic cat and one that has to act. On set, there were four trained cats, so if it was just walking from A to B, they would just use the live-action cat. When the cat had to do something dangerous or somehow emote, we did either a face replacement or an all-CG cat.
fxg: How did you capture the reference for Azrael?
Liedtka: Well, it’s funny because the four cats didn’t actually look the same on close inspection, although you won’t notice this in the movie. But when it came time to build them we had to pick just one, and it was actually a hybrid. It was the head from one cat and the body from another.
Right after the shoot wrapped, we set up a reference photography session in Los Angeles where we took a lot of very carefully planned out photos with a multi-camera rig and we also took measurements and close-ups. Then we would model the cat and pose it to match the photos and check the angles.
Liedtka: At first, they were cutting with a live cat so we needed to keep him on the realistic side. There were lots of shots where we did very straight, realistic cat behavior. We have done cats before so we looked at our reference we already had and of course the real cat plates.
When it came to pushing the limit, which we had to do, William Groebe, our animation supervisor, would try out a few things or pitch a couple of ideas. There were meetings where we were down on the floor and pulling stunts. It got everybody going and laughing, and really got the director’s trust.
There were clearly some shots they knew they could never get a live action cat to do. And for those shots the cat didn’t even show up for work that day! They had a whole sequence where Smurfette and Azrael are in an extended chase sequence. First she hits him with a shovel and then gets him to step on tacks and jumps on his back – there’s just no way they could get a real cat to pantomime a character that’s not even in a scene.
There were other shots where they were hoping to use the real cat – they’d try a few takes with the real cat and keep going or decide to do it with our CG cat. We’d sometimes just replace the face, do an eye roll or a puzzled look or make a mean, menacing look.
Liedtka: Well, we had come off Cats & Dogs 2 and Kitty Galore had been a huge project for us. On that we had to do ton of work on the wrinkles and have it working with the fur system, and she also talked a lot. In this case of Smurfs, we had a cat that wasn’t going to deliver any lines, although we did do a face rig that enabled expressions and eye controls.
We used Maya and Mudbox for modeling, and some internal tools to do the color and fur painting, along with Photoshop. Then for anim we used Maya with our proprietary tools. We comp’d with Nuke on this show. There were some fluids effects in Houdini, and rendering in RenderMan. We actually took one of our cleanest cat models from Cats & Dogs 2 and re-worked it a little, but still used Furator, which is out node-based fur tool. We had to push the fur around to match the cat actor. Orange fur, because of the color and saturation, is a little trickier.
fxg: You mentioned having to do shots with Smurfette on Azrael’s back. How did you share assets with Imageworks for shots like that?
Liedtka: Overall what we would do is basically pretend we were doing the whole shot. We needed a stand-in for whoever Azrael was acting with, say for the scene of Smurfette riding on his back and pulling his ears. Those two characters needed to be animated together as 3D objects, not just as renders. So we planned it out with Sony. In shots where Azrael was the key player, we would block that whole shot and then add blocking for the Smurfs that were going to be in the shot with him. On the flip side, where the Smurfs were the major thing, they would block Azrael for us.
Both Sony and Tippett approach animation in a similar way with roughing out the layout and getting approval on that, and then moving to the next step which we call blocking, and then to a final. We generally do three or four different reviews of animation, and only the last review has fur on it. In the early stages we would trade models and rigs and block each other’s characters.
In order to finish the shot they obviously needed to be lit and composited consistently. We were using image-based lighting and Sony has pretty much moved over to a physically-based system with the Spheron camera, which we could use for our lighting too.
Images © 2011 CTMG, Inc. All Rights Reserved.