Tintin: Weta goes animated

With Steven Spielberg’s Tintin, Weta Digital drew on its definitive performance capture experience on Avatar and so many other visual effects advances made in the studio’s relatively short history to help bring Hergé’s detective series to the screen. We talk to Wayne Stables, one of Tintin’s VFX supes, about some of the film’s major challenges.

fxg: Weta has really been perfecting over the past 10 to 12 years in live action films its expertise doing huge environments, water sims, hair, cloth, performance capture and so many other effects. Watching Tintin, it felt like all of that was there in just about every shot. Did it feel like that while you were working on it?

Stables: Well, yes, I guess it comes down to that we have traditionally been a live-action visual effects house, so that’s a world we understand. We’re working with Steven being a live action director, so that’s the world he understands. We never really approached it as, ‘Hey, let’s make a live action film’ – it was just the only way we knew how to do it and it seemed natural for us. And I think it felt like a live action film by the end of it too.

– Above: watch a featurette on the making of Tintin, including interviews with Steven Spielberg and Peter Jackson, plus behind the scenes footage of the capture sessions at Giant Studios.

fxg: But even with that live action feel, the film obviously had to be stylized to some degree to match the original Tintin by Hergé – how did you go about matching that and where did you decide to depart from it as well?

Stables: Certainly the characters are stylized, and even the buildings and environments were based on any reference we had from Hergé. But even Hergé would tend to stylize things a little bit. We tried to stick as close to Hergé’s world as closely as we could, but there were a couple of things where we had to break away or go a step further on. Some of the locations we simply didn’t have reference for, such as Bagghar. A couple of his books are set in North Africa, but there wasn’t a lot of artwork for it. Hergé had a very particular style too, with strong lines and colors, but he didn’t necessarily have a lot of detail. So we would have to go in and fill in a lot of the detail. I don’t think we tried to depart from his style intentionally, but we just had to augment his work a number of times.

fxg: Joe Letteri [Weta’s senior visual effects supervisor on Tintin] said in an interview recently that Tintin was drawn relatively simply in the books and the challenge for Weta was to give him more life. What were the things that Weta had to do to make Tintin more complex?

Stables: Well, I love Tintin, but to be honest he’s kind of just a balloon with a couple of little dark eyes and a funny little oval-shaped mouth as he’s drawn. So we spent a huge amount of time going from that starting point and working out a character that could be a hero in a story, who would talk, who you would sense emotion from and that you could believe could carry a gun. If you ever look at our Tintin versus the original artwork, there are many similarities, but we just brought a few facial features in. Our guy has a jaw, cheekbones and his eyes are recessed, and he has very realistic ginger hair.

fxg: What were some of the tools you captialized on that had been developed for Avatar and Apes as you worked on Tintin?

Stables: We did do a lot of tissue work for the muscle work and face, and we were working on Apes at the same time so we could share a lot of the hair technology. We also developed more of our tools for layout for dealing with thousands of assets – a really large dataset – and how to render that. There was Naiad for the water sims and waves. We also developed tools in our lighting pipeline for our visual look, pushing on our indirect pipeline and our area lighting technology. This was really the first time for us that we would be raytracing true area shadows as the norm across the shot as opposed to an exception.

fxg: The other things is that Weta really approaches the facial performance in a very practical and real way with FACS and then for texturing you even use real skin displacement maps and obviously a realistic approach to hair – was that continued on Tintin?

Stables: Absolutely, I mean we have research scientists here who work on that stuff all the time. A lot of work was required for Snowy – he had a particular style for his hair and he also has white hair which can be difficult. We did do work with hair, shading wise, particularly with Tintin’s ginger color. We also did a lot of work with the transmission of light through hair and how it scattered. We used Barbershop – that’s our grooming tool. The hair is done by our modelers. So for Snowy they would use Barbershop to do the grooming. It enables them to paint maps and draw curves for the flow of the hair, and there are tools that let you visualize the hair.

fxg: There are some huge sequences in the film, like the one in Bagghar, but there’s also a fun sequence where Snowy chases Tintin’s kidnappers from outside, into the apartment and then he jumps out the window onto a firetruck. Can you talk about where you actually begin with a sequence like that?

Stables: We had a story point where Snowy is going to have to follow Tintin onto the SS Karaboudjan. Our previs department headed up by Jamie Beard who was the animation director – they took that as a scene and did a pass across it telling that story. We actually start off that sequence as a foggy day and end up in sunshine down by the dock. There’s crowds, cars and environments. That was based on discussions with Peter Jackson and Steven Spielberg. They form a scene out of it. Steven would come back with a bunch of notes, and they would iterate on it for camera moves and the action beats. We would also try and do stereo at that point in time, until Steven was happy with the previs.

– Above: watch the ‘Snowy to the rescue’ scene.

We would also talk to Steven at that time about how it should end up looking. So for example with the fog it needed to start foggy and then get broken up as we get down to the dock. Then we go away to a large extent and build a lot of assets – streets and cars and everything else – and then start to put together. If you saw our previs and then the final shots, there’s a huge amount of similarity between them. A lot of the same buildings were used for the previs but just used in a more cheaper and cheerful level than in the final shots. Similar cars are used for the previs. The tools our previs artists use for layout and set-up are the same tools we use in shots down the line. So we can take the previs world, up-rez the assets and the animation, and bring proper lighting and shadows. Once we’ve done an initial pass of the shots done, we’ll show Steven and keep talking. At that point we’ll start to bring the facial animation through. We just really iterate on it like that – it doesn’t really do a huge quantum shift from previs through to final lighting – it just gets built on.

fxg: How were the camera moves devised?

Stables: Steven Spielberg operated a lot of the camera moves on the Giant Studios stage, or they were done inside our previs area. They are very conscious of how they deal with the cameras. At times you are doing big sweeps that you wouldn’t be able to get with a real camera, but the camera work is designed as part of the shot along with the animation, as well as the stereo with the interocula and convergence. We also render it in stereo and review it early on. We also do various diagnostic renders, a top down render of the scene to show us what the camera is doing, where the focus is being set, where the convergence is – because we know there are some things that work and fit the style of the film.

fxg: In the scene there are men interacting with Snowy – how was that done on the performance capture stage?

Stables: They had a stand-in model of a dog built by Weta Workshop that looked like Snowy. That was a really important thing. When Jamie was out there doing the performance capture, they made sure that the actors had at least parts of the environment around them. So Snowy would always be there at about the place he would eventually be. That meant the actors had an eyeline to look at.

He ended up with a reasonably sophisticated muscle system, because with the ways dogs’ shoulders and hind legs work is pretty unique. Then the biggest thing was getting his fur to look right. One of the tricks was that we couldn’t put all the fur on Snowy that a real dog had, so we have to approximate to a certain extent by making the hair a bit fatter or shorter or wider to give it volume. So we spent the most time getting the groom right so that he felt like a slightly wirey-haired dog.

fxg: Again, I think Joe Letteri said they aimed to have Snowy steal the show in just about every scene.

Stables: Snowy does a lot of key poses and shapes in the book, and he does some crazy things as well – getting drunk on whisky, or digging up bones. So the animators did a fantastic job of taking the essence of Snowy and creating the joy of living, almost, that little dogs seem to have. They probably couldn’t have given him another second of screen time or he really would have taken over the movie.

fxg: A feature of that Snowy scene, I think, is the lighting inside Tintin’s apartment – can you talk about how you approached that?

Stables: The apartment was a real beast to get right, and I can tell you that because I did most of the shots inside there. The reason why it was kind of tricky was that, when you think about it, it’s actually a fairly basic shape. It’s just a box and the only real visible light source you have are the sets of windows on either side. But it’s also incredibly colorful so the color saturates inside there very quickly. We found that when you’ve got things like big red chairs or green things, they actually bounce a lot of light around. It gives you a really nice look, but we had to treat that – and maybe this is a live action thing as well – we had to treat that as if it was a live action shoot. When we lit it, we had a little bit of trouble when we looked out the window and would see these grey buildings that really weren’t that interesting, and the glass itself didn’t look that great. So what we decided to do was hang up a set of net curtains on the windows. That meant we got this big broad diffuse light source that was coming in. It’s not that different if you’re on a stage, and you don’t want to be looking out the windows at the concrete soundstage.

We took a very live-action philosophy, so if we took a light in front of the camera, it had to be an actual light source, such as a lamp. But there were no ‘magic’ lights. The result was that everything felt plausible and really felt like you were in a lit environment. That was even more important in places like the apartment because it enables you to use those curtains and get away with a very soft light that you’re going to have in a foggy environment.

fxg: What tools were you using for the area lights?

Stables: Everything was rendered with raytracing. We have an in-house renderer called PantaRay, a pre-computation engine, which we use in conjunction with RenderMan. It does the shadow calculations and it creates a huge point cloud of shadows, that our RenderMan light shaders look up and use to calculate the beauty render. We were also doing multiple bounce and direct inside there as well, which helps fill in those shadows in a very natural way.

fxg: When Snowy jumps out of the window there’s then the foggy atmosphere – can you talk about lighting outside versus inside?

Stables: I think lighting outside takes us to a place a little bit more similar to where we were with Avatar. We tend to use a lot more image-based lighting for skies. We would calculate occlusion using spherical harmonics. There was probably just more indirect bouncing for outside to what we had done on Avatar because that technology had been developed more. It would be fair to say that a lot of our external lighting setups in Tintin was using an IBL for the sky, putting a sun light inside – we even go to the extent of making sure it has the correct solid angle look-up as what the sun has which gives you the correct looking highlights et cetera. Then we’d also put a couple of beauty lights inside there – pretty similar to what you would have if you were shooting, really, and had a natural sky there.

fxg: You mentioned having to manage the assets efficiently on Tintin, especially for the big shot in Bagghar, but also for the Snowy to the rescue scene. How did you do that?

Stables: Well, Bagghar was fully 3D, there are no matte paintings in there at all, so what we would do is build things in components in order to deal with hundreds and thousands of assets. You might build seven base buildings and a whole bunch of awnings and staircases and fountains, and then you put a whole range of different pieces on that and a whole range of color palettes. Then we use some tools here – there’s one called Ensemble that we use for our layout to put them together with bounding boxes or much more simple visualizations in the scene. Also, these days if you have any type of foliage or tree, you have to put dynamics into it as well. You expect to see the leaves moving in the breeze. There’s no particular magic to deal with that, just a case of having tools that enable you to deal with logistics and work with things very quickly and have procedural-type variance across the shots.

Stay tuned in early 2012 when we we’ll feature a fxpodcast with Giant Studios’ performance capture supervisor Matt Madden about Tintin and Giant’s other recent work for Real Steel, Rise of the Planet of the Apes and Happy Feet Two.