Populating Mr. Popper’s Penguins

In this interview we talk to visual effects supervisor Richard Hollander about Rhythm & Hues’ CG creations for Mr. Popper’s Penguins, the story of a New York businessman (Jim Carrey) whose life is turned upside down after he inherits six of the sub-Antartic birds.

fxg: So, where do you start when you want to capture the essence of a real penguin?

Hollander: Getting a realistic penguin was certainly incredibly important for the director, Mark Waters. We started with all the real data about penguins, going to biologists and getting to understand the actual skeletal structure of our particular penguin, which was a Gentoo. The film consists of real and CGI penguins throughout the film, and we had eight penguins at the studio in Brooklyn where the film was shot. I set up 18 still cameras that were sync’d to take a picture at the same time. I shot penguins posing for us, effectively, in different positions. We took that and made sure all our deformations did the same thing as the real penguins volumetrically – that they followed the same kind of shapes and forms that they had.

We also took a lot of close-up stills and did a lot of videoing of the penguins walking, running and doing general preening. We did that with three cameras – two in line and one orthogonal to them so we could understand the overall action. Keith Roberts, our animation director, was with me when we shot all of that. His team then started pulling cycles out of that video material. It was almost like a roto process starting there. So that was Rhythm’s overall work – the digital penguins. We also had some crowd shots and environments, and then Hydraulx and Centro did a lot of great work for other types of shots like backgrounds and snow additions. And there were penguin shots where we had to augment the real penguins with their identifying marks, and those companies helped us with those shots as well.

---

– Above: a shot progression from background plate, tracking, lighting through to final composite for one of the penguin marking augmentations.

fxg: How did that 18-camera set-up work?

Hollander: The 18 cameras was like a simplified bullet-time rig. Instead of 18 cameras firing a little bit at different times and space, I just had them in a circle and they all went at the same time. The penguins are about 20 inches tall and the cameras were about 10 inches off the floor the penguin was standing on, which happened to be an apple box.

I also set up the rig so that I had nine cameras that went in a circular bridge, but vertical, so I could look at the penguin from above and to the side as well. All that stuff is just extraordinarily useful – we just keep coming back to that data. We also took an HDR image at the time we shot that and tested our lighting and rigging under the auspices of a known condition where the penguin really was, because we A-B’d the two to make sure they were the same.

fxg: What were some of the things you got out of that reference, both in terms of the performance and say the way the light hits their feathers?

Hollander: In terms of performance, we got to see what kind of movement a real penguin does and what movement they really don’t do. Mark Waters was very interested in making sure that whatever we made the penguins do, when they did it they weren’t doing what a penguin couldn’t do. So, certainly no penguin can dance, especially in sync with their friends. But if they were to dance, their body and their ligaments and their muscle structure would allow them to do the dance we made them do. We weren’t going beyond the physicality of what a penguin was. We didn’t want to break those rules and we kept to that strictly. All the videotaping and reference led us to understand the envelope of what we could animate and how far we could go.

For the technical parts of the sheen of a penguin, light certainly does reflect off the feathers, and it does this because of an oil gland. They pick up that oil and preen their feathers with it, which is the first layer of defence against water – the other is a fat layer. That oil material is really reflective. And the feathers themselves are anisotropic – they don’t reflect the same in all directions.

You put that together with the fact that their white feathers not only reflect light, but they absorb light, scatter the light and of course re-transmit it. They have their own sub-surface way of dealing with light. We shot with the ARRI Alexa, and we did a series of tests in four lighting conditions and then we mimicked the penguin next to the real penguin and checked that work. I kept on joking that I wanted to take a live penguin and put it in one of those Light Stage devices – but of course I’d have to refrigerate the rig down to 35 degrees and, also, no penguin is ever static.

fxg: What tools were you using at Rhythm to animate the penguins?

Hollander: The artists started with the live imagery and worked their way through to the finished animation in our Voodoo package. Voodoo allows us to do roto and then have layers on top of that so you can modify it and give each penguin their own personality. In the soccer game that happens outside in the snow, for example, we picked up running and sliding stuff from our real data, but in almost every shot it was the animators putting layers in on top and making it happen.

One of challenges on this particular show was that within say just one shot in Popper’s apartment, a penguin could run from one place to another and go through three or four lighting situations. We had a HDRI camera on set and we took many HDR images from places that I felt the lighting was really significantly different and warranted having another HDRI there. An external shot with nothing close by is one thing, but even an exterior shot where you have people coming up to the penguins, you have to model the fact that you have people there affecting the light onto the penguins.

Then inside the apartment there are so many other objects and they all affected the light. So we did some pretty sophisticated imagery and geometry building where we were projecting the actual imagery onto the geo. So in the 3D abstract world, the penguin actually did go by a couch that had a similar color and could be affected by it.

fxg: What were some of the extra details you had to add in to sell the penguins – I’m thinking about their wings and twitches and the fat layer?

Hollander: We had a procedural layer in there for the fat. We tuned it in to something that was fairly realistic. There was a tech-pass that would be run to move the fat and the muscles. We took a lot of care to add little flares of wing movements. In the film, there are six different penguins and each one has a marking that helps you identify the character, and each behaves in different ways. When we’re animating a particular penguin we were animating to that character. Say, for Nimrod who is clumsy and bumps into things, he doesn’t walk the same way ever as any of the other penguins.

fxg: Can you talk about the Guggenheim sequence where the penguins slide down the spiral ramp?

Hollander: The premise here is that they’re chasing after Popper and they want to be with him. The penguins just miss him at an elevator, and then they turn around to walk down the spiral ramp and they run into a bucket of ice water and champagne. They trip the guy who spills the water. Captain, the lead penguin, goes through the thought process of, ‘Hey, I could slide down this thing!’ and off they go.

Well, you can’t have live penguins in the Guggenheim and as a matter of fact you can’t have water on the ground there either. There was actually a show going on there that we didn’t have rights for the artwork so we had to cover that up. We had to shoot as if the ground was wet and the penguins were running amuck. We rehearsed with stuffies and people running around with that so Mark could get the timing for the actors. We had a water slick that was a water sim, as well as of course all the penguins. Tracking the CG camera for the Guggenheim was also a challenge – there isn’t a right angle in there at all!

fxg: How did you tackle that water sim, because it seems like a much thinner volume of water than you might normally do?

Hollander: Our water tech group showed me some early sims of the water coming down the ramp. Originally, they had beautiful waves and things happening – it was just gorgeous but it had to only be thin. So we steered away from a pure volume simulation to more of a sheet of water. They then put in a secondary wave on top of a water surface and that’s where most of the detail came in for when the wings were touching the water or the body was causing V-shaped ripples behind the penguins.

fxg: You mentioned the stuffies. Can you talk about how they were used on set?

Hollander: Any time anyone holds a penguin in the movie they’re actually holding a very specific object that has the shape of a penguin, but it’s not the full penguin’s body. I weighted it down so that it was close to the real weight of a penguin. We had three of these things we called ‘rice penguins’ because they were filled with rice and were soft molded foam. It had no legs or head, but there was a copper wire going up for the eyeline, so anybody holding it could be looking directly at the eyes of the penguin. It had some markings on it which we used for tracking, and then we’d put in our penguins from there.

fxg: What about some of the more subtle scenes of just the penguins hanging around the house – what approach did you take to those?

Hollander: Well, there’s a shot of Popper’s estranged wife Amanda who comes to his place. Popper is in his living room and all the windows are open so everyone’s in blankets. His two kids are there, and all the penguins are there watching Charlie Chaplin. Amanda walks in carrying a penguin that got out of the house. So, the penguins are there watching the TV but they’re kind of reacting also to the conversation between Popper and Amanda.

It wasn’t an action-based scene – the difficult aspect of the shot was working out what we were going to make the penguins do during the scene were most of the action was between Popper and Amanda or the kids, but not the penguins themselves, and still tell the story that they are there and kind of understand what’s going in the film with the adults. We ended up approaching this with very nice subtle animation.

fxg: How emotive could you make the penguins, especially with their eyes which I’m guessing are smaller and harder to emote?

Hollander: As it turns out, there are hardly any facial muscle changes that are recognizable on a real penguin. The beak is also a solid object and doesn’t smile or frown. We played it as strict as it is. The eyes have some expression – we do lift up eyebrows, feathers in this case, sometimes, but we hardly used them. We stuck with the original idea that these were real penguins and if we were going to animate them and make them sentinel beings, if you will, that we would do it with movements that were natural to a penguin.

So luckily penguins do move around quite a lot. They can squish their head down into their body and chest. They can pull their wings back behind them. They can do crazy twists with their necks and extend them. And they can look at you in the eye or look off to the side of you. So we played it that way. We had to understand penguin movements and make a language to follow.

fxg: How did you orchestrate that language for each of the penguins?

Hollander: Well, there’s one scene where Popper walks into his apartment after discovering one of the penguins on the toilet spinning around. He comes out of that bathroom and it’s a wide shot and each penguin is basically doing what their character represents. Loudy is honking, Bitey is pulling a blanket off the couch, Stinky is riding a chair, Nimrod is sliding down the stairs an Lovey is throwing bits of banana peel. It’s a great scene that shows each penguin doing something which represented their character. When we shot it, nothing was there and we used quite basic interaction. So someone just pushed the chair for Stinky – it wasn’t even on a mono-filament rig. Jim Carrey did an amazing job of actually imagining all these guys were there. It’s almost kind of inhuman how he does it. Mark worked with him to show what he think might happen – he does it so consistently.

fxg: I think you said the film was shot with the ARRI Alexa? How did you find that process?

Hollander: The camera was really amazing. It’s a full aperture camera with 800 ASA. That means we had two stops on everything. For the DP, Florian Ballhaus, he was really interested in it because of two reasons. Firstly, it was a great camera, but also we had to refrigerate our set for the live penguins. When you go to light a set, lights produce heat, so not only did we have to refrigerate the set but also pull heat away from the light. Because we could down two stops on the lights, we had that much less refrigeration required. It is just amazing the kind of stuff that you see with that camera – especially outside at night. There are some sequences that needed to be shot at 60 and 72 frames per second, and at the time the camera did not shoot those speeds, so we shot those on film. So there were just four or five shots on film, which was a good test also just to put film against ALEXA and make it look seamless.

All images © 2011 Twentieth Century Fox Film Corporation. All rights reserved.