The Muppets: It’s time to raise the curtain

Renowned puppeteer Jim Henson created The Muppets more than half a century ago. Since then they’ve had their own show, movie and several other spin-offs. Now the Muppets star in a new film directed by James Bobin in which they have to save their precious theater. Look Effects visual effects supervisor Max Ivins recounts his studio’s joy at working on 350 shots for the latest Muppet adventure.

– Watch a montage clip from The Muppets, featuring effects work by Look.

fxg: 350 shots sounds like a lot of visual effects for a film that people probably don’t think contains any at all?

Ivins: Yeah it is, and I think there’s probably several hundred more done as rod removals and other things. We were the lead facility but they also had an in-house crew on it. When they approached us at first, I was thinking, ‘The Muppets? Really? That doesn’t sound like a lot of effects. What are we going to do – put legs on them? We’re not doing CG Muppets, right? That would be weird.’

fxg: So how did you approach that kind of visual effects work?

A bluescreen comp from the montage sequence.

Ivins: It turned out that what we ended up doing was using modern keying and computer compositing effects to allow the puppeteers to do a lot more than they’ve done in the past. There were a lot of shots in the movie where there were four puppeteers doing one puppet, because they have legs. Pretty much all of those shots were done on a full bluescreen stage with puppeteers in full blue outfits – it was like the ‘blue men group’ puppet. There were some pretty wild shots of puppets with all these guys behind them.

Ivins: The key thing for us, really, was to stay out of the way – to be like the puppeteers – do a bunch of stuff but try to make it look like we were never there. That’s challenging when you’ve got about 30 different furry Muppets. That’s one thing I learned about Muppets – they’re all really, really furry, which is tricky when you’re doing bluescreen work. It was more of an artistic endeavor to make everything feel like a grounded reality of things you could touch and feel, instead of having any CG slickness to the whole thing. That was also the director and supervisor’s mandate. For example, we did some driving comps, which were pretty traditional bluescreens, except that there were a bunch of Muppets in the car, who were all furry! So there were all these fine hair details we needed keyed nicely. They were actually the most finely combed and analyzed driving shots I’ve ever worked on – making sure they looked like they were shot on camera and not revealed as composites.

fxg: Can you talk about the shoot generally?

Ivins: Janet Muswell was the overall VFX supervisor and producer. She wanted us to come on set on the bigger shots and collaborate, which was fantastic and a lot of fun. Janet did a fantastic job of giving the puppeteers direction and what they needed, as far as playback and things, so that they could do the crazy things with the puppets. For example, they did a miniature Beaker running around on a platform once he gets shrunken by the professor. They built a merry-go-round rig with a raised platform in the middle that would allow four puppeteers to sit on this thing while they pushed it and it was spinning. This meant they could make Beaker run around in a little circle when he was tiny. They went to great lengths to give the puppeteers what they needed with big rigs and what-not to make things work.

Kermit on bluescreen.
Final shot.

They also did a shot where they pulled a Rolls Royce out of a lake, and we turned it into Cannes Beach, a big matte painting in the background. On a stage they made a ‘Flinstones-reveal’ – a dolly track with a shade cover so it would look like they were underneath the top of the car. They put the actors and the Muppets inside of this rig and pushed it along to simulate the same motion of the car being pulled out. We would then extract them and put them inside the car.

There was a lot of pretty big production effects shots that were, on our end, pretty straightforward tracking and compositing once we had all the pieces and polishing. But the big effort in terms of effects was building the rigs so that the puppeteers could do things that they couldn’t do with puppets in the past.

fxg: The film was shot digitally on the RED – how did you deal with that workflow?

Ivins: You know, the production was fantastic on that side. It was shot on the RED ONE MX and the EPIC. But actually I never really had to take note of the camera or format because it did not fall on us at all. It was really one of the first RED projects we’ve had come through here that had no compression artifacts, because others in the past have. This went really smoothly. They were really happy with how it came out looking.

fxg: Did the production ever want to keep some hints of the puppeteering in, like rods, for instance?

Ivins: There was a concerted effort to wipe out just about every single rod possible – although they might not have been able to. They might still be in the shot but they don’t look like a rod anymore. They were finding rods – and puppeteers’ heads – up until the end! We were doing shots getting into the 20th version and then suddenly someone would notice a puppeteers’ head! There would be 40 Muppets on the screen at one time, shot in say four passes, because literally we didn’t have puppeteers to do all of them in one go. So you get this ocean of Muppets and then between two of them there’s this round brown shape – ‘Oh no!’ – some guy’s head.

fxg: Bluescreen composites and traveling shots can be very traditional work, but you mentioned how furry the Muppets were – did that mean you had a particular approach to the compositing?

Ivins: Funnily enough, keying technology has progressed in little tiny baby steps over the past ten years. There are more options and several different packages. Some of them are better at certain things, but it still comes down to devising a key for the general Muppet and then making specific keys for specific areas on the puppet. It comes down to garbage matting off certain areas, especially if they’re lit brightly on one side and not so much on the other.

Walter being puppeteered bluescreen.
Final shot.

In the shots where there were puppeteers behind the puppets. That is an extremely dubious proposition – you do not get something you can key out of, so you end up with a lot of roto and specific keying in areas to try and maintain the integrity of the puppet. A lot of it was using small tricks of the trade and sleight of hand in restoring pieces and tracking things back in. I think the Muppets had a real labor of love – for example, we’d look at the underside of Fozzie’s arm and say that doesn’t look Fozzie enough at all and you have to fix that.

Also, Janet Muswell has been a digital compositor in the past so we couldn’t really get anything past her. She’d say, ‘Well, my pet peeve is that gray de-saturated edge you get…’ and I’d be like, ‘Oh no, really!’. So we’d have to do an edge detection and restore the saturations. Almost every puppet ended up with its own procedure to restore the edges, because you do the color suppression and it sometimes messes with the color. There was really no way around it other than brute force compositing.

fxg: So, Kermit is green, obviously. Did that dictate whether you used a blue or greenscreen?

Ivins: Exactly, keying green wasn’t really viable because you can’t pull him at all, and most of the blue characters – strangely enough they were a lot easier to key against blue than the green characters against green. I don’t know if it was the way we were approaching it, but after a lot of testing we ended up going with blue. It’s also a lot easier to sit on a bluescreen stage than on green. There’s something really intense about big greenscreen stages. The blue is definitely easier to take on the eyes.

fxg: There’s a fun scene with Animal and the angel and devil. Can you talk about that?

Ivins: One of my favorite shots was Animal’s drumming dilemma. They’re trying to get the band back together and they find Animal in an anger management session. Later in the show, he’s presented with his own drumsticks, and this angel and devil appear over his right and left shoulder and start talking to him. They’re a little angel and devil version of him shot on bluescreen with puppeteers which we removed. We added a little fireball for the devil and a white pyro element as they appear. The angel stood on a little cloud and the devil on a fireball, and they talk into his ear. We took a completely 2D approach for this.

Original El Capitan Theater plate.
Final shot.

fxg: There’s a crowd shot outside the El Capitan Theater doubling for the Muppet Theater – how did you do that?

Ivins: We had to create an active crowd and wrote our own particle system and collision detection set-up in Maya with a bunch of small characters that were walking or jumping up and down. It was a fun exercise because we were writing it ourselves instead of opening up a package. We found the most interesting thing about that was that the difference between an excited, active crowd that you can’t tell is just not a bunch of noisy models is actually pretty small. The active, happy crowd – you just notch it up about 15 per cent on the activity level and all of a sudden it looks like a riot! It would look like they were killing each other. So we toned it down a bit for the crowd.

fxg: What were some of the other challenging shots you had to work on?

Ivins: We did the show opening with more than 40 Muppets, some fireworks for the finale, a vacuum cleaner that attacks Beaker and the Muppets in picture frames. The most challenging thing we did was probably the car coming out of the water. It wasn’t shot motion control – it had the right angle on the characters but tracking to the car and getting the windows and water surfaces over the back windows was tricky. We had to re-position the characters just slightly as well and make sure the parallax between the matte painting and the background and the beach in the foreground was right. The lens also had a giant reverse barrel distortion on it, and by the end of the shot we were like, ‘Nobody touch it!’.

fxg: Can you tell me more about the picture frame sequence?

Ivins: That was Kermit’s ‘pictures in my mind’ song when he’s thinking about his friends, and the portraits on the wall come to life, from the portrait to the puppet. We made portraits out of the puppets and then tracked them in and had them turn back into the puppet. It sounds really easy in concept, but we had to do it in a way that doesn’t look cheesy. It had to be subtle and have the right dimensionality – how do you go from flat to three-dimensional? There were three of these shots in a 360 degree move.

We had Muppets inside the squares where the paintings would go, in little dioramas with bluescreen behind them. There were some props but we put the backgrounds in. It was a mixture of making the 2D painting and turning it into this three-dimensional background we built. Then it just got massaged. We had them move out of the surface and leave the whole background kind of flat – a drop shadow would move down behind them. So the painting would recede and they move forward. Then the painting turns into a three-dimensional environment behind them, but it’s really subtle. The whole thing was to make it feel natural and organic, but not have you go, ‘Oh, that was a strange effect.’ It has that dichotomy dilemma, making a complete fantasy look completely realistic. Blowing stuff up is actually a much easier assignment!

fxg: What were the tools you were using?

Ivins: Nuke was our primary package. We did some shots in After Effects. There’s one shot where Walter gets electrocuted on a fence which is a funny sequence. After Effects gave us a platform to try a bunch of stuff with electricity plug-ins. Then we went to Sparks for the end shot.

– Watch Walter get shocked in this clip.


fxg: It sounds like a pretty great film to be involved with as both a Muppets fan but also in terms of doing invisible effects.

Ivins: It definitely was. You know, fundamentally, puppeteering is the oldest form of cinema. I mean, shadow puppets from 2000 years ago in China is probably as far back as cinematic projection goes. For us as visual effects people, puppets are the legacy of visual effects. At least for me, puppeteering is the predecessor. We do it with computers now, but it’s kind of the same thing – the father of animation in a way. And it’s still viable. Also, there’s something irresistible about the Muppets. The greatest thing about this movie is not the visual effects – we try and stay out of the way – the greatest thing is the Muppets. Our sole thing was to let the puppeteers – the actors – who are the Muppets, do their thing.

Listen out for more on the RED workflow from Lightiron in an upcoming RC podcast here at fxguide soon.

Images and clips copyright © 2011 Walt Disney Pictures. All rights reserved.