Comic effects: Men In Black 3

Men In Black 3, Barry Sonnenfeld’s follow-up to his 1997 and 2002 films, features a host of aliens, time travel effects, street chases and even a fight scene atop an Apollo 11 rocket. We take a look at just some of these key sequences from the film with the team at Sony Pictures Imageworks, as well as the previs and postvis work by The Third Floor. fxinsider members have special access to extended video interviews below.

Visual effects supervisors Ken Ralston and Jay Redd oversaw a total of 1214 VFX shots for MIB3, the bulk completed by Imageworks, with additional work by Method Studios, Prime Focus, EFIlm and Cantina Creative. The film is a 3D converted picture – supervised by Corey Turner – and carried out mostly by Prime Focus. MIB3 was shot predominantly on film although some night sequences, including plates for the monocycle chase, relied on the ARRI Alexa.

Watch a Promo for this week’s fxguidetv MIB3 episode.

Watch the full fxguidetv episode here, which features five in-depth interviews with Imageworks’ artists.

Boris the Animal and the weasel

One of those realistic, but fun, aspects of the movie is exemplified in Boris the Animal (Jemaine Clement). Having been imprisoned – on the moon, of course – by Agent K (Tommy Lee Jones) in the 1960s, Boris escapes with the help of his creature friend, nicknamed by Imageworks artists as ‘the Weasel’. With the ability to fire lethal (and replenishing) spikes as daggers, the Weasel is Boris’ ultimate companion and in fact resides inside his keeper’s hand.

Original plate.
Rotomation and wireframe.
Animation.
Final shot.

Imageworks modeled the Weasel based on a design and maquette from Rick Baker – who also delivered numerous other aliens and prosthetics for the film. Animation-wise, artists had to communicate a symbiotic relationship with Boris, and suggest a sense of longing for another Weasel from Boris’ missing hand which had been shot off by Agent K. “We had a couple of fun shots where we could play with the affection,” recalls animation supervisor Spencer Cook. “There was one where it rubs against his face – I think they even put in a purring sound.”

Watch VFX supe Jay Redd discuss both the retro and new approach to the aliens in MIB3.

Wu’s Restaurant
Animation pass.
Slime sim.
Final shot.

K and J enter Wu’s Restaurant in China Town, a known alien hang-out, where they encounter a giant fish escaped from its tank. It attacks J and the ensuing fish fight wreaks havoc on the restaurant and ends up on the street. To film the sequence, Will Smith and his stunt double were pulled on wires and occasionally interacted with a foam stand-in. The fish, and sometimes Smith, were digital Imageworks creations.

Artists replaced much of the live action set with the characters tumbling, as well pieces of furniture and debris, smashing windows and debris. “On set there was an attempt by the prop department to make a huge foam thing,” says Cook, “like a dome with flippers on it, but there really weren’t many shots that came back to us that had anything in it. The foam was so soft that it was as if Will was pushing against nothing.”

“We’d look for places we could make the switch between live action and CG,” adds Steiner. “In the case of the one shot that has Will coming flying out of this kitchen door, Will actually flew through frame and wiped over the top of a lot of this stuff, and we could make switch, but then he actually hit a table and the table flip up and we could switch out. So at the beginning of the plate we’d have plate elements and then at the end of the shot we’d have the fish element.”

The fish went through a number of design evolutions before a 30 foot prehistoric creature – a Dunkleosteus – was referenced. “The thing I always wanted out of the fish was that it should be dangerous,” says Cook. “It should be a monster and have expressive eyes. We had a couple of shots in there of having the fish react to what Will is saying, and what the owner of the restaurant is saying. We can see him in the tank and see his brows going up – pretty subtle.”

Back in time
Splitscreen showing the live action Chrysler Building set (on left) and the final shot.
Agent J readies for his time jump.

In order to save Agent K from being killed by Boris in 1969, Agent J (Will Smith) must jump back in time – literally. J makes a leap off New York’s Chrysler Building with a time device that must be activated right before he slams into the pavement. To film the sequence – which follows J down the side of the building as he drops the device and than transports to the past – features a digital New York skyline, associated cars, pedestrians and movement and even glimpses of alien invaders brought on by an alternate reality.

Imageworks managed a data set of buildings and assets for the city and controlled lighting throughout the sequence. “Essentially it’s a lot of reconstruction of the city at that point,” explains CG supervisor Brian Steiner. “Most of the sets are purely digital, except for the area they’re standing on. They had a little piece with a falcon, and they had a few walls. The rest of the building was digitally extended, and then all the surround buildings were CG with matte paintings for the background skies.”

“On the top of the Chrysler where Will is standing we did model a lot of the interiors so you could see in see desks and pictures frames,” adds Steiner. “But as soon as we get away from that building top area, then the interiors are just flat cards – we have shaders that cheat perspective. We shoot a 360 of room interiors. We have a way of randomly throwing in room interiors – if you’re looking at a certain angle you’ll see one wall and you move to another side of that window you’ll start to see the other side of the room. The shader is actually intelligent in that it knows what direction you’re seeing that room from and tries to cheat the proper perspective.”

Imageworks transitioned between live action and digi Will Smith at points in the fall. “There’s a lot of digi-double stuff in some of the bigger shots – many seconds long – following Will down the building,” says Cook. “He goes past us and we’re dropping with him and going around him as he’s trying to grab this time device. The first thought was that they would actually use Will and partially transition to a digital guy. But we had all this performance – there were some specifics we took out of that and used that in the animation.

Uncanny alley

Searching for Boris in 1969, Agent J and a younger Agent K encounter aliens at a bowling alley. One of the aliens, Tony, has a removable head that J uses as a bowling ball. The scene was filmed with a stand-in soccer ball painted blue and matching smiley face. Imageworks then used the actor’s on-set performance to re-create the head in the palm of the agents’ hands. “We animated the face completely,” says Cook. “It was fun to design the head which had a human look but it’s a bowling ball. The head is only partially attached to the body and the body is still gesturing while Will is carrying the head around – it’s so absurd but funny.”

Original plate.
Animation pass.
Final shot.
Shea Stadium

An alien with precognitive powers, Griffin, leads K and J to Shea Stadium – a location that no longer exists. Imageworks re-created the stadium from archival images and added it to footage shot on a sound stage in New York. “One interesting dilemma was that once we build the stadium and plop our cameras in and set up virtual shots, Barry would come along and realize it isn’t the angle that he was hoping the stadium seats would be at,” recalls digital effects supervisor Ken Hahn. “For the majority of action, which takes place on the upper deck, he actually asked us to cantilever the seats a little differently, so that he could get the composition of the stands and the playing fields for storytelling purposes.”

Original plate.
Final shot.
Monocycle chase

Boris kidnaps Griffin on a motorcycle from the stadium, before Agents J and K pursue them on 1969-era MIB monocycles through Brooklyn and Queens. Production shot most of the backgrounds first before filming foregrounds of the actors on bluescreen, something Redd says “worked great for a lot of it, doesn’t work great for a lot of it.”

Watch part of the monocycle chase.

Plates were filmed with the ARRI Alexa allowing a significant amount of detail for the nighttime scenes, which required a mix between live action and digital enhancements. “Most of the buildings still exist there today, but we did change some awnings, street signs that were more period pieces,” explains Hahn. “The vehicles had to be built for 1969 with different paint jobs. We were also intercutting live action and digital shots – we’d transition from live action background plate with a set dressing and then go to a CG shot afterwards and back.”

Apollo 11 rocket launch
Final shot from the rocket launch.

The film’s climatic scene is a fight between J, K and Boris atop the Apollo 11 rocket gantry at Cape Canaveral (K must place a special protective shield generator for the Earth on the rocket before it launches). “They built a small set piece of the gantry,” says Hahn, “a section about two feet tall with a gantry arm. They built a small section of the upper catwalk and the top section of the rocket. Everything else was shot against bluescreen or it was shot and we had to roto out the appropriate pieces.” The Cape Canaveral location also required digital marshes, ocean and sky replacements by Imageworks – from panoramic photographs taken in the area that is now Kennedy Space Center – plus of course the rocket lift-off.

Also in that sequence, more is seen of the villain Boris. “In the past we’d seen the weasel out of his hands,” says CG supervisor Darren Lurie, “but when he gets really angry he has the same kind of claws around his goggles and they flare up and they’re kind of a character onto themselves. One of the cool things was that once we went digital with that area, and once it opened up, we got to reveal all the gooey sticky insides that were right back into his head.”

Watch digital effects supervisor Ken Hahn discuss re-creating sets for Cape Canaveral and Shea Stadium.

Previs and postvis

Helping to flesh out many of the sequences in previs and postvis form was The Third Floor, which worked on the prison break, the fish fight at Wu’s, Shea Stadium and the monocycle chase. “One of the final sequences that closely echoes the previs is the Chrysler Building time jump,” says Third Floor previs supervisor Shannon Justison. “Working with Barry Sonnenfeld, Will Smith, Ken Ralston, and Jay Redd, we began with brainstorming ideas for the different time periods that Agent J would fall through on his way back to 1969. We then built several different CG sets representing the history of Manhattan, all the way from the Cretaceous Period to the V-Day ticker-tape parade, and animated our digital J jumping through time. Designing seamless transitions between the time periods was one of the biggest challenges.”

Previs by The Third Floor for the monocycle chase.

“We didn’t want J to ever pass through the ground,” she adds, “but we also didn’t want to rely on cutting back to a close-up for every transition, not only because it’s not as much fun, but also because quick cuts with significant depth changes are very distracting in stereo. Additionally, we converted the previs into stereo to play with the depth and ensure that the overall design both supported and maximized the 3D opportunities.”

With many sequences shot on bluescreen, postvis became an important element in helping the editorial and effects teams. “Postvis was useful in helping the director, the visual effects supervisors and the editors evaluate and make choices about how the story would evolve,” says The Third Floor’s post-visualization supervisor Scott Hankel. “We also did a lot of previs during the postvis stage on the Cape Canaveral sequence so the filmmakers could explore and plan shots for what was happening when and where people were during the fight.”

“The Cape Canaveral sequence was shot on the stage at Kaufman-Astoria,” adds Justison. “There was only room to build the upper portion the structure, but at one point J and Boris 2012 jump from the very top gantry and fall through time towards the launch pad. Starting from a model of the launch tower from art department, we built a digital Cape Canaveral and surrounding environs. We then worked on melding the blue-screen footage of the actors with the digital set.”

“Most of the fall was shot with the actors suspended but static on the stage, so the actual effect of the motion was accomplished by the camera falling through the CG set. From a pure physics standpoint, falling off the top of the tower would only take a few seconds, so stretching time also required us to scale the actual tower. While we could easily add extra stories on to the tower, we had to carefully tweak the scale of the rocket to avoid obvious distortions. There were also several extreme camera moves in this sequence, such at the establishing spiral boom-down past the Saturn 5 rocket that meets up with Colonel Williams and our Men in Black at the launch pad, and a massive push-in from a superwide of the tower into J and K riding up the elevator. The former involved seamlessly hooking up our CG camera and digital set to a practical camera move on the stage set, while the latter involved finding an ideal point to switch from the J and K digital doubles riding up the CG elevator to the footage of the actors in a static elevator set, all the while maintaining the illusion that they were moving upwards in space.”

Dimensionalizing and delivering MIB3

Before shooting, Sonnenfeld, DP Bill Pope, stereographer Corey Turner and VFX supes Ken Ralston and Jay Redd collaborated on stereo tests, shooting with a number of stereo rigs and also shooting on film and digital and post-converting the shots. Eventually, they settled on 35mm and post-convertion (handled predominantly by Prime Focus), a process that enabled Sonnenfeld to establish depth down the track but also shoot and compose scenes with stereo in mind. “Because of Barry’s shooting style [the director famously favors a 21mm lens], even in 2D his shots look very deep,” notes Redd. “By the time you get on a 21mm lens and your ears are pushed back and your face comes forward, the background’s nice and soft, it’s got a wonderful depth to it. The stereo adds another layer of whimsy and another layer of comedy. There’s a lot of epic situations in terms of large scale sets.”

Director Barry Sonnenfeld and Will Smith on set.

“Because this was shot on motion picture film,” adds Ken Hahn, “we also had to work out how to deal with the grain in conversion. We had worked with Relliance Mediaworks on trying to come up with an elegant de-graining solution but still not altering the plates or degrade or soften them. There are a few plates that had some issues but for those we took advantage of the new wavelet technologies that Nuke has for de-graining plates. Even though Barry and Bill Pope liked film, they actually preferred a cleaner image, a less grainy structure – so they asked for to only retain back about 25% of what the original film grain was.”

Imageworks’ tools of choice were Maya for animation, Katana for lighting, Mari for texture painting, Houdini for effects, Arnold for rendering and Nuke for compositing. The final delivery was in 2K as a 185 centered mask image, protected to 1:6:6 for a HD release. For VFX supe Jay Redd, whose passion for space led him originally into visual effects and served him well in particular on the film’s Cape Canaveral scenes, MIB3 was the opportunity to work on a ‘whimsical and whacky’ franchise established by the first films. “It’s kind of a live action cartoon,” says Redd, “in that we follow physics, we follow reality, but stylize those things a little bit. You sit with Barry and you realize he’s taking the sense of humour seriously. There’s a fine line in his movies of feeling realistic but fun.”

All images and clips copyright © 2012 Sony Pictures Entertainment.