With Captain America: The Winter Soldier, Marvel advances the story of WWII super-soldier Steve Rogers (Chris Evans), now a key player with S.H.I.E.L.D. but also struggling to come to terms with his modern-day transposition.
Directors Anthony and Joe Russo created a number of challenging settings and sequences that introduced a new character – the Winter Soldier (Sebastian Stan). Many of these sequences would call on the innovative services of visual effects supervisor Dan Deleeuw and visual effects producer Jen Underdahl and their teams of effects artists, including locating S.H.I.E.L.D.’s new headquarters on the Potomac River, crafting a digital make-up solution to depict ‘Old Peggy’, delivering a digital arm for the Winter Soldier, and bringing to life the flying Falcon character and orchestrating the flight – and destruction – of three massive new Helicarriers.
In this article, we explore several of the major shots and sequences in the film, focusing on the visual effects by ILM, Scanline, Lola VFX, Luma Pictures, Whiskytree, The Embassy and the previs completed by Proof.
Above: watch a breakdown of ILM’s visual effects for the Helicarrier crash that fxguide produced with our media partners at WIRED.
Taking back the Lemurian Star
Action: A S.H.I.E.L.D. team including Rogers and Natasha Romanoff (Scarlett Johansson) is called to save an agency satellite launch vessel – the Lemurian Star – in the Indian ocean from Algerian pirates.
Innovation: Scanline combined two real Sea Launch vessels – the Commander and Odyssey – into one for the S.H.I.E.L.D. ship.
The nighttime sequence begins on board a S.H.I.E.L.D. Quinjet before Rogers jumps out sans-parachute and the rest of the team follow. Scanline delivered shots of a CG jet pushing through clouds, with a digital Flowline-created ocean and CG Lemurian Star below. Rogers’ jump also made use of a Captain America digi-double.
– Watch part of the Lemurian Star sequence.
For the Star itself, production gathered reference of the Commander and Odyssey while they were docked at Long Beach. The vessels were LIDAR’d both from exterior views and from on board, as well as being photographed extensively. Scanline modeled and textured the Star as a CG ship that would be used for set extensions and, ultimately, for a helicopter fly-by that was based on an aerial plate.
“Because we had the helicopter plates and textures, it went together pretty easily at the end,” says Scanline visual effects supervisor Bryan Grill. “We’d already been building the Star and then when we decided to do it all-CG we had a lot of material to work with. “So then we would just keep updating it, and making sure we had the right ocean and sky backgrounds working with it.”
Action: S.H.I.E.L.D.’s new headquarters are revealed to be in a massive building known as the Triskelion located on Theodore Roosevelt Island on the Potomac River.
Innovation: The visual effects team created views of the Triskelion and surrounding areas of Washington D.C. from live action plate photography and CG-created environments.
“I’d shot in D.C. before and for obvious reasons it’s a little restrictive in terms of what you can do there,” says DeLeeuw. “I’d known there was a no-fly zone surrounding the Mall, the White House and the Capital. Outside of that you could fly. But that no-fly zone actually ended at the Potomac. What I didn’t know is that the portion of the Potomac where the Triskelion was going to lie is in the flight path of Ronald Reagan Airport. Luckily, our helicopter pilot had worked there quite a bit and could schedule his passes between flights. He was able to call the airport and basically say, ‘Hey we’re going up now between the jets taking off. So we had these little windows.”
With an Alexa fitted to the helicopter, members of the VFX crew shot moving plates and still photographs of the area which were then tiled together. “Even with the background plates we used directly,” says DeLeeuw, “based on the size of the Helicarriers, they were actually too long for the width of the river. So we had to widen them anyway, just to make sure the Helicarriers could fit out of the river!”
The Helicarriers are first revealed as Rogers and S.H.I.E.L.D. director Nick Fury (Samuel L. Jackson) visit the underground hangar. ILM artists built up the Helicarriers based on concept art and a library of parts – almost like an assembly kit. “There was so much geometry and detail that we couldn’t even load in three of them for the more complex scenes,” recalls ILM visual effects supervisor Russell Earl. “On top of that, the Helicarrier bay itself was fully tricked out. Originally we had some concept art and it was more of a walk and talk with the carriers behind, but as the film progressed everything got bigger and bigger.”
ILM dealt with the imagery via its 3ds Max and V-Ray pipeline, which also involved texturing in MARI. “I would have loved to just keep building this thing until someone fired me,” comments ILM hard surface supervisor Bruce Holcomb. “In some shots I went into dailies and I literally said, ‘Wow’, out loud, because some of the shots they have it in it is just so real there for me.”
Important additional elements to the Helicarriers in this film were the Phalanx-type guns capable of precision targeting and rapid firing. “We had to go back and do some research on WWII and look at some of the biggest guns that the Nazis had developed in secret,” says Holcomb.
The studio’s in-house tool Zeno was also relied upon for carrying the project through the studio. “When we hit the caching part of the pipeline we go into Zeno, a way to cache it out to Alembic format,” explains ILM CG supervisor Johan Thorngren. “It takes all the textures to EXRs that we’re using. Then we are picking it up in the finishing departments – the generalist group or TD group.”
“We’re using Katana and a pre-release of V-Ray to do all the TD lighting,” continues Thorngren. “On the generalists side we take the same caches and using Max with V-Ray to do say the Triskelion or the Helicarriers in the carrier bay. We can take them into the generalist department to up-res the textures. We can take these assets and use them to project more textures on them.”
Through doors and walls
Action: Rogers chases the Winter Soldier onto a rooftop for their first encounter.
Innovation: Although a raft of visual effects were necessary for the Winter Soldier’s metallic arm, Cap’s shield and other environment work, the chase scene made heavy use of practical effects work.
“We had Dan Sudick as special effects supervisor and Thomas Harper as the stunt coordinator on this film and they were just awesome,” says Deleeuw. “From the very beginning, I was a fan of doing a lot of things practically. I went to them and said, ‘Hey, what can we do to help you so we don’t have to do a lot of visual effects? Let’s do it practically.’”
For the chase sequence, Rogers runs ‘through’ doors in pursuit of the Winter Solider. “SFX would rig the doors practically,” explains Deleeuw, “and when Cap is running through the hallway and parkours up the wall and jumps across and slams through the wall, they said, ‘Let’s just build footholds on the side of the wall that we paint out later.’ So instead of doing so much digitally we actually have a real person running through and break off the doors off the hinges. We obviously augmented some chipping and pieces of wood for the door jams but for the most part it’s all stunts and physical effects blasting through the hallways.”
The sequence also enabled Rogers as Captain America to be depicted with an ‘updated fighting style’. “If you go back to the first Captain America film it’s more classic fighting,” notes Deleeuw, “but now he’s more of a martial arts expert. The idea is that Cap is always better than the best person on the planet at something. So if someone was a certain speed, Cap was always 10 per cent better than that.”
“It’s great,” adds Deleeuw, “because it lends itself to the strength of Cap and how fast Cap is, and doesn’t take the time to open a door – he just goes through it!”
From young to old Peggy
Action: Captain America visits his now elderly former love interest Peggy Carter (Hayley Atwell).
Innovation: The final shots of ‘Old Peggy’ were realized by Lola VFX by transposing the facial features of an elderly actress onto the face of Atwell who had performed her lines with no make-up and only a few tracking markers.
Lola had, of course, made a significant contribution to the first Captain America film with their ‘Skinny Steve’ visual effects, which are also part of The Winter Soldier in a couple of flashback sequences (see below). But here the studio faced a new challenge in digital makeup and, in fact, one that differed from the intended approach.
The initial plan was to shoot Atwell with no make-up while talking to Evans, then do a further shoot with old age prosthetic make-up applied to Atwell. Lola would then carry out a digital facial re-projection of the ‘old age make-up Atwell’ onto her original body.
And this is exactly how the shooting took place. The first shoot with Evans and Atwell employed no make-up and just six tracking markers on Atwell’s face. The team then began testing old-age make-up prosthetics. When Atwell’s performance under the prosthetics proved somewhat restrictive, the filmmakers continued make-up tests with an actress who resembled the young Atwell.
“We did three tests with the lookalike,” says Lola visual effects supervisor Edson Williams, “and each test got increasingly better than the prior test. With each iteration of the test, the make-up artist would make the appliances thinner and thinner and by the end was some of the best prosthetics I’d seen. But it still wasn’t quite what we were looking for.”
Meanwhile, production also cast an elderly actress “with the intention of seeing what real skin looked like,” outlines Williams. “Translucency, how does the skin sag, how does it move as she talks?”
It was at this point, however, that Williams had what he described as the ‘Hail Mary’ idea. “I said, ‘Let’s try something that we’ve never done before.’ I wanted to take the performance of the elderly woman that we had shot in a rig with eight cameras and project the skin onto the original Hayley footage that had been shot on set.”
With a week’s turnaround, Lola performed a test. “We took a still frame of the skin and tracked it onto the original photography, and it looked so good,” says Williams. “The way we did it – it was amazing – you could still see Hayley, her eyes, her mouth, her underlying structure, but we just lifted the creases and cracks and age from the elderly woman and transposed it onto Hayley’s young face.”
Based on the success of this initial test, Lola decided to re-shoot the elderly woman at a higher frame rate and with more precision. “We brought her back to Lola and re-shot her with this new technique in mind,” says Williams. “She performed all the lines and all the emotion of the original Peggy footage. The first time she was just delivering her lines, but the second time she met the cadence and the performance of Hayley.”
Lola then took that new footage and time-warped it to line up more appropriately to Hayley’s original performance, so that the words would hit on very similar frames. “Then we isolated the skin of the old woman and re-applied that onto Hayley,” explains Williams. “And we had to do that for every shot – about 15 in the end. We matched the lighting, the timing and performance and literally stole the creases and cracks and put them onto Hayley.”
Since Atwell’s original performance had only utilized a limited number of tracking dots, Lola tracked the actress’ facial movements at the pore level – ultimately relying on around 75 dots. “Then we matched the same 75 tracking dots to the elderly face,” says Williams. “We used a deform node in the Flame to snap the two together. So we literally used the performance of Hayley to drive the performance of the elderly woman.”
Once the skin was tracked, artists lined up other facial features such as the lips and cheeks. “We had to bring back some of the subtle moments of Hayley’s performance,” notes Williams. “Actually, 75 points weren’t enough. We could have used 150 points had we had those points. So for around the corner of the mouth, around the nostrils, some of the forehead wrinkles, we came back and fine-tuned some of the elderly skin to match her to Hayley’s original performance.”
The technique Lola used for the work involved animated points on blend shapes inside Flame with two facial meshes. “We found a trick where we could snap points,” describes Williams. “You have a mesh on the original photography and you have a mesh that lined up to the old age woman mesh. So a corner of the mouth goes to the corner of the mouth, nose to nose, et cetera. So you have these tracking dots that you replicate on the elderly person and then you literally snap the two points together. And it reforms itself and it corrects a lot of differences in the facial structure, so when you snap it the mouth now becomes correct, because it’s snapping to Hayley.”
Williams says using just a still frame from the elderly actress in the test gave promising results, but that moving footage was of course even more precise. “What the running footage gets you is all the nuances – as someone says a word, the corners of their mouth will pucker just right and the creases on the top of their lip will form. You have to get that eye expression, wrinkles and creases on the forehead that are just natural.”
“The running footage gave us animated cracks,” adds Williams, “Cracks that really changed shape. The shadows and luminosity of the cracks would change as the dialogue was being delivered. The technique works with a still frame, it just looks better with running footage.”
Williams says he was incredibly happy to be able to achieve a photorealistic effects shot with “a technique we hadn’t planned on when we started.” He also notes that from an editorial point of view, it meant that Atwell’s performance as chosen by the editor could ‘remain’ in the scene and not become something different after visual effects. “The editor might say, ‘OK, I don’t want to lose Hayley’s eyes’. So with this technique we were literally able to maintain her performance that they obtained and captured on the initial photography.”
Action: Rogers is attacked in an elevator as he attempts to leave the Triskelion.
Innovation: Scanline composited greenscreen photography with Washington D.C. backgrounds and glass elements for the fight and then worked on additional shots for Cap’s later dramatic escape from the building.
A locked-off greenscreen elevator set was filmed at Marvel Studios. Here, scenes were captured with a mix of stunt glass in and not in place. This meant that Scanline had reference for the immediate glass of the elevator when implementing the backgrounds. “It was still a challenge, though,” outlines Bryan Grill, “because there was a double plane; glass in the elevator and outside glass attached to the building structure. So we had to figure out how the reflections and light would play with two separate panes of glass.”
Scanline built the entire environment in NUKE, incorporating 3D renders of parts of the elevator, reflections from witness cams and D.C. background plates. Wider views of the elevator made use of ILM’s fully CG Triskelion. “We gave them camera angles based on our sequence where we would see parts of the building and they rendered out high res single frames,” explains Grill. “On top of that we would do a paint or render of CG trees and we would do a big pan and tile setup because of the fight – there were a lot of quick cuts and we were able to get away with a lot of projections.”
Rogers jumps out of the elevator – through two panes of glass – and falling onto a glass atrium. “When he jumps out,” describes Grill, “everything is completely fabricated when he’s out of the elevator. We had to take the Triskelion and do CG renders of it up close too. It was concrete and you’d think that would be simple, but when you’re that close, there’s certain qualities that makes things look real.”
Dueling with a Quinjet
Action: Rogers encounters a S.H.I.E.L.D. Quinjet that blocks his exit on the bridge.
Innovation: Scanline transposed footage filmed in Cleveland into a composite view of D.C. with the Triskelion, and delivered a CG Quinjet that Rogers ultimately brings down.
Scanline had been part of Washington D.C. plate shoots with ILM, including filming on the Theodore Roosevelt Bridge that reaches into the Triskelion. “We were able to shoot for two 30 minute chunks so that we could film without any traffic,” says Grill. “We took a RED camera out there for ground level footage like moving cars on the shoreline, too. At some points we even used the real water, but mostly it was CG.”
“What was interesting about that sequence,” adds Grill, “is that it was shot on a bridge in Cleveland that had not as many lanes. We ended up extracting pretty much all of Captain on a bike and putting it into our completely synthetic bridge.”
Rogers throws his shield into one of the Quinjet’s turbines then launches himself onto the vehicle – a model Scanline adapted from ILM for the sequence. “That was all shot just on a buck out in a parking lot,” says Grill. “We used footage of a stuntman doing flips and hanging off the edge of the Quinjet and then it was a nice mix of real footage and full CG for the final shots.”Watch part of the scene in this clip.
Since the Quinjet was low to the ground and also ‘shot up’ the road near to Rogers, Scanline added heat distortion, smoke effects and other atmosphere into the shots. “I love being able to dirty stuff up in visual effects,” describes Grill. “When you’re shooting with a camera, things hit the lens, especially smoke or charges. Because there was one shot where they had charges on the road and stuff hit camera, that set the tone from the practical shot. We took that and ran with it.”
Bucky and Cap flashback
Action: In a flashback sequence, Rogers (in his pre-Captain America state) talks to Bucky Barnes at a tenement housing area in Brooklyn.
Innovation: Lola VFX and Whiskytree combined to deliver shots of Skinny Steve and a younger Bucky in the 1940s setting – with a tricky camera move.
Production filmed live action plates on a partial location, which Whiskytree extended to establish as Brooklyn with the Manhattan skyline in the distance. “This is a talking moment in the movie,” says Whiskytree CEO and Creative Director Jonathan Harb, “and you wouldn’t expect it to have a lot of VFX, but what happened was it became a very hand-held camera move as the camera walks backwards, up steps and through a landing. So the big challenge of the sequence was nailing the matchmove, especially since it was shot anamorphic.”
Lola drew on techniques it had pioneered on Captain America: The First Avenger for making ‘Skinny Steve’. “However, on the first Cap,” notes Lola’s Edson Williams, “90 per cent of the shots were all Chris Evans. Marvel were pretty adamant that it be his hands, his legs – they really wanted to maintain his essence. For this film we did some shrinking of Chris Evans, probably about 60 per cent, but then the other parts were face projections and other tricks.”
Lola then worked with Whiskytree to produce the final imagery. “Because of the amount of imagery that we were removing,” explains Whiskytree visual effects supervisor Votch Levi, “we thought it would be easier for us to just create an entirely new background, finish that shot and send that to Lola and they would comp Skinny Steve on top. They would also comp Bucky on top – Bucky got some work done to his face to make him look younger. The reason for this workflow was that as Skinny Steve gets ‘smaller’, he reveals parts of the background that were occluded, including parts of Bucky.”
Whiskytree’s Brooklyn environments were CG models and a replacement sky with an HDRI designed to match the original plate lighting. “We received some concept art from production that showed this back alleyway and then really far off in the distance there were these water towers in the city, and then way beyond that there was the Emerald City,” describes Levi. “To get that layout just right, it required a lot of layout and camera changes. But we could do that very quickly because we’d built things in CG.”
Whiskytree also contributed shots of an old underground S.H.I.E.L.D base in New Jersey discovered by Rogers and Romanoff. Here, the studio worked on buildings for an establishing shot and the aftermath of a missile strike on the base, while Lola worked on a brief Skinny Steve flashback as Rogers recalls his training at the same base.
Action: Rogers and Romanoff confront Jasper Sitwell on a Baltimore rooftop.
Innovation: The Embassy worked with rooftop and greenscreen plates to produce shots of what would ultimately feature the Baltimore skyline.
“The foreground live action photography was completed on a partial greenscreen which they shot in LA,” says Embassy visual effects supervisor David Casey. “Then the partial set that they’d built replicated a rooftop that they had done location scouting on in Cleveland, but Cleveland was standing in for Baltimore.”
In Baltimore, production had captured multiple exposure tiles that the Embassy stitched into a 32 and a half K dome to use as its main backdrop. In Cleveland, the real rooftop was also shot as reference for the VFX crew (as well as for the production design department to replicate it in LA).
Artists carried out individual modeling for some rooftop objects such as air-conditioning units and pipework, or utilized texturing and photomapping where required. “For shots with camera moves,” says Casey, “we’d put some of the buildings on cards for parallax, then bring them to life with moving traffic, flags or reflection gimmicks.”
At one point, agent Sitwell falls from the roof but is retrieved by ex-paratrooper Sam Wilson (Anthony Mackie). These shots were shared with ILM who created the digitally enhanced flight. The Embassy realized a digi-double of Sitwell as he falls off the roof, animating the character in Softimage with cloth sims done in Houdini.
Winter Soldier ambush
Action: Rogers, Romanoff and Wilson are ambushed by the Winter Soldier and engage in a fiery road battle.
Innovation: Luma Pictures re-orchestrated Cleveland plates for downtown D.C. and also provided extensive CG arm replacements for Winter Soldier.
“We plotted a course on Google Maps and used that as reference to re-create these industrial buildings seen in the shots,” explains Luma visual effects supervisor Vince Cirelli. “A lot of them had to be 3D along the road because of parallax. Where we could keep the backgrounds, we would, but we’d have to lop off skylines and buildings because you couldn’t have tall skyscrapers in the middle of D.C.”
Luma contributed a number of hero CG elements, including a bus that Rogers flies through after getting blown off an overpass. Digi-doubles of each the characters also feature in the sequence. “We had moments where Black Widow jumps off the bridge and she’s entirely CG as she swings from a grappling hook to safety below,” says Cirelli. “A lot of the fire pyrotechnics were CG or enhanced, too.”
For the Winter Soldier, Legacy Effects had crafted a prosthetic arm piece to be worn by actor Sebastian Stan and stunt doubles on set, where it was covered in a few tracking. Both Luma (and ILM for other Winter Soldier sequences) then worked on a CG replacement arm that would articulate as slatted metal. “He moves so fast that there were so many actions,” comments Cirelli. “It was a real tech challenge to get the arm so that it was in not too many broken pieces but allowed rotation and twisting and all the things you’d expect.”See b-roll of the hand to hand fight between the Winter Soldier and Rogers during the ambush sequence.
Since the arm had to be completely integrated into Stan’s body, Luma had to also establish the appropriate blend location for the CG appendage. “Sometimes that can be more complicated than just doing a full character replacement,” notes Cirelli. “Because if you have close-up shots of a character moving with cloth, and it’s rotating and twisting, well that begs the question where do you split in the CG? How far up the body? So along with the arm actually came cloth simulation and other sorts of texturing and modeling for the seamless operation of the arm.”
Despite its rigid metallic nature, Luma’s arm rig allowed for muscle bulging. “What we had,” says Cirelli, “was a rig underneath that allowed for contiguous skin and then it would deform and expand for muscle. That skin would drive the slats and how they move. It was a multi-tiered system similar to Destroyer from Thor but a little more complex because the Winter Soldier was lightning fast – jumping, rotating, twisting, moving. We found we had to have a more robust system so that when the arm twists and bends, it does so so quickly so the rig doesn’t blow up! We had lots of renders where it looked like he had two arms coming out of his elbow.”
Luma also detailed the interior of the arm, adding a layer underneath the exoskeleton of moving rods and mechanisms – elements that were more visible in a later lab sequence in which the Winter Soldier’s arm is being repaired. The studio relied on its Maya to Arnold workflow for the CG work.
Action: The Helicarriers launch from under the Potomac River amid an aerial battle featuring Captain America, Falcon and S.H.I.E.L.D. Quinjets.
Innovation: ILM delivered hangar shots, the launch and the battle – orchestrating a ship ballet that then became an enormous Helicarrier and Triskelion destruction sequence.
When the carriers launch, the Potomac ‘opens up’ – requiring digital environments and water simulations from ILM. Once airborne, ILM’s Russell Earl considered they should move in a formation “where all three can guard each other. And then if they were stacked with two below the other the two below would sustain more damage, and that would make the scene more believable.”
Proof handled previs for the launch sequence as well as shots of them firing on each other and eventually crashing. “Making them readable as huge things was a big challenge,” says Proof previsualization supervisor Monty Granito. “The Helicarriers are like flying buildings. How do you possibly give them scale while they’re fighting each other or have Cap on them? We had to make them seem real and not just big buildings in the air.”Rogers and Wilson make a run for the Helicarriers.
ILM made use of a Katana and V-Ray pipe in scenes that involved significant animation and destruction on the Helicarriers. “We would make full use of the physicality inside V-Ray wherever possible,” says ILM CG supervisor Daniel Pearson. “We had an environment light with a sun, and any extra lights were just engine interaction and explosion interaction lights. We were also rendering full 3D motion blur throughout through a raytracer, and that’s no mean feat.”
During the battle, Captain America and the Winter Soldier fight underneath one of the Helicarriers in a dome area. Production built a small section of the dome – dubbed the ‘salad bowl’ – that was LIDAR scanned and photographed for reference, then extended by ILM along with background Washington environments.
ILM implemented progressive damage upon the Helicarriers. Explosions required further attention to detail. “We ended up taking a mesh representation of our explosions and caching them out to Alembic,” says Pearson, “and using them as mesh lights inside V-Ray for diffuse and reflection. For shadows we actually ran re-projections of the explosions and dust clouds just as a matte pass, and used them as a slide map – a quick and dirty approach.”
ILM’s proprietary fire and smoke generator Plume was used for a number of effects pieces in the battle, in tandem with tools such as FumeFX and Thinking Particles. “We instanced a lot of the effects work because there was just so much of it,” says Pearson.
“From the muzzle flashes to the explosions,” adds Pearson, “and all the dust and smoke. We actually cached out 20 or 30 variations on muzzle flashes and explosions and populated shots as we went with them. If we didn’t have to hero-simulate everything, we didn’t. The large majority of effects work was per shot, but we also on this show used quite a lot of cached explosions.”
Eventually, the Helicarriers fall back to Earth. “I knew if the Helicarriers had to crash they had to crash into the Helicarrier bay,” says DeLeeuw, “and if they crashed into that then the bay would have to flood. It’s one of those ideas you have in the spare of the moment, and then you realize how complex that’s going to be!”
The building crash also featured shots of Wilson running to avoid a debris field of glass, furniture and dust. Proof helped previs this sequence. “I had no idea how we were going to crumble the building on a previs scale to sell the idea,” says Granito, “but one of our artists developed a cool way of doing it.”
He brought the shots into Blender – normally we work in Maya – and created a simulation where you could just drag the Helicarrier into the building at different angles in realtime and see how the pieces would crumble and how the entire wall would give way. And once the previs was approved, we handed off a techvis to the second unit director, and used our debris field for how that scene was shot.”
“We wanted to make all these crashes feel like an epic moment,” adds DeLeeuw. “If Cap says he’s going to bring down the Helicarriers he’s going to bring them down in the most spectacular fashion imaginable! When you’re planning this, the kid in you really comes out – if you’re playing with Helicarrier toys you’d crash them into each other, and you’d have them open fire on each other. It was creating that epic backdrop against Cap’s story.”
Action: Scenes of Wilson in his ‘Falcon’ winged exoskeleton play out predominantly in the Helicarrier sequence.
Innovation: ILM brought Falcon to life, drawing on a mix of greenscreen wire work and digital double creation.
An important consideration for Falcon was to consider his flying style. “We wanted to make sure no matter what he was doing that he was in control of his wings,” says DeLeeuw. “When Falcon flies and lands he’s exhausted. There’s a mechanism that helps him but it’s actually his muscles that are controlling the wings.”
“It’s very easy to have a lot of fun with Falcon in terms of what he can and how he flies,” adds DeLeeuw. “He won’t be as fast as Tony Stark as Iron Man, say, but he was definitely nimble enough and flexible. Flying is second nature to him just like walking down the street.”
With this in mind, the production enlisted Proof to previs Falcon’s flights – especially for a sequence in which he is chased and shot at by a Quinjet amongst the Helicarriers. “There was a concept drawing we were given of Falcon being chased by a Quinjet,” states Proof’s Monty Granito. “I gave it to my guys and I asked everyone to do a Falcon shot that was in the spirit of the concepts – and we did it for Comic-Con. And then most of the shots ended up in the final sequence. Marvel’s so great, they just let you really have a lot of freedom.”
From the overall previs, production established which shots might be suitable for a wire work and greenscreen shoot, and which could be achieved as digital Falcons (either partially by retaining a live action head or as a fully CG hero). Some individual shots were achieved via wire work against greenscreen. “It was pretty impressive wire work,” recalls ILM animation supervisor Steve Rawlins, “but it was too slow so they did a re-time in editorial to speed it up to give it the energy. Our plan was to put the wings on this, but you get this footage in and because it’s sped up it has that high frequency motion you get when things are sped up that gives it away.”
ILM then continued the Falcon development, including in relation to the operation of the exosuit wings. “The wings have a mechanical quality so we had to deal with the fact that they fold up,” says Rawlins. “Once they’ve unfolded they become more like a hang gliding wing, more of a membrane. We had to deal with the technical issues of a rigid set of objects that would unfold, Transformer-like in a way, but more mechanical. Then once they were out it would be deforming in a flexible way.”See how some of the live action for Falcon was filmed.
“So our layout department would match-animate him,” says Rawlins, “and hand it over to us (in animation). We’d clean it up and take off all those high frequency movements that would be those giveaways. It turned into an all-CG Falcon but we had that basis of something that was real to use, so we weren’t inventing a performance from nothing.”
In order to create digi-doubles, each of the main characters including Wilson was scanned – some using the Light Stage – and also heavily photographed. “We set up a controlled environment,” explains Earl, “where we had still cameras, light boxes and you would put the actor there on a turntable, they’d go in on a neutral position, we’d rotate the turntable 15 degrees, take another picture. So you had calibrated photographs – you know where the tripod is and you can shoot a calibration object. Then we would do close-ups of the face, polarized and non-polarized so we could go back and use those to generate the textures we needed.”
The digital character was animated in Maya and then the wings and exosuit rendered in V-Ray. ILM’s animators mostly worked with a low resolution proxy of Falcon. “It had pretty much the accurate form and shape of the wing,” says Rawlins, “but without all the details of the internal mechanism and the way it would deform.”
Earl considers Falcon one of ILM’s standout achievements in combining digital double work with the live action photography of Wilson, and incorporating him into the frenetic Helicarrier battle. “My philosophy is, well if we can shoot something, let’s shoot something. It’s always great to have something real. We built the digital double and there were shots where they’d come up and I’d be ‘Is this a digital one?’ I couldn’t tell the difference!”
All images and clips copyright © 2014 Marvel Studios.