Super-charged VFX: The Amazing Spider-Man 2

The Amazing Spider-Man 2 is the latest film to feature the swinging superhero but it also introduces some fresh villains, including the super-charged Electro. Helping to bring the heroes and villains in the Marc Webb-directed pic to life was Sony Pictures Imageworks as principal visual effects vendor, aided by VFX from MPC, Blur Studio, Pixel Playground and Shade VFX, with previs and postvis by The Third Floor and The Cavalry. fxguide finds out how some of the film’s key characters and sequences were achieved.

For an in-depth discussion on the visual effects of The Amazing Spider-Man 2, listen to our fxpodcast where Mike Seymour talks to visual effects supervisor Jerome Chen.

Rise of Electro

Electro is able to harness and control electricity.

Imageworks’ principal challenge on the film lay with realizing the character of Electro – a living electric generator born when OsCorp employee Max Dillon (Jamie Foxx) falls into a tank of genetically modified electric eels. Electro was conceived as having a lightning storm constantly brewing inside of him, and he is able to shoot bolts and materialize in and out of electrical currents.

KNB Effects Group was responsible for Electro’s special effects make-up, which Imageworks used as the basis of further manipulation to achieve a more transparent, glowing and lightning-filled look for the character. Lighting Electro was a crucial part of the character since he emanates a blu-ish hue himself. For scenes of Foxx wearing a hood, the filmmakers experimented with a remote LED light set-up incorporated into the actor’s clothing. “When Jamie was wearing the make-up it would absorb the light and become very dark,” explains Imageworks visual effects supervisor Jerome Chen, describing why the LEDs were used. “In some cases it gave the right illumination, in other cases it blew the face out too much. We’d dialed up the LEDs for the illumination and would find a sweet spot – 20 to 30% of the actual power.”

Jamie Foxx in Electro make-up.

Imageworks looked to lightning and the way it would illuminate atmospheric storm clouds in developing Electro’s subsurface appearance. The storm inside the villain’s nervous system changed color depending on Electro’s mood. A typical shot began with the live action plate of Foxx in KNB make-up, which was then tracked via veins (no tracking markers were employed), with roto-mation then employed to match facial and muscle movements to a virtual Electro.

Foxx was cyber-scanned and carried out 40 to 50 FACS poses for scans that would be used by animators to re-create facial movement. “We provided the animators with a full facial rig from which to work from,” says Chen. “If you had the FACS poses to register against, we got a pretty tight tracking out of it. So we would do a pass that we thought was pretty good and then we would render the CG and as we got further along, if we noticed areas that would slip, we would go back and do further tracking on the troublesome areas and try and massage it in the composite.”

An internal structure for Electro was created to enable light to bounce around inside the character. Imageworks also built-in flesh and vein volumes that the energy would light up. Finally, animators ensured that the internal lightning traveled through Electro’s body in an interesting way.

The VFX team found that Electro’s eyes were a major challenge. “Originally we were going to have the sclera and iris almost one value,” says Chen. “They were just going to glow. It was almost by accident that we came across a version where the sclera was black and the eyes were very bright and we realized that’s what we should do. You direct your attention right at his iris and we can see the whites of his eyes are sheer black.”

“We’d have a real face and we put the CG eyes in and they weren’t moving correctly and suddenly you have an uncanny valley happening, and all it was was the eyes,” adds Chen. “The eyes were so critical. We made sure that the movement and the eyes matched what Jamie was doing.”

Electro’s bolts and plasma blasts were conceived after inspiration from Tesla arcs, plasma balls, high voltage burn-off and lightning. Artists also studied space nebula plasma. Each bolt was made of multiple layers – a pre-charge, arcs, specks, particulate, flashes, branches, aura and other elements – with colors inspired from sea life (since Electro receives his power from eels).

Imageworks’ animation team would drive the positions of the bolts, as Chen explains: “An animator would time out how long it took an arc to reach from Electro to a target place and then we had these various gizmos that would give it the same amount of noise and frequency of amplitude to the look we wanted. Out of animation it would go into Houdini fx pipeline and different layers of fx would be driven off the original curve that the animator supplied.”

The many passes were then driven from the animator’s curve, which meant that the bolts could be approved by the director and be bought off at the editing stage. “If the lightning appeared in one shot and we cut away and then there’d be a wide shot,” notes Chen, “we’d need to see the pacing of where it went. That was a good way to get a timing approval in the animation phase and have it driven by an effects pipeline so that it was consistent.”

Spidey’s new suit

Although Imageworks has had significant past experience in creating and animating Spider-Man, this new film presented an opportunity to re-visit the suit and Spidey’s swinging style. Partly, that was because the practical suit this time around was slightly looser than previous incarnations. “Andrew’s physique was different in this movie than the last one,” says Chen, “and the suit also clumped his muscles in a different way. So we had to basically start over with the Spider-Man asset – the one thing I figured we’d be able to re-use we of course had to re-build!”

Andrew Garfield and his stunt double were scanned and used as the basis for Imageworks’ Spider-Man digi-double. The looser suit fit meant that a cloth sim approach had to be taken to show, for example, the rippling effect of wind rushing past the suit. “We thought it was initially going to be a real challenge,” says Smith, “but it actually helped us I think because we had this slight gathering of cloth and wrinkles and it gave it a more organic feel to the movement and surface of the suit.”

Spider-Man swings through New York city.

“The suit also had material qualities that were a little more anisotropic and more silky,” adds Smith. “So from a lighting standpoint, the response from that was more dynamic and it changed a lot depending on the lighting if there was a strong light on him or in an overcast environment.”

Visual effects artists also had to deal with changes to Spider-Man’s eye shields which are now much larger. “They had a light mesh and then a plastic shield that went over the top,” explains Smith. “The stuntman couldn’t wear the plastic shield because it distorted their vision or fogged up. So we had to put that plastic shield back in. It actually gave us a place to reflect a lot of the environmental lighting into it. Both Imageworks and Pixel Playground did that work.”

Spider-Man’s swinging style was also enhanced to be more realistic from a physics point of view in this film, both by carrying out movement studies of Andrew Garfield’s stunt double and looking to CG tools for help with real life physics. Says Smith: “We buy the fact that he’s bitten by a spider, that he’s got superpowers – but we wouldn’t buy into the fact that he can defy gravity. Our animation supervisor David Schaub and his team set up these tools where you can actually do physical simulations where you could drop an object the weight of Spider-Man off the end of rope and see how fast that should swing.”

Filming on the Times Square set.

“Then we would compare that against the CG animated one,” adds Chen, “and blend between the two of them and adjust the animation to make sure the arcs felt right, but also try and incorporate a Spider-Man pose or things he had to do that fit within a correct arc, a correct pendulum. We were basically paying more attention to the physics and looking at the animation from a mechanical point of view but also looking at the orthographic views to make sure they obeyed those motions also. We called it obeying Dr. Gravity.”

And the buildings Spider-Man swings through were also updated, especially for the day-time New York City sequences. Imageworks enhanced its levels of detail on objects within the city for the final shots. “We would add in small things like disturbances to the ground surface, little things you’d pick up in the day,” says Smith. “We had to upgrade the textures, add more color variation in the sidewalks, repairs to the surfaces. Our shading team figured out from a global illumination standpoint where the sun would be that time of year, that time of day and then mimic that illumination model.”

Times Square stand-off

Electro and Spider-Man first face off in a grand Times Square sequence in which the newly electrified villain showcases his powers and nearly kills several innocent bystanders. For the nighttime shots, production built a portion of Times Square on an outdoor set. On set, DOP Daniel Mindel – who shot on film – utilized interactive lighting rigs to simulate Electro’s bolts, and the production had bright LED Barco arrays on cranes as stand-ins for the Times Square ‘jumbotron’ screens. Imageworks then embarked on significant 3D set extensions, plus the addition and augmentation of Spider-Man and Electro for many of the elaborate fight scenes.

A small crew traveled to New York to carry out LIDAR scans of Times Square. “It was an enormous acquisition of geometry and textures and hi-res Sony F65 video footage to try and get as much material for the texture painters to work with,” says Chen. “It looks like we took photographic stills and mapped it onto the cards around the space, but because of the camera moves and choreography of the fight, it necessitated geometry for that, so it becomes an invisible backdrop to the fight between Spider-Man and Electro. That sequence was 300 shots and took a full year for the crew to do.”

Filming Electro in the Times Square set.

The Times Square shots involved a significant compositing effort from Imageworks – greenscreens were employed but much of the work involved delicate rotoscoping. Smoke was also something the filmmakers had to deal with, as was heat ripple emanating from burning cars on the set. “Because Mark really wanted to cover it with a long lens,” explains Chen, “the long lens would often shoot through this sheath of heat ripples – you’d get this distortion flickering in the air that the lens would be shooting through. So if you have a composite in Times Square you had to run the same type of ripple through it to simulate the look of the air distortion. We liked the look so much that there are a few CG shots towards the end of the movie that we decided to add this kind of fake heat ripple across the whole frame.”

In one part of the Times Square sequence, Spider-Man utilizes his ‘spidey sense’ to save a group of on-lookers on the bleacher seats located on Duffy Square. The action here is depicted in slow-mo, as Spider-Man launches his web slingers against encroaching bolts from Electro.

Watch part of the slow-mo Times Square sequence.

Chen says the shot was filmed in a “remarkably simple way. We basically hired as many athletes, dancers and people with extremely good muscle control to be the background extras. For those shots they held their pose as still as possible.” Stands and wires were used to control their awkward poses and show them in a frozen moment while the camera moved past them at 72 or 150 fps. Imageworks then painted out any supports, carried out stablizations and added in the digital elements to the scene.

Power plant face-off

Spider-Man and Electro have a climatic battle amidst a next-generation power plant. Here, Electro is able to control his electrical powers and de-materialize in form. He is then able to re-materialize in what Smith describes as an effect Imageworks “intended to relate to the coalescence of ionized gases.”

For the materialization shots, VFX artists animated a full Electro first, then simulated electricity and gas effects. The power plant sequence shows Electro’s powers reaching ‘teleport’ status as Spider-Man chases him around the electrical pylons. This sequence also shows Electro using bolts of lightning to attack Spider-Man and blow up the pylons.

“We ramped up our demolition pipeline for these shots and ones in New York city,” says Smith. “We brought in a tool called Digital Molecular Matter or DMM (from Pixelux), which relies on these tetrahedral shapes. It really allows for the DMM to become aware of where the pieces break apart. We could set up where they break – so instead of them just cracking and exploding they actually bend after the forces are put upon them they break under pressure.”

Meet Rhino

Audiences get a brief introduction to the character Rhino (Paul Giamatti) towards the end of the film when he confronts Spider-Man in a weaponized OsCorp exo-suit. Designed by Weta Workshop and also contributed to by Blur Studio and Imageworks, the “idea was that it was a leftover 80s hodgepodge of OsCorp designs that they sold to the Russians,” explains Chen. “It was fun because it didn’t need to look so polished in terms of its design but it still needed to have a lot of textural detail.”

Giamatti was filmed on location surrounded by a stand-in ‘contraption’ that had him at the right height off the ground and with markered pieces representing where the final Rhino suit would be. “We built a matte gray canopy around Paul so that the shadows would be cast correctly on him from the CG version of the Rhino,” says Chen. “When I was talking to Mark I said we’ve got to make sure the design doesn’t change otherwise the shadows won’t be right, and he said that would be fine – but of course we changed the design!”

The importance of previs and postvis

Crucial to fleshing out the action and pacing of The Amazing Spider-Man 2 was the role of The Third Floor in providing both previs and postvis services for all the film’s major sequences.

This included designing Spider-Man’s early swing through New York. “The first thing we did was download as many Spider-Man comics as we could find and bring in our own, and we’d flip through it and find some really great poses,” says The Third Floor’s previs supervisor Patrick Smith. “We do a rough first pass to block out the animation – we just hit the key poses, and then we slug that together into an edit and then we tweak it from there. We wanted to accentuate those Spider-Man poses that are very iconic poses and make it feel like what it would feel like if you were him – it had to be a roller coaster of action.”

For the power plant confrontation, Patrick Smith says the director looked to make the pylons appear like an electrical bamboo forest that would give Spider-Man more space to swing around as he chases the de-materializing Electro. “We came up with a pretty cool effect for that which seemed to translate well into the final thing,” explains Smith. “We used After Effects to animate over a series of 3 or 4 frames and in that series of frames we play with the glow and the distortion of the look, and give that transparent almost phantom look as Electro travels across. We wanted to make it feel like he was traveling on the electricity or the mist in the air – he’s using that as a current to travel along.”

Postvis from The Third Floor was employed for nearly 1000 shots, too, including the Times Square sequence. “We’d drop in all the set extensions,” says Smith, “and get rid of all the greenscreens. We’d add in any sort of effects if Electro was shooting any sort of blasts, and we’d add in a digi-Spider-Man character in there. Their intention was to get to as close to final as possible so they could use it for screenings and they would make sense to everyone.”