Effect-ing the story

In our continuing series on invisible and seamless visual effects work, we take a look at some recent films – War Horse, Safe House, Big Miracle and Underworld: Awakening – where the VFX made a crucial difference in enhancing the director’s vision on screen.

War Horse

Prior to working on Steven Spielberg’s War Horse, Framestore visual effects supervisor Ben Morris’ last film was Prince of Persia: The Sands of Time – a tentpole release which, for Morris, involved complicated sand and snake simulations. War Horse was, of course, somewhat of a different challenge where the director pushed for as much of the film to be realized in-camera as possible. “It was refreshing but also took some getting used to,” says Morris of taking a different approach on War Horse. “It was very clear from the beginning that that was the only way Steven wanted to make the film. If visual effects were required they were absolutely only to enhance the story.”

Watch the trailer for War Horse.

War Horse tells the tale of Joey, a thoroughbred that makes its way to France and Germany during World War I. Despite their supporting role, Morris and his Framestore VFX team were on set in England on a daily basis (very little was filmed on soundstages). “My role was quite often to think of every possible solution we could use whilst filming that would make the shot not require visual effects,” recalls Morris. “So in a funny way I was trying to put myself out of business! Needless to say I didn’t, because we actually did quite a lot of work.”

Framestore was the film’s sole visual effects vendor and delivered 200 shots. For each of them, says Morris, “Steven was very adamant about not doing shots that couldn’t be done without real world equipment. He didn’t want any virtual camera moves. He wanted the film to have a certain look and a certain period. He wasn’t keen on doing zooms, he didn’t like doing huge camera moves, he very much wanted to ground the film in a reality and we very much springboarded our work off of that.”

A sequence that maintained that kind of practical filmmaking was the British cavalry charge, in which soldiers on horseback hide in reeds before surprising a German unit – ultimately, however, many are gunned down by machine guns. “It’s the point at which warfare became mechanized,” notes Morris. “So Steven wanted this ghostly transition from ‘Tally ho, gentlemen’ British cavalry to total devastation by a small number of machine guns.”

The sequence was filmed in the Duke of Wellington estate using about 80 horses and trained riders, with Morris anticipating ramping some shots up via visual effects to 400 horses. “Initially we thought of CG horses in the background, but mostly the set-up became slightly tighter camera angles,” he says. “We had the Russian Arm rig, which is a Mercedes with very large crane arm and it can drive along at 30 or 40 miles per hour and get the camera into places you can’t normally manage. So for that scene – the opening – production designer Rick Carter provided a patch of reeds and in the story that’s where the cavalry hides before charging.”

“It turned out the patch was a little too small for the kind of shot Steven was after,” adds Morris, “so we extended the cavalry and reeds into the distance for 20 shots using real reference footage of the reeds, then with roto and crowd replication techniques. We also added the fluff flying through the air. Rather than re-create it digitally we shot elements of the bullrush fluff on a soundstage at Longcross. It kept a textural feel that matched the live action, even though it was comp’d all in Nuke.”

“Also, when we got there,” continues Morris, “one of the things I’d heard is that Steven moves so fast on shoots. It’s just incredible – it’s exhilarating the speed he shoots at. One of the things about crowd replication is that it’s quite methodical and a slower technique. So with a bit of trepidation we got up on a crane and did one pass, a second pass and then a third. Then I had one of my compositing supes who would stream the takes off the video assist and comp them up. Steven was there and he just loved it. So the shots came together with three passes and he said, ‘No, no, let’s do another two!’. And I could see his eyes light up and he thought it was such a great old-fashioned technique.”

Additional effects for the sequence included muzzle flashes for the machine guns (which were actual First World War artillery), a large plate extension as the camera pulls back to reveal the devastation on the cavalry, as well as significant paint-outs for temporary fences that had been placed to guide the horses as they jump into the German area. “They didn’t want horses jumping over people as there was a chance the hooves would kick people,” says Morris. “So we put up livestock electric fence strips with green poles. To the horses it looked like electric fences and instinctively they didn’t want to go there – although of course we didn’t actually electrify them. So they were trained to run through these channels – for us it was an easy solution to not have to rub out gray-suited riders, which we did do later.”

Director Steven Spielberg on the set of War Horse.

In a subsequent sequence, Joey is in the German trenches and is cornered by a tank. To escape he gallantly leaps over the vehicle to safety. “That was an example of where we had a plan for a live action solution,” explains Morris, “and over the course of a week or so of shooting it didn’t quite work. Steven had effectively said to me, ‘Did I want to go off and shoot this?’ So I went off with a camera team and the original plan was that the tank was going to tip down into the hole, it was going to jam in the mud just for a wheel spin, at which point Joey was meant to think, ‘Ah, I know how I can get out of here,’ and leap over.”

“We built wooden rostrums over the tank in its jammed position,” adds Morris, “and we attempted to get horses running over it. We spent a number of days acclimatising horses to the leap up onto it and the leap off. We shot that for maybe two days to try and get the camera angles Steven was after. Then I took them over on an iPad and reviewed them with him. I think rightfully he said we had lost the dynamics of the shot and we had to make the tank continue to move.”

For Morris, this meant the shot became somewhat more complicated as the horses had to make majestic leaps, especially for a planned shot of Joey coming down and landing close to camera. “So then I said to Steven that we had this other CG horse being prepared for a different sequence,” says Morris. “Steven was very honest and said ‘That horse has to be perfect, I don’t want a CG horse in this movie.’ So we shot the plates clean and had the tank blasting through, and then within a few days we’d roughed out a layout over the tank. Then Steven suggested I shoot the rest of the scene so we grabbed the camera crew and shot the rest. There were a lot of trainer interactions and rub-outs and re-syncing of the tanks.”

For the digital horse shots – a side angle of Joey running over the tank and the front-on landing – Framestore modeled the horse entirely from reference. “We did a photogrammetry setup where we had six Canon 5Ds in a semi-circle and shot lots of synchronized stills of the horses,” explains Morris. “There were actually two horses playing Joey. From those we took it into ImageModeler and our internal software tools and built a hybrid of the two horses that was a generic Joey. We also had huge amounts of photographic reference separately. The saddles and pack were also cyberscanned and we modeled all of that and had a whole secondary sim in nCloth in Maya. We used our internal fur and groom system and we built a muscle system – their bodies are just these incredible continuous moving and clenching and skin sliding things.”

In addition, artists hand-placed mud textures that replicated the real dirt that had been hand-applied on set. Reference of soil being kicked up was also added, along with digital earth where necessary. “For animation,” says Morris, “we did have some gallop cycles, and at one point we thought we were using it for the cavalry. But the final animation was done with a very small team of two animators, using reference from show jumping.”

Another digital horse shot was required for a scene of Joey attempting somewhat unsuccessfully to jump a trench – something the filmmakers were never going to consider attempting with a real horse. “The plan was to shoot a first horse element from the same camera angle that’s up on a ramp of earth that we could then comp in for the first jump,” describes Morris. “A quad bike with the camera on it would then race off down the trench and we’d do a Texan switch and bring in our real horse with a gray-suited rider on it to ride down the edge of the trench. We then had special effects supervisor Neil Corbould primed to set off his explosion for the real horse to react to, and at a certain stage we knew we would do a digital takeover. We also had a practical rig of some sandbags falling off the edge of the side of the fence which gave the cameraman something to aim for. As the sandbags fell he would then follow them down to the ground, at which point the quad bike would screech to a halt.”

“For the second plate,” says Morris, “we then took the camera off the quad bike and stuck it on a set of legs and brought in a separate horse that had been trained to lie on the floor and get up and then run off and we shot a second element with that piece with the lights flashing and him running off.”

Ultimately, the horse elements filmed for the first jump were not as strong as Spielberg had initially intended. “As we were bringing the shot together,” says Morris, “Steven thought our second jump looked so amazing and asked, ‘Wel, what can we do?’ So our animator who was doing the second jump did one of the classic ‘I’m going to stay up for a couple of nights’ and he animated the first jump. I was a bit sheepish because I knew how sensitive Steven was to digital horses but he said, ‘Oh wow that’s so much better! Where did you get the other element from?’ And I didn’t tell him until the next review that it was CG. He just laughed his head off and said, ‘Well done, that’s super-cool – let’s go with it.'”

“For reference,” adds Morris, “we had to look at steeplechase jumps where they hit the fence or fail and crash – our animators wanted to get that ugliness into it. Steven was very keen to show the horse landing and winding itself. We also painted out most of the practical sandbags to replace with our own that slid and reacted to the landing.”

Further digital augmentation was required when Joey launches through the surroundings and becomes entangled in barbed wire. “For the running shot,” explains Morris, “Rick Carter had designed a three-ways of compacted soil with drainage that the horses were trained to run down. Then we had another road section hidden into the no-man’s-land for the camera track. We had a gray-suited rider who rode the horse to keep it calm and make it ride in the right direction, which then required digital removal and reconstruction of the plates.”

The shot of Joey hitting the barbed wire was filmed with the horse contacting string painted with knots to look like the real thing. “So for the initial hit the horse actually did contact something,” notes Morris, “but it was very safe and the string actually snapped as it hit. And then from then on we were never allowed to have anything hanging from the horse, so all of the barbed wire that you see attached had to be tracked and simulated.”

Joey accumulates more barbed wire and gates and star-shaped ‘jacks’. For this, the on-set horse ran through the required area with tracking markers and the camera array. “Then we got a quad bike,” adds Morris, “with a practical wooden gate frame and dragged that through just for reference and we simulated a digital version. As the horse finally comes to the end of its run, Neil Corbould shot an animatronic crash-horse that was on a rail that could be fired down the rail and do a full somersault and land. For two shots there we got rid of his rig and enhanced the horse a little bit and then put in lots of gates and wires.”

Joey is then seen thrashing around helplessly covered in barbed wire. On set, the horse had nothing covering it, but two trainers co-ordinated its movements, gently coaxing it up or down. To help with the horse body tracking before any digital wire could be added, Morris set up an array of digital witness cameras on set that were synchronized to the shutter pulses of the ARRI film cameras. “For the wire itself,” says Morris, “Steven wanted it to be very alarming and harsh. We painted up still concepts with varying complexities. As soon as we had that basic design approved, we built an enormous spider-web of digital wire that was 3D body tracked to our digital horse. The wire was procedurally built in Houdini and then exported to dynamic curves that were simulated in Maya and connected onto the horse. We added CG dirt and posts as well. It was a great collaboration between the real horses, practical effects and digital work.”

Framestore also completed several further clean-up shots, including vapour-trail removal, telephone wire fix-its and other enhancements. But interestingly, one scene in War Horse did not require visual effects at all, even though it might typically be the kind required. “People often ask about the end of the film with the sunsets in Devon,” says Morris. “‘Surely you didn’t shoot those in Devon’ – they say – but we did, and we had three sunsets in a row.”

Safe House

Visual effects studio Image Engine recently had what they considered was a unique opportunity to enhance the story in Daniel Espinosa’s Safe House. For the film – which tells the story of a CIA rookie tasked with protecting a fugitive in Cape Town, South Africa – visual effects supervisor Simon Hughes knew his team would be responsible for certain shots, including crowd replication in a soccer stadium and after car chase shots. But after shooting wrapped and the director delivered an initial edit, Image Engine was called upon to pick certain moments in the film where visual effects could enhance some key moments.

Original plate for van crash.
Final shot.

“The director basically handed us the film,” recalls Hughes. “He said, ‘Right, we have a visual effects budget, but what I want you to do is go through the whole film and look for any opportunities to improve on the sense of danger and excitement.’ It was such a fantastic opportunity that doesn’t happen much, really. Often in visual effects you’re just handed a very small part of the film and kept out of the loop and you don’t always get a feel for things holistically.”

Image Engine visual effects producer Geoff Anderson says that this process involved refining the visual effects budget during production in order to create shots that would best serve the story. “What’s really interesting,” notes Anderson, “was Daniel coming to us and saying, ‘Let me know what works well in these sequences’. We looked at the movie as a whole – and at one point we saw it as a whole and could re-bid the entire show based on Simon’s suggestions.”

Despite the addition of shots, Image Engine had ensured a continual on-set presence during principal photography aimed at acquiring all the necessary survey and lighting data necessary. In one particular add-on shot of a van crashing into a wall, production filmed the sequence for real but then looked to VFX to create a more grand moment. “We made the decision to focus on the main impact of the van into the wall,” says Hughes. “We ran lots of animation tests whilst the build was going on with the van. We brought the wall closer in and had a bigger impact with it to seriously damage the wall and the van.”

“The asset was built in Maya but we did a lot of deformations and damages,” continues Hughes, “especially on the side which we created in ZBrush. “We more or less made two vans and had a blend shape between them. That was then handed off to animation which was done in Maya. For the effects work, that was done in Houdini which we used for the smashing glass and headlights and wheel skid smoke. It was composited in Nuke. Camera tracking and matchmoving was done in boujou and rendering in 3delight.”

Original plate for stadium.
Final shot.

Image Engine rounded out its enhancement-style shots for Safe House with face replacements, matte paintings, CG bumper bars, debris, gore addition and muzzle flashes. The studio also completed crucial invisible crowd shots to fill Cape Town’s Green Point Stadium during a soccer match. Although these were always tagged as visual effects work, Image Engine was involved early to plan the shooting of the necessary plates. “I went out to Cape Town and we did a lot of camera tests with a Canon 7D at Green Point Stadium,” says Hughes. “We would look for solutions that would enable us to show off as much of the stadium as possible but at the same time still manage a tiling shoot.”

For the actual filming, 500 extras filled up one section of the stadium – out of 200 – and then moved around to complete the tiles. “We had to shoot in an evening when there was nothing going on there and combine it with a helicopter crowd shoot as well,” explains Hughes. “So essentially we had from 8pm to midnight. We positioned two film cameras to cover the actual real shots and then set up cameras on opposing positions which were four 7Ds, pretty much on the four corners of the stadium. That gave us one half of the stadium and then you could just flip the plate and cover that for the whole stadium.”

The extras were filmed in opposing red and black colors and directed to stand, cheer and go through various behaviors for the tile shoot. Image Engine also had the stadium lidar’d before assembling the shots. “With lidar, you get a very dense point cloud which can be hard to handle in Maya,” notes Hughes. “So we took that in as a new asset, we simplified the model and cleaned it up. Then we took that into matchmove and into Nuke. It became pretty straightforward to isolate the tiles to each part of the stadium, although there was a lot of work in matching the plates, adding and balancing atmosphere, doing roto and adding in the game.”

Big Miracle

Ken Kwapis’s Big Miracle, a film based on the true story of three California gray whales trapped in a hole in the Arctic Circle ice, began as a “fairly small visual effects film”, according to Rhythm & Hues visual effects supervisor John Heller. Tasked initially with the majestic underwater shots of whales, Rhythm’s role expanded to include background replacements and new whale shots as the filmmakers realized the potential of visual effects to enhance the story.

Watch the trailer for Big Miracle.

Production filmed near Anchorage, Alaska – a location aimed at providing a realistic white-out and freezing setting. For visual effects, this proved to be a challenging one, since buildings, mountains, trees and wires were often in the background and required digital removal after extensive roto and plate preparation. “The good thing was that they did shoot it on a set that gave the white light bouncing up,” says Heller. “We didn’t have to do much work to the characters to make them feel like they were in that environment, because they were lit properly.”

“I think once it was decided that visual effects could enhance any environment,” adds Heller, “the director also felt very free to shoot a film that looked much larger in scope than I think he would have done. The biggest challenge, though, was continuity from to scene to scene – it went from bright blue from bright red to sunset. We built a library of skies and ice – we have this great proprietary software called Rampage – it’s a 3D environment software that lets you stitch backgrounds together – it does an incredible job.”

The whales, of course, were Rhythm’s primary concern and for those they looked both to the on-set animatronic versions created by Glasshammer Effects and a significant library of materials sourced via television documentaries. “The main thing was that we had to find a way to create whales that weren’t too anthropomorphized,” notes Heller. “They couldn’t be too human but they had a personality.”

Rhythm & Hues created hybrids for the main gray whales in the film – two adults and a baby – between the animatronic whales and from the reference. “Our animation director Craig Talmy and animation supe John Goodman built a library of all the gray whales and some humpback whale reference,” says Heller. “For nearly all movements in the movie, they were able to find something close to it. And so nearly every animation is a rotoscope of that movement – you take that as a starting point. There’s even some reference footage of a baby whale struggling and being helped by older whales that made it into the movie as inspiration for our final shots.”

The underwater environments showing the whales under the ice shelf were mostly completed as detailed paintings, combined with water sims. “We hired a photographer to go down to Antarctica and photograph underwater ice for reference,” says Heller. “Then we always knew there were several components that would go into creating the underwater environment. One was the murk – the detritus. In the Arctic environment it’s usually a bit clearer than other environments, so we had to find a balance where that played to a value but not take away from it being majestic and pretty.”

Houdini was used to create the detritus and Naiad for the water surface simulations through which light was projected to provide real caustics and light rays. “We also wanted that to play on the whale’s skin,” says Heller, “but we did use some shaders to enhance the caustics on their surface. There was always a play-off between light, color and depth of field. We tried to mimic a real camera in that environment too, but also make it look filmic and pleasing.”

In addition to the environment enhancements and whale shots, an ice-breaking scene was added late in production utilizing a Kerner Optical miniature (that studio’s last miniature filmed on the Kerner stage before it unfortunately closed down). Rhythm also completed entirely CG shots for the hover barge and helicopter sequence, while Modus FX worked on several shots including monitor burn-ins and Gradient Effects assisted in background enhancements. “I think what satisfies me most with this film,” says Heller, “was that there was more than meets the eye in terms of visual effects supporting the film – it really couldn’t exist without some of the shots that you’ll never really notice.”

Underworld: Awakening

Although it doesn’t shy away from being a heavy effects and creature flick, Måns Mårlind and Björn Stein’s Underworld: Awakening also required significant supporting visual effects to round out the vampire and lycans film. We look at just some of these enhancement stereo effects by SPIN, Look Effects and Fido.

Click here for our article on Luma Picture’s Lycan visual effects for Underworld: Awakening

Greenscreen plate
Final shot.

SPIN provided the film’s destructive opening and closing sequences, several Eve Vampire face shots and city set extensions and digi-double work. “The biggest task was to build the city that the movie takes place in to look like Budapest but surrounded by newer buildings, a Brutalist style of architecture,” says SPIN VFX supe Jeff Campbell.

“For the shots that had a slight camera move, we rendered a lit frame from Maya, sent this render to a matte painter to add all the details and then re-projected back onto the same geometry in Nuke. We would also provide all matte passes needed to isolate areas in Nuke such as window reflections, glows, key, fill lights as well as street level lighting like store fronts, street lights and cars. Having control of lighting and textures in Nuke’s 3D space has many creative advantages.”

“SPIN used a mapping system for the city sections that divided each area into 6 zones organized by type such as old and new building styles,” adds CG supervisor Ahmed Shehata. “Designing a custom pipeline and proper asset workflow was essential to connect all departments from art to 3D to compositing. You still had to design all those buildings individually and our team developed several tools to manage the city assets. This ranged from collecting textures based the structure to generating projection camera shaders based on city sections.”

For Eve, who has a combination of vampire and lycan features, SPIN used three blend shapes to achieve the transformations. “Veins were also added to signify an increased blood flow that initiated her transformation,” says Campbell. “The veins and her face were projected on to a geometry .obj sequence that was fed into Nuke. Most of the shots involved manipulating her live action makeup by tracking a series of mesh warps in order to shape her eyes and brow. We also gave Eve new irises, changed her skin tone and added cg fingernails.”

Look enhanced the silver nitrate explosions

Look Effects contributed to a sequence in which vampire Selene (Kate Beckinsale) attacks a lycan facility with silver nitrate gas. Visual effects supervisor Max Ivins oversaw a number of driving composites, monitor shots, the addition of bombs inside elevator doors, a CG lift-well and extensions and interior and exterior explosion augmentations. “They had done a lot of previs and post-viz to make the lift scene cuts work together,” says Ivins. “We got from them a bid package that had the cut scenes and descriptions of the scenes. It seemed to be a very intelligent way to stay to the budget and make it clear what we had to do.”

“One of the most challenging set of shots involve filling up hallways with cloudy sparkles for the silver nitrate explosions,” says Ivins. “They had really big air mortars that blew out debris and dust out of the elevators. We added a particle system in Maya with a lot of MEL scripting and used Maya Fluids for the internal and external explosions. It was unusual matching the way smoke was swirling with a particle system of our own. But it’s nice to do something that I call traditional effects – let’s blow stuff up!”

Fido enhanced a series of shots of Eve battling a lycan, Dr Lane, ripping his throat out before he transforms back into human form. The sequence was shot with a stunt performer wearing make-up appliances which ultimately required visual effects enhancement. During the fight, Fido performed stereo wire removals and also replaced and added to the lycan make-up, using an on-set HDRI for reference, made difficult by the flickering car park lighting.

Above: watch a breakdown of Fido’s effects for Underworld: Awakening.

For the gory death of Dr Lane, the team began with a Luma concept suggestion that was pushed into 3D. “We recieved cyberscans of the stunt guy in the makeup,” says co-visual effects supervisor Anders Nymab. “We also knew that we had to dig a hole into his throat which was shot with a foam latex piece in its place. We did the buck shot makeup as one asset and then we also did the throat as an asset in itself.”

The transformation then required a specialized animation rig. Says Fido animation/character supervisor: “We baked everything into the same animation asset, so that we could all the transformation in one animation rig. So we had the untransformed geometry hooked up to the same rig, so that we could do transitions. We used ZBrush and Maya for the models and animation, then Mari for textures and displacements. We did some of the simulation of the blood in Houdini and comp’d in Nuke.”

Original plate.
Final shot.

“We were very free to design the transformation in itself,” adds co-visual effects supervisor Kaj Steveman. “Doing an A to B linear transformation also looks boring, so we had to figure out how to have him do something more. So we had being framed out of liquid and muscle and fat tissue – there was a mummified stage and it re-growing to a new creature.”