Hereafter & Scanline: go ahead, make my wave

Well-known for its CG water creations in films such as 300 and 2012, Scanline VFX faced new challenges in creating a near-death tsunami experience for Clint Eastwood’s Hereafter. We take a look at the studio’s digital water live-action destruction effects, digital make-up work and and afterlife visions.

Getting to Here(after)

Hereafter’s tsunami sequence takes place in the first nine minutes of the film as the central character Marie (Cécile De France) walks through a tropical resort. Marie and hundreds of other holidayers are caught up in the resulting wave of destruction that sweeps her away and demolishes buildings and other surroundings.

To create the requisite tidal wave, resulting water effects and ensuing destruction, overall visual effects supervisor Michael Owens chose Scanline based on the studio’s successful water simulations in previous films. “As a company, I think we feel we have resolved how to simulate water and make it look realistic in any kind of scale,” said Scanline visual effects supervisor Stephan Trojansky. “So with every new show it’s more about the artistic challenges of it – not just how to do things like spray and dynamics and scale. In fact, although we did so many intensive water effects and destruction and buildings falling and tearing cloth, for this movie they called it ‘incidental background destruction’. There weren’t meant to be any shots about waves or buildings collapsing, it was just there and happens as the camera passes by. The film isn’t about the tsunami.”

10Oct/ha/0113_DL_040_comp
10Oct/ha/0115_DL_040_comp


Still, Scanline leveraged previous R&D efforts on its proprietary Flowline simulation software, which is built into 3ds Max and V-Ray for rendering. As a node-based system, developments made to Flowline on previous shows could be easily ported to work for Hereafter. “We have a team of software developers who are constantly working during the show on improving Flowline,” said Trojansky. “We’re working right now on the next generation of the software. We can already run a huge part of the actual computation and simulation on GPU and mix this between GPU and CPU. It lets us get a performance speed on a single workstation of a factor of 20 that was once something previously done on a CPU.”

10Oct/ha/0116_DL_040_comp
10Oct/ha/0117_DL_040_comp


A certain percentage of development time was spent by Scanline on increasing the efficiency of Flowline and taking advantage of new hardware and software environments. A large percentage was also based on developing new tools to adapt to the actual artistic challenges. “We realised that the control this time was needed for constraining the fluid simulation to the live action water,” said Trojansky. “For the main shots what we had to continue to develop was the ability to track the live action water in the live action plate, getting some kind of motion capture of the actress by rotoscoping her, tracking the waves partially by hand, partially by optical flow techniques and converting them into 3D space – and then using that bit of information we can get out of the live action and adding it together with the overall motion of a huge tsunami wave down the street with all the turbulence and flows.”

“Actually, the swirls became very significant in an artistic way,” added Trojansky. “For one part of the sequence that we called the ‘Tea Cup’ sequence, like one of the rides at Disneyland, we had to tell the water to behave exactly like the motion-captured water movement of the live action water when it was very close to her. We then had to loosen up the constraint so that a few feet away it actually could do what it was doing in the overall shot – interacting with floating cars, dead bodies, collapsing buildings, additional flows of water.”

10Oct/ha/0111_DL_008_FG
Original Photography
10Oct/ha/0110_DL_008_comp
final shot

The wave approaches

Visual effects for the initial flooding of the hotel resort started with reference from a plate shot in Maui. Scanline stitched together photography to build up the beach setting and resort. “We replaced the ocean water so that there wasn’t a long sandy beach,” said Trojansky, “because at first the water disappears and then comes back. For the big crashing wave on the hotel resort, we came up with a new technique where it is not just white water that rolls in. The water actually picks up the material of what’s underneath, like the sand, to make it look different and be a little heavier than white water. In other areas where it was green grass, it would get that refractive feel of the grass and then slam into say a retaining wall, throw that heaviness away and then hit say the umbrellas at the resort. We’d have simulations of collapsing umbrellas with all the fine details and they would tear apart and then the wave would go into a cafeteria and we simulated Coke cans and plates and wrappers to blow away partially from wind of the wave and the wave itself.”

10Oct/ha/0175_TS_059_FG
Original Photography
10Oct/ha/0170_TS_059_comp
final shot

Although Scanline had developed previs for the sequence in consultation with Michael Owens, it was mainly used for very quick on-set reference by Eastwood, who shot scenes quickly and with minimal takes. This ‘brute-force filmmaking’ approach meant the studio had to be prepared for less time to acquire clean plates, HDRs and survey data. “It actually takes a lot of pre-planning because of the way Clint shoots,” noted fellow Scanline visual effects supervisor Bryan Grill. “There’s no time to get a clean plate. We would barely have enough time to put up a bluescreen. Once the flow is going, Clint just wants to keep moving and get through the day. We had to do a lot of pre-planning to know what we were up against.”

10Oct/ha/0195_TS_075_FG
Original Photography
10Oct/ha/0190_TS_075_comp
final shot

As the wave continues, it hits palm trees, the resort pool, lounge chairs and people running down the street. Scanline created digital environments and buildings either through re-projections or detailed 3D modelling. “We completely LIDAR’d the whole city street,” said Grill, “with and without the various kiosks and market stalls and dressing. Any time the water had to interact with the stalls or the stands or the hundreds of knick-knacks in the street, then we had to completely replace the area with CG elements and light and match it to the real location.”

10Oct/ha/0225_TS_325_FG
Original Photography
10Oct/ha/0220_TS_325_comp
final shot

“The whole street that Marie is running down was a set that we had to convert into CG,” added Trojansky. “It was partially a re-projection of HDRI imagery and re-lighting in Nuke. For some foreground buildings, we had to construct them as highly-detailed models and simulating their collapse in our simulation engine, and making them interact with the water. We even did a burning simulation for a boat that floats by at one point.”

10Oct/ha/0260_TS_135_comp
10Oct/ha/0270_TS_140_comp3


Digital doubles

Mostly, the destruction is seen from the point of view of Marie, another part of the ‘incidental background destruction’ motif. Production shot the actress sometimes just floating freely in the ocean, taking shots of her trying to swim and survive in the water. Scanline developed a method for blending real water up to a foot around around her and dissolving this into matching CG water. Shots of people also caught up in the water were a combination of tank footage from Pinewood Studios and motion captured digital doubles shot at Giant Studios, animated in Massive and MotionBuilder.

10Oct/ha/0235_TS_380_FG
Original Photography
10Oct/ha/0230_TS_380_comp
final shot

For her near-drowning, a fully CG double of Cécile De France and fully CG environment were created by Scanline. “In this shot,” explained Trojansky, “we had to combine the motion capture data with our fluid simulations. Our initial idea was always to have the character animation in the water not being keyframed but based on some motion capture. So we had a motion capture shoot with the actress in a rig which would be around her waist and she would slam against an obstacle in the motion capture session. So when you have the actual session with this realistic movement, now it’s a question of how to combine the motion capture footage with the simulation. What we realised was we didn’t want to use exactly the animation that was coming from the rig, but we wanted to use the overall motion of her travelling through the water.”

10Oct/ha/0285_TS_150_FG
Original Photography
10Oct/ha/0280_TS_150_comp
final shot

Artists then subtracted the global motion from the motion capture shoot data. Using tracking objects in the fluid simulation, they could tell exactly where the actress should be. “If at the end of the shot the character is supposed to end up here under the balcony, then we had to know where she had come from,” said Trojansky. “As soon as you start to cheat it, then you would feel like she was being pulled by a string underneath the water, like as if it was being done on a live action.”

10Oct/ha/0365_TS_005_FG

10Oct/ha/0360_TS_005_comp
final shot

“So basically what we were doing was throwing like 10,000 tennis balls into the water,” explained Trojansky. “In the first frame of the shot it would be like this perfect grid of tennis balls, and then over the course of the shot, you would see that certain tennis balls end up in this corner, and others go against the car and this one lands on the balcony. So we’d grab the one from the balcony. Then we would attach in the last frame the mocap data that has the global movement removed, so that only her internal motion is left – the splashing and head and arm movement, and we would attach her to one of the tennis balls, then back-track where she would come from. And then because we had this freedom of making these weird mobile camera-like movies from a rooftop, we would adjust the camera motion to match the actual motion in the water. That created the most believable results because it was the most natural way of showing someone caught up in these swirls.”

10Oct/ha/0375_TS_075_FG
Original Photography
10Oct/ha/0370_TS_075_comp
final shot

The digi-double shots were particularly challenging given how close the camera was on the CG character, which also featured – depending on the strength of the water movement – wet clothing, floaty clothing or sticky clothing. On top of this, Eastwood pushed for the addition of digital make-up enhancements on the actors to show cumulative bruising, scars and scratches, a technique he had adopted for the rugby players on Invictus. “The digital make-up makes a lot of sense,” noted Grill. “If you were to film Marie and have her made up, the way Clint was shooting you would never know how intense she was being banged up at any given time. When she was in the tank or in the ocean, that could go in very different parts of the scene. So once it was all worked out and we had done the water effects, we were able to build up her bruising and scratching by painting and compositing.”

10Oct/ha/0395_TS_010_FG
Original Photography
10Oct/ha/0390_TS_010_comp
final shot

Close encounters with the afterlife

In addition to the tsunami destruction and digital make-up effects, Scanline also completed shots featuring ethereal visions of persons in the afterlife world. Composting supervisor Joe Farrell worked closely with Michael Owens to come up with a look for these shots, ultimately basing them on a famous scene in Steven Spielberg’s Close Encounters of the Third Kind. “This is the scene where the UFO lands and you have that big bright light and all the people coming out of the light,” explained Grill. “That look really struck a tone and we showed it to Clint and he was very responsive. It was very simple. You can’t make out what the imagery is but at the same time you know it’s people, an old person, a baby, or whoever.”

10Oct/ha/0320_TS_035_comp
10Oct/ha/0350_TS_005_comp


After watching an early cut, Spielberg (an executive producer on Hereafter) recognised the scenes as being from his 1977 film. “We had actually masked it and filtered it and done things to it but he still noticed,” said Grill. “Afterwards Steven says, ‘Clint, that’s the Close Encounters stuff, isn’t it?’ and Clint said, ‘Oh, I’m sorry, I hope you don’t mind’. But Steven was actually flattered that Clint was using his footage to set the tone of what the sequence should look like.”

The final afterlife shots incorporated shots of people from the main shoot and a greenscreen session, with the filmmakers relying on a large 20K light and a Red camera to acquire footage. Scanline created the shots in Nuke as re-projections, adding the necessary lights and glows. “Mitch Glaser, our VFX editor, took the material and the pacing of how it goes in and out and the dissolving and set the tone,” said Grill. “It was actually the first thing we started development on and it was the last shots we finished.”

10Oct/ha/0385_TS_120_FG
Original Photography
10Oct/ha/0380_TS_120_comp
final shot

High-tech effects you shouldn’t be looking at

For Scanline, Hereafter was a chance to use its expertise at photorealistic waves, water creatures and detailed simulations for a more story-oriented film. Notes Trojansky: “I think almost the whole tsunami sequence is right up at Marie’s face. Yes, in the background there’s this huge wall of water raging and buildings collapsing, but it’s almost that you need to see the sequence twice to see that action. You should just be focussing on Marie.”

“We actually had to realise this ourselves because as visual effects geeks and artists, you are usually totally keen to show off your skills. But Michael Owens was saying to us, ‘You know what, in this sequence we should go more with the real optics of the camera and all the stuff in the background should be de-focused by at least 20 or 30 pixels.’ Well, it meant that all the models we built and all the droplets of foam that we had been working on for months, you would almost have not have seen them! But it’s more important, of course, to be more realistic. The filmmakers only wanted to make a good story.”