Tippett Studio visual effects supervisor Matt Jacobs takes us through his studio’s work on the treasure vault sequence in Harry Potter and the Deathly Hallows: Part 2. The Tippett team took on the sequence based on some initial Double Negative work – including matchmoves, scene geometry, look dev, textures, models, turntables and both 2D and 3D scripts – to create the final treasure popping and cascading effects.
fxg: This is a scene that called for a mass of replicating treasure. Can you describe the sequence?
Jacobs: Harry, Ron and Hermione have returned to the Gringotts bank to find another horcrux to destroy, which is in Helga Hufflepuff’s Cup, in Bellatrix Lestrange’s vault. They enter the vault with Griphook the goblin and another goblin. The first thing we had to do was the lumos effect from the wand tips. That was a mixture of 3D tracking and 2D compositing to create a very specific glow.
Then as they’re wandering through this dark environment with treasure everywhere, Hermione knocks over a golden bracelet which falls on the floor. There’s a still moment as it starts trembling, and then it basically pops like popcorn. That was the way it was described to us. It replicates into several different other bracelets. Then Ron is startled by it and he knocks into a silver platter which falls on the floor, and a golden chalice falls. All those things begin to replicate as well. At this point the treasure just starts taking off. So in a very short sequence we had to fill the entire room up with treasure. The volume just goes wild.
Hermione has the Sword of Gryffindor and she throws it to Harry while she is being surrounding by treasure. Harry starts climbing across this ever-expanding pile of treasure and pretty soon it’s a mountain. There’s a shot where we’re pushing in through Ron and Hermione who are up to their waists, and the pile is growing and Harry’s climbing up the treasure trying to get to the chalice. So it’s actually working for him and against him as he’s struggling through it.
At one point the treasure completely covers Harry and you think, ‘Has he been buried alive?’. But then he comes bursting through this massive amount of treasure, falling back towards Ron and Hermione and the goblins. Then Griphook steals the sword away from Harry and the kids are left behind to be buried alive again. Eventually the treasure pours out through the door of the vault. So in this scene everything on the floor and the mounds that start growing were basically CGI treasure that we put into the shot.
Jacobs: In the beginning of the sequence, there was some treasure lying around the base of Hermione and Harry. It helped create some of the atmospheric lighting effects because it caused the light to be reflecting back up on them. Then as we go further into the sequence, they had some pneumatic lifts with gold mylar over them to create light bounce. Harry was climbing up this pile, five to six feet off the ground on these lifts. In that wide shot of Harry struggling through the pile, in all those shots he was climbing up these lifts and then all the treasure was treasure we put in the shots.
We had a LIDAR scan of the set and models that were representative of Ron, Hermione and Harry. We had to do match moves of all the kids in the room, and then it was left to the compositors to work out the integration between the kids and the treasure in Nuke. It was really a brute-force roto job, lots of z-depth mattes – especially when Harry comes bursting through the treasure at one point. I know the compositors spent a lot of time making sure the right treasure was in front of Harry and the right treasure was behind. I think down the road we would try and integrate deep compositing into the process, but we really only had six to seven weeks to complete the shots.
fxg: What approach did you take to the animation, especially considering there were so many pieces of treasure?
Jacobs: We had a two-pronged approach to this. We quickly realized that we weren’t going to be able to do everything procedurally, say by using particle simulations or rigid body dynamics for everything. So what we decided to do was block out the entire sequence using very rough spheres for the level and the height of the treasure. So we would work out that the treasure needs to start at a certain volume at the beginning of the shot, and end at another.
We knew also there’d be an aspect to the treasure replication where it would be flying in the air and replicating in the air. So the first place we went was, ‘What’s the volume and how much do we need out of each shot?’ Those were things that we could show to production and get feedback on the size of the mounds and have say more treasure flying over camera.
Once we had those mounds defined, we went into Houdini and we were able to do the replications we needed to do – create these piles replicating and growing. The basis for the overall activity and the volume was done in Houdini. Once the simulation was 90 per cent of the way along, we actually turned it back to our animators and they had a tool where they could take the simulation, instance the geometry and see it in Maya. In Maya, they were going in and hand-placing where the treasure was replicating. By the end of the process, the animators were making their own piles of treasure around Harry. It was a lot easier to do it in animation than through simulation, because you could direct the performance and composition of the piles.
fxg: What were some of the compositing challenges?
Jacobs: Getting the right depth of field was a real challenge. It was shot in a dark environment. There was a shallow depth of field. There was a fair amount of back and forth between the compositors, working with the technical directors to break up the scene so that the compositors had control over various layers to play with the focus. One issue you’ve got when you’re pulling z-depth mattes to pull focus is if you’ve got a highly blurred object in the foreground, it can can affect everything in the background, unless you’re using a deep compositing set-up.
fxg: For the treasure itself, how did you approach the look dev and rendering?
Jacobs: Everything was raytraced – a mixture of multiple bounce raytracing and single bounce, depending on the level of detail that we needed and how close the objects were to the camera. The reflective nature of the treasure was based mostly on environment maps of the set and then a few key lights to add a few hot kicks where we wanted them to highlight the metallic nature of the treasure.
The one thing I found that really tied the treasure to the look of the shot plates was leaning heavily on the lumos effect, and creating light pollution onto the treasure from the wands. We tried to create pools of light that mimicked the blue light from lumos onto the treasure. We did this by adding a balanced mixture of diffuse blue light and some really hot kicks with a blue glow around it.
All images courtesy of Warner Bros. Pictures.