Luma transforms Underworld: Awakening

For Underworld: Awakening, the fourth film in the Underworld franchise, Luma Pictures devised impressive CG werewolves – known as Lycans – to again wreak havoc, and one beast that would transform from human form. We take a look at how Luma utilized motion capture, the Light Stage process and renderer Arnold in bringing the creatures to life.

A new kind of transformation

The film’s complicated transformations, seen as the character Quint (Kris Holden-Reid) becomes a 12 foot ‘Über Lycan’, firstly had to be photorealistic, but also had to hold up for considerable screen time – one transformation last 20 seconds. “We have done transformations before and seen others where there’s always this point that at least I feel the characters reach an uncanny valley,” says Luma visual effects supervisor Vincent Cirelli who worked under overall VFX supe James McQuaide. “That uncanny valley part can be right during the middle point of the transformation when you go from the plate into the semi-human stage. It was that stage that we wanted to make sure we nailed. The best way to nail it was to use the dataset from a human and the reflectance from a human. This is where Paul Debevec’s and ICT’s Light Stage came in.”

Above watch Luma’s making of reel, (with the best choice of ‘making of’ audio so far this year !)

Originally developed from research led by Debevec at the University of California at Berkeley and published at the 2000 SIGGRAPH conference, Light Stage is now in its multiple incarnation as Light Stage X. The system, set up at USC’s Institute for Creative Technologies, uses multiple LEDs – in this case 346 lights – to capture detailed facial reflectance so that almost any lighting condition can be re-created. For Awakening, Kris Holden-Ried was scanned in Light Stage X and delivered several distinct expressions. ICT then gave the resulting data sets to Luma, which transferred that to a mesh designed to work with the studio’s Lycan rig.

“Light Stage provides you with all sorts of different types of maps,” explains Cirelli. “They capture reflectance in a 360 of how a certain point would properly reflect – a reflectance normal. They give you these maps and what you need to do is figure out how to create a shader so that when a ray hits a surface on a piece of geometry, it can do a look-up as to where the light is hitting that surface normal, and the incident angle at which it is hitting it. And it retrieves that lighting information from the map – the value or the map itself, and basically display that and blend that map in.”

There were two approaches to the capture at ICT’s facility – Static and Image Based Relighting – with each capture taking about eight seconds. “The Static capture provides us with high a resolution mesh and texture maps that can be used to create a renderable digital asset of the actor,” says Luma Pictures’ Justin Johnson. “ICT used six Canon cameras firing in sync with the Light Stage X. A projector was used to cast structured light onto the actor to help create the mesh.” The Static capture also generated very high resolution texture maps, providing the diffuse component of the skin used as a color map, a specular component used as a specular map, a specular normal map and three diffuse normals maps for each channel (RGB).

Luma then worked with the Arnold renderer developer Solid Angle to write two shaders to take advantage of the high resolution maps; a Hybrid shader for the Static capture and another to process the IBR information. “This separation of the normal maps is really the key to the success of the Hybrid skin shader,” says Johnson. “It’s much closer to what happens in the real world compared to the standard ‘one bump’ shading models.”

“One of the challenges Luma faced was converting world space normal maps converted to tangent space,” adds Johnson. “One of our programmers wrote a program that could convert an 8K texture in a matter of minutes.” Then, to properly capitalize on the detailed IBR data, Luma also wrote a plugin for Nuke as well as the shader for Arnold. “The IBR sequence with both diffuse and specular was used to visualize from camera perspective what the actor would look like in a given lighting environment,” says Johnson. “This was done in Nuke with our plugin. The IBR sequence with the spec removed was mapped into the UV space of our model and rendered using both our Arnold shader and Nuke plugin.”

Sculpting transforms

To create the transformations themselves, Luma began with the design of in-between poses. “He’s got this huge elongated arm, a little short arm and one of his legs was turning Über-sized and had a huge thigh,” says Cirelli. “It’s supposed to feel very awkward and strange and gruesome. These were 2D concepts for the directors but they also wanted 3D versions, so we created rough turntables in ZBrush of what the character would look like at these intermediate poses.”

The design process gave Luma some precise moments to hit in terms of the look, especially for the hallway transformation. “For example,” notes Cirelli, “when the character first starts transforming, his arms are elongated and then his chest starts to pop. There’s a moment where he’s out of control and he has to brace himself against a wall, and that was a key moment for us. So we sculpt-matched one of the key poses that we got approval on. It was all about making sure our rig deformed in such a way that you get from 20% Über-werewolf to 30% to 40% to 50% – all these approved poses throughout the sequence.”

Luma accommodated large changes in volume between human and werewolf in its models, as well as secondary simulation for sliding skin. Says Cirelli: “We got the sim as far as we could and then we went for another layer of clean-up – we took the baked-out portions of where the sim was blowing up and we cleaned those up manually by sculpting them.”

In addition, the studio also took advantage of its motion capture stage to film scenes of Holden-Reid reacting in pain to the transformation with a specialized helmet for facial reference. “The helmet couldn’t be cumbersome,” says Cirelli, “yet needed to be stable enough to ensure accuracy when translating the motion to animation. A custom framework was fabricated from lightweight piping and fitted with a wide-angle Go Pro camera. This was then secured to the helmet with the lens facing the actor’s face.”

The tunnel Lycans

Luma also contributed Lycan shots for a parking lot scene and a city car chase involving ‘tunnel Lycans’ – smaller and more dog-like than the Über – chasing a moving van. “For those shots,” explains Cirelli, “they didn’t have any stand-ins for the Lycans running down the streets, but they had some fantastic previs. They shot the plates according to the previs and luckily they really did a fantastic job of making this character traverse the cityscape and work out how to make these shots line up.”

Luma based its digital tunnel Lycans from a macquette, altering the design slightly to allow for the creatures to run on all fours. “We had to shorten the back legs and change the anatomy to make it work and then did our blocking as we were defining the model,” says Cirelli.

For scenes of the Lycans tackling the car occupants, production filmed a blue-suited performer who Luma then replaced. Occasionally, entire cars and even characters in the scene were also CG. “A couple of shots of the Lycans biting at the characters were all CG,” says Cirelli, “because we had to replace the actors’ arms and limbs that we being bitten through the windshield. We had to replace parts of the van as well to get proper lighting to cast shadows and that sort of thing.”

Über Lycan’s lair

The vampire character Selene (Kate Beckinsale) faces the Über Lycan in its lair, a stone temple-like setting that pit the two against each other in a relatively confined space. A bust was made of the creature’s upper torso, although for most of the fight sequence and others in the film a tennis ball on a tripod was used to help the actors gauge the correct height for what would be added in CG later.

Production filmed vignettes of Kate Beckinsale dodging massive blows from the Lycan, with Luma working to constrain the action to the small area. “It was tricky in animation to work out how to make this character feel like he is somewhat agile inside close quarters inside a small area, and then keep the pacing alive,” recalls Cirelli. “If you have a creature in a small area, it takes them a few steps to get to the other person. It’s one step and she’s dead! So in order to elongate that sequence, we were always playing with different ideas of how we could sustain the action – whether he’s falling into the walls when he’s jumping to get her, or use his hands instead of walking a few steps. We had to ride a fine line sometimes of him being off-balance but not clumsy.”

The small space also necessitated a unique approach to lighting the scene, especially as on set a number of flickering lights were involved. “They wanted to create this feeling of doomsday with everything breaking down,” says Cirelli. “So all the lights are flickering, and they’re all flickering at random times all over the set and there’s hundreds of them. There were literally PAs flicking switches on and off, on and off. It looks cool in the plate, but it is not so cool if you have to put a creature in the center.”

Luma relied on lighting diagrams and a Nuke-based approach to re-create the on-set lights. “We took the plate, sampled the brightest values, wrapped it on a sphere and that gave us a good idea of where the lighting was coming from,” explains Cirelli. “Then we did some importance sampling to find the brightest points there, then one for flickering. Then we put that into a light inside of Maya, and based on the sample data off the Nuke script. We could feed that right into the light itself, so that the lights could flicker.”

“There was still a lot of hand animation on the light flickers,” adds Cirelli, “where we couldn’t get a good read on the exact value we were trying to obtain – say if you had multiple lights going off at once. Everything was casting pretty sharp shadows, so if the angle of the light was off you’d feel like something was wrong in the scene. So we had to deal with that as well.”

The Über Lycan is seen up close for many of the fighting shots, requiring Luma to deliver some extra detailed skin moisture and drool. “We actually have a ‘drool rig’,” admits Cirelli. “There’s one or two unfortunate animators who are tasked with doing hero drool. Sometimes the client will want a very specific drool happening at a specific time with a very specific action. Our rig stretches and condenses and will break off different little spittles.” Additionally, Luma relied on its internal muscle simulator plugin inside of Maya for rigging the Lycans.

To realize the later final destruction of the Über – in which it is poisoned with a grenade containing silver nitrate – Luma devised a means to have the creature’s skin melt and blister. “For that shot,” says Cirelli, “we have the hybrid where he’s not quite human and we used multiple sculpts of his skin melting and simulations of the skin being pulled down by the gravity. On top of that we’ve got all this procedural shading for blistering, and displacement of the blisters so it looks like they’re popping. Then there are sims of the puss coming from the blisters and pouring down his body and coming out of his eyes – it’s basically general nastiness until he explodes.”

All images and clips copyright © 2012 Sony Pictures.