The newest installment of the Mafia game series from 2K and Hangar 13 was announced recently in this reveal trailer for Mafia III crafted by cinematic experts Blur. We go behind the scenes of the trailer, which shows a re-imagined 1968 New Orleans, with key members of the team and making-of images and video. Blur used real world reference and significant performance capture techniques to bring the imagery to life.

fxg: What was Blur’s overall brief for this reveal trailer?

Dave Wilson (Director): Our brief was pretty open. We love that. And it’s even more rewarding when combined with such a strong IP centered around a fantastic protagonist, Lincoln Clay. The three tiers of the brief were to introduce Lincoln Clay, his lieutenants — his new family and the new world this story takes place in. The most intriguing part of the show was playing with the notion of FAMILY, especially given what the genre typically defines that as. It was really great to put a fresh spin on that concept within the world of MAFIA.

fxg: Can you talk about what kind of concepts, boards or early lookdev ideas were explored?

Dave Wilson (Director): The story is pretty nuanced, so the beauty of it was definitely in the details. We explored it all early in boards and key art. We wanted to go into our facial capture with a clear idea of timing, and were lucky enough to get Alex Hernandez (Lincoln) in early to help us lay the foundation of the spot recording early voice over for the storyreel. It’s all about the monologue, once we knew where that stood, we could build the rest around it.

fxg: Did you produce storyboards?

Jerome Denjean (CG Supervisor): We initially did produce storyboards, for a piece like this it helps with overall pacing and framing. It was especially helpful for all of the shots in the car, it was great to have those before we dove into the mo-cap so we knew exactly what we needed. As a first step we created some production paintings early on that were done for the pitch. These paintings had that early day, end of night kind of quality and tone and we really liked that atmosphere for this trailer. So it was not quite nighttime but right before sunrise. Those production paintings were the cornerstone of everything we did as far as look dev on this trailer.

fxg: How was the mocop shoot planned out, what was the tech involved in terms of suits/cameras/software?

Warren Grubb (Head of Animation at Blur; Animation Supervisor on Mafia III): There were three main phases to the performance capture on this show- layout, face with body, and final body. But first, we locked timing of the entire show with a dialog track cut to a storyreel- this was necessary to weave together the multiple performances.

For the Layout shoot, we built a scale automobile structure with everything from the seat and steering wheel to the gear shift and radio recreated with measurements from the specific vehicle in the game. The actor was then able to perform within all these natural limitations, preventing us from having to “cheat” in animation, adjusting the real performance to fit a mismatched asset. It’s critical because not only does it waste time, it’s an area where the viewer feels something is not quite right, even if they don’t know why. We even provided the actor with a period-correct Zippo lighter so his fidgeting and actions would be accurate, even though we didn’t capture the hands (they were later keyframed), but the fact that he was handling the real, correct item translated into details in his performance that just aren’t there if you try to fake it. Once all this was in place, we captured all the rough body and acting motion using the storyreel with sound on-set to ensure the performance was matched to the subtle and powerful read.

During the facial shoot, we captured audio on-set, but we still referred back to the storyreel and Alex’s first read between takes. He added nuance and some alternates to the performance but we made certain to stay true to the established structure. Even though this session was primarily about the voice and face performance, we captured his body so we could later retain and pair as much of his body mocap data to the broader body performance we’d already captured.

Finally, after our Layout team merged these first two performances, we updated the storyreel with a 3D Animatic and used it to plan a final mocap session where we recaptured any performances that were artistically or technically lacking, including anything where the head performance didn’t quite mesh with the body.

All this planning made the show exceptionally smooth in animation; our team was able to focus on cleaning up and enhancing the performance rather than fighting with technical details.

fxg: Can you discuss directing mocap, both in terms of action scenes but also more subtle shots like driving?

Warren Grubb (Head of Animation at Blur; Animation Supervisor on Mafia III): This was a tough one, probably more so for Alex, than from a directing standpoint. Tough because the acting was segregated. In order to get the fidelity of data in the facial capture Alex had to be seated in front of a bank of cameras, this was fine for driving scenes, but when he’s moving around it gets much trickier. Trickier because — and I remember saying this directly to Alex who was getting up to move around now that I told him he was getting outta the car — “well… no you need to sit in that seat there and today you’ll play your head, and tomorrow you’ll be your body.” It’s tricky. Lines of sight, syncing performances… in my opinion it gets in the way of the acting a little. Alex, turned in an AMAZING performance, but ultimately you want the tech disappearing into the background as MUCH as possible.

We’ve since worked with DI4D to find a way to get both, high fidelity facial capture and roaming body performance all in one, of course audio too. It’s working great. But that was definitely a challenging part of our early collaboration with DI4D. But testament to their team, they’ve worked with us to improve and re-invent… you can’t ask for more from your technology partners.

fxg: How about facial animation – what was the approach there?

Jerome Denjean (CG Supervisor): The DI4D process is great. We have to build our own rigs to get the eye and mouth animation; details like the corners of the mouth, movement of the lips and eyes.

On this project we also welcomed our new character modeling supervisor, Ramahan Faulk who comes with incredible credentials from Digital Domain and WETA, and he did a lot of post sculpting to the facial animation—where sculpting is done, sometimes frame by frame, on top of the animation to make sure that the lips were deforming properly, that there was no penetration and that you get realistic compression of the lips when Lincoln is talking.

fxg: What approach did you take to building your CG characters – were they based on real performers/game assets?

Jerome Denjean (CG Supervisor): We wanted to go into our facial capture with a clear idea of timing, and were lucky enough to get Alex Hernandez (Lincoln) in early to help us lay the foundation of the spot recording early voice over for the story reel.

The DI4D technology is great because if you have the same actor that matches the scan, then you get really realistic results in facial animation that don’t require too much tweaking. All of the main characters and the lieutenants that you see at the end of the trailer were provided by the client along with head scans from the real actors who portrayed them. We made those scans as high res as possible for the cinematic so we could push the render quality and skin detail as far as possible.

fxg: Can you discuss the skin, clothing, hair and overall texturing and rendering solutions for the characters?

Jerome Denjean (CG Supervisor): This was pretty much our standard pipeline and process—all characters were modeled in 3ds Max and Zbrushy rendering out in 3ds Max with VRay, and animation and rigging were done in XSI. VRay 3 has a great skin shader, and for textures we used both Photoshop and Mari. For Hair we use the Ornatrix plug-in for 3ds Max.

fxg: The swamp environments are fantastic – can you talk about reference for this, and also what approach Blur took to digital foliage, water and skies?

Sergej Eichmann (Lead Scene Assembly): Mafia III is set in a fictional New Orleans environment. As Dave made the decision to base the story in the pre-dawn bayou, solely driven by the protagonist’s monologue, we knew our main focus could not be on guns and explosions. Instead we had to make sure our environments were supporting the current emotions of the story. Whether it was the lonely road through the bayou, the flashbacks or the execution clearing, we needed to create a moody and organic environment; a perfect fit with the Louisiana swamps.

Thanks to Jed’s personal collection of photos he had taken on his trip to New Orleans, and our large library including Forest Pack we were perfectly prepared to recreate those swamps. Putting the flashback scenes aside we knew there were two environments we needed to create; while the set in the final scene was more of a standard scenario with multiple cameras facing the same direction, the road through the bayou needed some special attention. Instead of creating miles worth of road we created a setup that allowed us to adjust the environment on a shot-by-shot basis.

Screengrab of road setup; spline driver that controls path shape of the road, while maintaining UVs, and distribution area of the pebbles and foliage set up within Forest Pack.
Screengrab of road setup; spline driver that controls path shape of the road, while maintaining UVs, and distribution area of the pebbles and foliage set up within Forest Pack.
Final shot.
Final shot.

With such an efficient way of creating the environment for all of the close up shots of Lincoln, we had enough time to perfect the lighting. We knew we would need a certain lighting and rendering setup to save big on render times during this process. There were three parts to it:

  • ‘Regular’ shot camera rendering the environment with a custom, unclipped HDRI.
  • The same camera minus traveling direction rendering Lincoln and the interior of the car.
  • A 360 panorama camera rendering a HDRI sequence of A used to light B.

View post on

Above: Rendered HDRI sequence sample.

This workflow allowed us to lock our HDRI created during look development and entirely focus on the environment to achieve the desired lighting and emphasize Lincoln’s emotions.

This image and below - the final shots. HDRI sequence to emphasize Lincoln’s emotions.
This image and below – the final shots. HDRI sequence to emphasize Lincoln’s emotions.
See caption above.
See caption above.
See caption above.
See caption above.

To achieve the right unsettling ambiance we went on a pre-dawn reference shoot through the bayou Playa Del Rey. Not only were we able to learn about the right balance of light distribution inside the car, it also played a big role in finding the proper feel for the atmosphere and color grading.

Reference taken in Playa Del Rey, 430am.
Reference taken in Playa Del Rey, 430am.
Final shot.
Final shot.

Using movie reference we created a color treatment board to visualize how present day and flashbacks contrast each other. While we kept the overall mood dark and brooding, we wanted to subtly exhibit the development in Lincoln’s life leading up to the bloodbath in Sammy’s bar.

RnD Grading; lookdev for the entire pipeline plays a huge role in the cinematic quality we strive for.
RnD Grading; lookdev for the entire pipeline plays a huge role in the cinematic quality we strive for.

fxg: What were some of the compositing challenges in the trailer?

Jerome Denjean (CG Supervisor): The main difficulty for a trailer like this is the time of day it takes place in. This poses challenges because we wanted to imply that everything was dark and moody, but you have to find the right balance between visibility and darkness—especially in something like this where we couldn’t rely on artificial lights. There were no streetlights in the 1960s bayou to help light Lincoln in the car, so not that it was a huge challenge—but it was a major consideration for many creative decisions throughout the trailer. Finding the right balance between look dev and lighting.

fxg: For the transitions to Vietnam – can you talk about how these differed in general approach?

Jerome Denjean (CG Supervisor): Another challenge of the trailer was to take the viewer back and forth through a timeline of different events and not make it too confusing, because there are flashbacks to Vietnam and Lincoln’s childhood in the clip. If we treated everything with the same look, it would have been confusing. There’s one particular shot where Lincoln in walking on a road towards Bourbon City and the color cues really play a big park in alerting the viewer when this takes place. We came up with the cool colors for present day and then as we go back in time we treated the scenes with warm colors and more desaturated tones.

fxg: How was sound and music accomplished?

Tiffany Webber (Producer): The client was very involved from the get go in how they wanted the audio and music to reflect the tone and content of the piece. They worked closely with director Dave Wilson to use just the right music and then on the not so easy legwork of being able to license it. The themes were very important clues to the subtext of the story…from the Italian crooner music Lincoln turned his nose up to, to the comfort and nostalgia Lincoln had with the Vietnam wartime music of Jimmy Hendrix. These all gave little hints to Lincoln’s past and state of mind.

We also worked with Shoreline Studio to help bring the rest of the audio together, and again, the clients were very dialed in and got their hands dirty to make sure everything had the right highlights, hits and subtleties. Overall, a very fun and collaborative process for us on this piece.

Mafia III is set for release in 2016.