I Wanna Dance With (the volumetric capture of) Somebody

 

Whitney Houston: I Wanna Dance with Somebody is a biographical musical based on the life of singer Whitney Houston. It was directed by Kasi Lemmons, written by Anthony McCarten, and it was made with the direct input of music executive Clive Davis, who helped discover Whitney Houston, – serving as one of the film’s producers. Naomi Ackie stars in the film as Whitney Houston, which co-stars Stanley Tucci, Ashton Sanders, Tamara Tunie, Nafessa Williams, and Clarke Peters.

The film’s key crowds were made with volumetric capture which was a first for this approach at this scale. The pipeline was pioneered at Zero VFX by Brian Drewes, Zero’s VFX Supervisor.

Zero used volumetric capture to capture a set of live individual crowd performances. The team volumetrically filmed around 1000 people in 2K and 4K resolution over a two-week period for each of the film’s concerts.

Each individual audience member was filmed separately in both a green screen booth, with an array of 80 cameras and also captured in 360 degrees, reacting and performing to each of the film’s songs. This technique captures the nuances of each crowd member’s performance and facial expressions, and their detailed clothing and props. Each concert had hundreds of separately shot audience members in specific costumes, hair, and makeup, reflecting the year and location of each featured Whitney Houston live performance.

The pipeline that allowed for the selection of 510 minutes of 2K and 4K captures from a thousand minutes of rushes. The 700 terabytes of raw data were processed, allowing control of digital prop tracking and motion blur, which was then delivered into the VFX pipeline.

We spoke to Zero VFX’s Brian Drewes about their work

FXGUIDE: I assume the sprite is a 2D card and the volumetric are comped as 3D volumes? The 3D volume capture allowing for camera moves that circle around members of the crowd, but what format was the Volumetric done in?

Brian Drewes: The crowd characters are full 3d assets of scanned live performances. No 2d sprites at all. Each performance accounts for about 10 secs of animation. Every frame of character animation is essentially decimated geometry with accompanying texture and motion vector information. The challenge with volumetric is that you get geometry and texture data for each frame of animation per character. Although that data set can be large, the result is extremely detailed character models that have photoreal facial and cloth animation built in. This allowed for a crowd asset that could be shot from any angle and very near to the camera. We used Houdini to create the crowd instances which referenced all of the volumetric data from disc. No new animation or data had to be created per shot. The data set did take a lot of upfront organization, but because all the data was referenced the Houdini files were relatively light and easily rendered. By procedurally randomizing attributes in Houdini we were able to create thousands of unique characters. We also had a shader system that allowed for changing colors of clothing, skin, and hair to further increase the number of unique characters. Once the flexibility of the asset was realized, the filmmakers asked us to create entirely new shots from scratch.

FXGUIDE: Which software did you use for both the 3D and 2D. I assume the end comp was Nuke – but upstream of that?

Brian Drewes: The 3d software was mainly V-Ray, Maya, and Houdini. For 2D we did use Nuke.

FXGUIDE: I assume also you would have needed some serious volumetric compression for such a large data project. Was it LOD style depending on where they were?

Brian Drewes: Level of detail was something we had to account for during the scanning process. The majority of the crowd was scanned at 2k resolution. We did a number of 4k scans each day to allow for close-up shots. From there we had an LOD system within Houdini that would swap in even lower-resolution textures for deep background characters

FXGUIDE: How many shots did you guys do? Did Zero do all the crowds or were other crowds done by anyone else?

Brian Drewes: ZERO delivered about 125 shots, towards the end of the project, about 10-15 other crowd shots were handled by Redefine.

FXGUIDE: Thanks so much, and congrats your approach is really effective and worked so well in the film.