Open source Deep Comp Extension (OpenDCX)

digipto-20
Jonathan Egstad

At last year’s DigiPro 2015 Dreamworks presented their extension to Deep Compositing, this year they take this initiative open source.

The technology developed by engineers Jonathan Egstad, and Mark Davis  and DreamWorks is keen to encourage industry adoption of the technique.

DreamWorks Deep Compositing Extensions allow anti-aliased compositing of OpenEXR (openexr.com) deep image files with a mixture of solid and volumetric elements, while keeping the memory requirements to a minimum. The addition of subpixel masks to each sample allows for coverage and opacity to be treated separately, without an excessive increase in memory consumption, and with the use of the new “hard surface” flag, the included flattening algorithm allows for any combination of volumetric and solid elements to be merged with plausible antialiasing.

The extensions provide the ability to apply affine image-space transforms to the deep image data, taking advantage of the sub-pixel information within each sample. Finally, a sub-pixel accurate pixel filtering method is provided to allow higher quality image reconstruction.

“FX at DreamWorks has utilized deep images for many years; however, Kung Fu Panda 3 was the first feature where Lighting fully embraced the technology. Previously, deep pixel information proved problematic in reconciling high-frequency features like hair and fur with intersecting hard-surface geometry, and flattening a deep image did not produce the same image as a full render,” said Jeff Budsberg, Head of FX on Croods 2. “With deep subpixel masks, an efficient EXR channel encoding, and a custom deep image flattener, the DCX technology yields a production-friendly solution for full deep compositing that resolves previous visual artifacts, while only requiring a small increase in disk footprint.”

“Adding subpixel Mask support allowed us to jump between a deep image workflow and the old workflow without a noticeable loss of image quality.” said Matt Titus, FX Lead on Kung Fu Panda 3.

“Our digital production requirements continue to drive industry-leading innovation in our technology,” said Andrew Pearce, Director of Technical Strategies at DreamWorks Animation. “We are honored by the recognition of the technique, and pleased to continue to work with industry to extend our collective capabilities.”

Deep already works very well with combining two volumetrics but there can be edge issues when combining a volumetric with a hard edged surface model. This is due to the lack of effectively anti-alising in the deep data. With Deep Imagery the 2D image has the capacity to store multiple color values and of course deep samples but it lacks any form of sub-pixel spatial information to resolve overlaps – so a deep sample lead to aliasing for the hard object transparency and anti-aliasing or sub pixel coverage in the same alpha at the edge – once combined it can’t be re-separated.

DreamWorks worked out a way to add sub-pixel masks (normally 8×8) one mask per deep sample (so 64 bits 0-63 :8×8). But the DreamWorks solution is to have a fixed mask 8×8 – the problem is OpenEXR is designed to store floating point numbers not masks.  Bit-pattern storage is not natively stored in OpenEXR, “so we had to hack OpenExr a bit to make it work.”

The system has two special cases: all bits off and all bits on (for full coverage). This means in reality one only ends up having to use the data on the edges of objects. The team decided that while one could argue for variable bits ( 4×4, or higher 16 x 16), it was just not viable to have such a variable mask format and so after experimentation settled on 8×8. The new system helps with pixel filtering, hard surface blending and matte object handling.

The next step on DCX “is to get more the renderers and the compositing products moving in sync” comments Egstad. Mantra is supporting it now, “and the aim is to get RenderMan, Arnold and Nuke to all buy into implementing it”.  The Foundry’s Nuke is key to the success of DCX as it is the primary high end compositing tool, Nuke 10 actually doesn’t have advanced deep tools but there is a hope these sorts of advanced Deep extensions will appear in Nuke 11. While there is no hard or fast rule, given the image dependent nature of the system, this process if implemented “could meant that there is a 50% reduction in deep cache pipeline” he estimates.

OpenDCX is available immediately at opendcx.org.