The tech of PIXAR part 2: Finding Dory – making waves

This story is written by guest writer Leif Pedersen, edited by fxguide.

From the first frames in Finding Dory, it’s clear Pixar’s aim was to make Dory’s world look more beautiful than ever, without losing the hallmark look developed for Finding Nemo. This unique challenge came during the biggest technical transitional point for Pixar’s RenderMan.

finding-dory

The technical ambitions for Finding Dory were vast. Pixar had already transitioned into physically based shading since Monster’s University, but this time they decided to break new ground by fully embracing RenderMan’s new RIS technology.

ris-vs-reyes-dory

ris-vs-reyes-nemo
Finding Dory vs Finding Nemo (RIS vs REYES)

“…We were delivering shots never before seen at the studio.”

 Paul Oakley
 Lighting TD

Departing from micropolygon rendering, which was a cornerstone of Pixar’s signature look for over 25 years was a big undertaking.  Pipeline changes had to be made and even with the artists on board,- fully embracing a pathtracing renderer such as RIS was “a big step forward” says Paul Oakley, Lighting TD for Finding Dory.

“RIS allowed us to reach our look development turnarounds incredibly quickly and scene complexity went through the roof. We were delivering shots never before seen at the studio. For the big pool establishing shot, I was able to approach the shot physically, just like aquariums art direct their tanks. I lit each tank with dozens of lights, simulating LED bulbs, which gave me beautiful rays through the volumes. By approaching my scenes in a physical way, I was able to approach lighting in a whole new way” he explains.

RIS was able to handle very complex scenes
RIS was able to handle very complex scenes

The recent addition of ‘check-pointing’ (partial rendering stop and continuing) was found to be incredibly useful for Finding Dory, because it allows a frame to stop rendering at a specified time. “The days of calculated guesswork are over, now, I can use check-pointing to overview shots for look development two or three times a day, that’s a huge time saver. The more we refine a shot the better it looks.” says Paul Oakley.

“We can do a lot more in less time, which means a more refined image,” says Ian Megibben, Director of Photography Lighting on Finding Dory. “With RIS we get an incredible richness of color thanks to its robust shader system and analytic lights, making lighting subtleties a breeze. This allows us to focus on the art instead of the technology”.

“We came very early on the development of RIS and worked very closely with RenderMan development to get the look we were after” he adds. “We needed to do multilayered shaders with subsurface, refractions and multiple specular lobes and RIS was able to deliver, it was great to get such level of flexibility.” RIS allows multi-pass workflows to be consolidated, allowing the Director to see a final image very early on the look development stages, without having to estimate or guess how global illumination or other complex rendering techniques will affect the shot.

REYES vs RIS.
REYES vs RIS.

With the last release of RenderMan the product is shipping with the same shaders and lights used at Pixar Animation Studios. The Pixar Surface material is an all-in-one shader, capable of combining advanced effects, such as subsurface scattering, absorption and refractions. The RenderMan team has expanded its layering capacity even further, providing tremendous flexibility. These new robust tools allowed artists at Pixar to execute the complexity of Finding Dory and these same tools are free to every RenderMan user.

The consistency coming out of RIS proved invaluable

 Ian Megibben
  Director of Photography Lighting

“We started these advanced shading requests when we tried to make the kelp,” said Ian Megibben. “We needed to have a shader that could do refraction and subsurface seamlessly, as well as behave well inside volumes and through refractive surfaces, it was a great change to avoid multi pass workflows, we no longer approached each shot like a new look development problem. The consistency coming out of RIS proved invaluable.”

The kelp in Finding Dory was a shading challenge
The kelp in Finding Dory was a shading challenge (note the chromatic flare interacting top right).

This ecosystem of lights and shaders allows artist to develop incredibly comprehensive rendering pipelines very easily. This level of complexity doesn’t come at the expense of the artist complexity, because the shading system puts artist friendliness first, by making the light behave much like it does in the real world.

RenderMan Ecosystem of Shaders and Lights
RenderMan Ecosystem of Shaders and Lights

“We can reach our look development and match concept art quicker than ever with RIS.” says Paul Oakley. In RenderMan 21, Pixar Surface Shader lets artists layer complex effects and interact with complex light transports.

At the early stages of development, before primary production, the shading artists at Pixar where trying to simulate water splashes with traditional multi-pass and particle methods for bubbles and fine mist, but the more they tried RIS, the more they realized they could do all this in a single pass.

By letting RIS handle the water geometry simulations and just refract through micro water surfaces,  the team were able to achieve an incredibly life-like results. The luminance inherent in very small and thin water droplets is caused by refractive light, this was achievable thanks to numerous refractive light bounces, something prohibitively expensive to do with previous technology.

Photographic reference vs. RenderMan RIS
Photographic reference vs. RenderMan RIS (note: the thin water sheet effect in the CGI water)

Finding Dory was the first Pixar feature film to use RenderMan’s Denoise, a technique developed at Walt Disney Animation Studios and Disney Research Zurich for the removal of the noise generated by pathtrace rendering. The cleaner the image needs to be, the more rays the renderer has to fire to refine the image, this can take considerable time, especially in the last 20% of the image. This is where Denoise proved most valuable, removing the need for expensive computations in the last bit of refinement and providing a clean image. “The results were surprising,” says Ian Megibben. “At one point we didn’t know if we were going to have the computational power to deliver a perfectly clean image, but thanks to Denoise we were able to deliver Dory in time, it was a real life saver.”

denoise-off

Before and after : Denoise
Before and after : Denoise

Denoise is so effective as it relies on RenderMan for a series of AOVs (passes) in order to understand the scene better. This allows Denoise to clean complex textures, bumps, hair and otherwise noise-like patterns very intelligently. This level of precision is needed to keep your scene consistent throughout animation, without compromising any of the shot’s textural and shading qualities. Best of all, for the Pixar team, they found it was automatic after a single click!

Thanks to Pixar R&D work for Finding Dory, the Denoise now runs on the GPU, freeing CPU cycles and speeding up the computations by a great margin. This technology is available to every user in the latest release of RenderMan.

Denoise needs data AOVs to understand the scene with great accuracy
Denoise needs data AOVs to understand the scene with great accuracy

Another one of the challenges of Finding Dory was the rendering of caustics. Caustics are patterns that form when light is not diffused when hitting a surface, but rather focused into patterns. Think the bottom of a pool with hundreds of lens forming interlocking patterns. Finding Dory used the new VCM integrator (Bidirectional Pathtracing) in order to simulate such patterns. They then baked these patterns and re-projected them, giving the team the artistic control the director wanted.

Caustics give Dory's uderwater world an extra dose of realism
Caustics give Dory’s underwater world an extra dose of realism

Not having to worry about generating caustics artificially was a great time saver over Finding Nemo. The team could trust RIS with physical calculations which they in turn used creatively for their shots knowing the results were going to match the water patterns and other participating models in the volume.

Virtual Cinematography

“…we wouldn’t have been able to do these effects at all if this technology wasn’t available.”

 Ian Megibben
 Director of Photography Lighting

”New in RIS was a set of physically based lenses, which allowed for pixel perfect simulations of lenses. The use of chromatic aberration, barrel distortion, tilt shift and other traditionally expensive filtering operations no longer have to be done in post, they can be simulated in the render. “We no longer had to post process the pixels and rely on filtering” Says Ian Megibben. “In fact, we wouldn’t have been able to do these effects at all if this technology wasn’t available, because the noise caused by pathtracing, as subtle as it might be, needs to be consistent throughout the frame, and blurring the image in post would give us jarring results.”

memory-dory-04

“Keeping everything in-render was a great step to keeping the integrity of each pixel” he adds. This was crucial in developing the look for Dory’s memory sequences, where the background needed to have very art directed lens effect look. “Those shots were final from RenderMan, we only added some defocused foreground particulate in post,” says Ian Megibben.

Switching to a physical renderer also came with many ‘aha’ moments. While developing the water tank sequence with Dory and Hank, the team noticed a very odd effect…Critical Angle and Total Internal Reflection.

The Critical Angle is when the refracted ray is at 90 degrees, creating a sharp switch between regular refraction and total internal refraction. This visual curve can seem like a bug, but in fact, it is physically correct.

Physical rendering acting err, physically...
Physical rendering acting err, physically…

Total Internal Reflection is an extension of this and more complicated to explain, but it is what allows fibre optic cables to not leak light. At certain angles in combined mediums of different index of refractions, the boundary objects tend to behave like mirrors…in the case of Finding Dory it can be seen when looking along the line of a water / glass boundary (1.33 and 1.5 IOR respectively). Again it is physically correct, and modelled by the RIS integrator.

These effects are common in the real world…for example,in addition to fiber optics cables, this is how one gets heat haze or mirages of water in the road of a hot summer road, (the heated air near the ground expands and changes refractive index). This was also happening inside Dory’s tank, as the tank was refracting unexpected parts of the environment and in unexpected ways…although physically correct, it was not the intended art direction…”which was a shame, because not many things make our RenderMan engineers feel all fuzzy inside like accuracy…” joked RenderMan’s Technical Marketing Specialist Dylan Sisson.

To solve this, the shading team manually changed the IOR of certain parts of the tank to coincide with the art direction required, this way the audience doesn’t see reflections everywhere, and importantly Dory does not seem to see herself, as she is trying to look out of the tank.

dory-critical-angle-no-fix
Physically correct but visually confusing reflection.
Critical Angle Solution
Critical Angle Solution : Solved by manually overriding the refractive index of the glass.

Another interesting development in RIS were Light Path Expressions, or LPE for short. These light outputs are more than traditional AOV render passes, they provide full creative control over every light and its corresponding ray type, ie. reflections, specular, caustics, etc… this allowed the lighting team to really customize the lighting visible in each shot, allowing them to track down any artifact causing rays or unwanted lighting, which gave the lighting team incredible control over their shots.

dory-lpe-no-fix

LPEs give manual control over rays
Light Path Expressions (LPE) give manual control over rays (and fireflies – random light ray dots)

The use of LPEs also helped tremendously in eliminating fireflies with the uni-directional pathtracer. Fireflies are hot pixels inherent to pathtracing, usually caused by very intense light sources. These pixel values can be very high, making them vibrate every frame, thus giving a flickering speckle effect to the animation.

Light Groups have been a huge benefit to my workflow

 Paul Oakley
 Lighting TD

The lighting team was able to isolate the contribution of these fireflies and eliminate them for the final render by turning off a specific light path from a light source. Although RenderMan takes many precautions to ensuring firefly reduction, including clamping and russian roulette techniques, having full control over rays provides a level of flexibility needed to ensure maximum art direction and avoids post-cleanup.

The ability to output Light Path Expressions (LPE) in combination with Output Variables (AOV), really allows artists to have full control over their renders within their RenderMan plugin, as well as in compositing.

Another great tool in the lighting toolkit has been Light Groups, which are per-light outputs, which allows lighting and compositing artists to reconstruct the lighting in post and have maximum control in real time, where changes in light balance, intensity and color can be done without sacrificing the render farm. This speedy workflow change, means better use of resources and faster turnaround times for art direction changes. “Light Groups have been a huge benefit to my workflow” Says Paul Oakley, Lighting TD on Finding Dory.

As an added bonus to RIS, Pixar artists found they could use the new VR camera in creative ways for debugging and checking frustum culling. This allowed them to see how the scene was behaving in 360 degrees, giving them an incredible look at what happens behind the camera…literally…

Creative debugging use of the new VR camera
Creative debugging use of the new VR camera

The new RenderMan RIS renderer has set fundamental building blocks with full focus on modern hardware. It’s extensible and moldeable architecture will help it adapt incredibly well to the future of technology and creative challenges to come at Pixar Animation Studios and the top VFX houses.

The move to the RIS Integrator rippled waves of change with Finding Dory and pushed animation at Pixar to a new level of interactive rendering. These changes were driven by a collaboration of engineering and artistry, which is helping Pixar focus on the filmmaking process more than ever, as John Lasseter once said:

“The art challenges the technology, and the technology inspires the art.”

3 thoughts on “The tech of PIXAR part 2: <em>Finding Dory</em> – making waves”

  1. That denoiser comparison looks “fishy”. If that’s truly the before and after, then that’s incredible. I haven’t seen any 3rd party tools on the market that can go from that amount of noise to what was shown in the final frame.

  2. It’s actually a similar method used by a denoiser called Altus. If it’s using the internal AOVs as a guide for the denoise function, that makes perfect sense.

    1. Have you gotten results like that out of Altus? I’ve tried it and it was nowhere close. I’d say that it was good for cleaning up the last 5% of the noise within the image.

Comments are closed.