RenderMan tackled some of the most complex rendering scenes yet attempted by Pixar, in the Oscar nominated Coco. The 2017 film is based on an original idea by Lee Unkrich and is directed by Unkrich and co-directed by Adrian Molina. It tells the story of a 12-year-old boy, Miguel Rivera, who is accidentally transported to the land of the dead, where he seeks the help of his deceased musician great-great-grandfather to return him to his family and the living. The concept of the film is based on the Mexican holiday of [el] Día de los Muertos or the Day of the Dead. In the ‘afterlife’ environment the team at Pixar needed to produce a layered and complex environment with an incredible array of lights.
The main afterlife city of the Land of the Dead in Coco is based on the real world city Guanajuato in Mexico that the team visited during pre-production. Director Unkrich conducted an early research trip to Mexico with Pixar head of story Jason Katz, production designer Harley Jessup and Pixar producer Darla K. Anderson, visiting Mexico City, Morelia, Oaxaca and Guanajuato. Guanajuato’s brightly terraced architecture on steep hillsides became the dominant reference, both for its colourful houses and the view at night when the hillside lit up.
The Pixar city extrapolated the hillside idea, with The City of the Dead’s layout being very much a vertical city, which contrasted with the film’s depiction of Miguel’s very flat hometown.
Interestingly, Pixar’s team came up with a set of rules for the design of the city. In addition to being vast in scope, it was built for the dead, so there are no handrails or safety barriers, as the inhabitants are already dead. The city is a mix of styles and periods representing its citizens and their timelines.
One the most complex aspects of the film was the scope and use of lights. 29,000 lights were used in the train station scene. The cemetery had 18,000 lights and the City of the Dead used 2,000 practical lights in RenderMan. The studio team used the RenderMan API to create 700 special point cloud lights, which, when expanded, is equivalent to 8.2 million lights.
Above: Our fxguidetv exclusive look at the City of the Dead.
Lightspeed team
At Pixar there is a special ‘Lightspeed team’ for each film. These teams sit between the tools team and the production, and they focus on solving project specific problems. For Coco the lead was Renee Tam. Her team was responsible for anything that might go wrong or slow down the film’s production. Her team was also responsible for any project based R&D. At Pixar, if something is found to have an unacceptable long render time then the Lightspeed is given the task of solving the problem. Render time issues can involve aspects in RenderMan or how the production is working with RenderMan. “Our department is responsible for research and development work to bring down the render time, or write new tools in the production pipeline to streamline whatever is needed”, explained Tam. In addition to the core Lightspeed team, the RenderMan team normally assigns a couple of their own team’s engineers to help with the enormous complexity of any Pixar film, This also allows the team to take lessons learnt from real world production back into the general implementation of RenderMan.
For example, when Coco first started the team looked at concept images from the art department, and the vast amount of lights in the concept art. “We were like ‘Holly cow, this is something we have never done before” recalls Tam, “And so at that very moment, we started working with the RenderMan team, and they were awesome”. The team outlined the seemingly impossible problem of rendering shots with so many lights. “At the very beginning we told them that Coco would need “a lot of lights”, but I never told them what that actually meant,” Tam laughingly recalls. The problem was not just the number but also the complexity as the production team wanted to maintain control over the vast array of lights, including the ability to have gels, gobos and blockers (barn doors), in fact anything that generic lights could have in RenderMan normally. In the old PRMan the vast lighting plans in the Coco storyboards and concept art, would have been impossible to implement. The Lightspeed team did a test with the old PRMan and it was beyond unworkable for what the production needed, in fact, the tests could never successfully complete a render. “So the RenderMan team went back and thought about the problem and came back with a proposal for a new lighting schema that would lower the render time substantially.” However the team hadn’t actually seen any concept art themselves. “Anyway they came back after a little time and we tried the new schema, and we told them great – it does now render but it takes 1000 hours a frame to get anything that looks like something we might want, and it was still really noisy.” she explains. “When we showed the RenderMan team they were confused, – why would it take so long they asked? And then they looked at our setup and said ‘wait a second – how many lights do you call ‘a lot of lights’ and I answered; at last count we had instanced our lights out to about 8 million lights. At that point we all went into a second round of R&D!”
The team did solve the problem, together they worked out a new point cloud lighting system. The new approach dispensed with some filtering, it also ignored lights that, due to position, could not contribute, and it reduced complexity on distant point cloud lights. The giant point light set system takes advantage of an Octree system that sampled the lights more efficiently. It then integrated with a new lighting picking system which focuses on the closer lights to have more articulated controls. After 6 months the Lightspeed and RenderMan team had a system that gives the effect of millions of lights and took the notional render time on the complex shots down from 1000 hours to 450 hours. The team continued and reduced this further to 125 hours and finally 75 hours a frame. With some additional work on the way the production team worked with the lighting in shots, the final per frame time at the end of production was just 50 hours per frame.
The Lightspeed team did other innovative lighting developments. Tam explained at Siggraph Asia last year how her team also took advantage of the Firefly special tool, that they had developed on The Good Dinosaur, that could bind sets of different lights to the same object. This was then improved for use in Coco: City of the Dead scenes, and it proved very effective. Other improvements from the Lightspeed team went into Presto and other areas, not render related.
RenderMan
RenderMan’s current version is 21.6. The RIS technology in the current RenderMan is a highly-optimized framework for rendering heavy geometry, hair, volumes, and irradiance with incredible efficiency in a single pass.
RenderMan has established itself as one of the most flexible, powerful, and reliable tools for rendering cinematic imagery. RenderMan’s RIS framework was designed from the outset to trace rays as fast as possible. With a new framework consisting of integrators, BRDF’s (written in C++), and intelligent light services. RIS is allowing RenderMan to set new standards for both speed and memory efficiency when rendering complex production scenes such as the ones In Coco. RenderMan v22 is expected to be released and shown at SIggraph 2018 later this year. RenderMan’s next film from Pixar is the Incredibles 2, to be released in June.
The film Coco has received praise for its animation, vocal performances, musical score, songs, emotional story and respect to Mexican culture. It earned numerous accolades, including Best Animated Feature Nomination (and Best Song Nomination) at this weekend’s 90th Academy Awards. The film has already won the 2018 BAFTA for best animated film.
