Monsters University: rendering physically based monsters

s101_1_pub66Pixar Animation Studios has once again broken box office records with the great new Monsters University. Interestingly, while the film is breaking new ground and adding new characters to the Pixar universe, behind the scenes the talented Pixar and RenderMan teams were working hard to provide new tools to allow the factious university students to be rendered more realistically than ever before. This time around there are many more monsters and a much wider range of background characters and environments. But the Pixar team Also used new physically based lighting and shading models to produce the exceptional images with less lights, greater accuracy and in a more artists friendly way.

A studio like Pixar can invest in serious pipeline developments such as moving to primarily ray tracing as they have both the longer studio planning horizon that a Disney-Pixar studio plan allows for, and of course the RenderMan development R&D team are also part of Pixar, producing newer and better tools all the time. But the process of improving the Pixar film production is not as simple as just using the latest version of RenderMan. The RenderMan team have developed RenderMan into a powerful hybrid renderer that can be used in a variety of ways. The animation part of Pixar still needed to decide how they were going to use RenderMan and which part of this award winning software to focus on. MU once again emphasizes the incredible relationship that exists between the team of RenderMan developers and RenderMan practitioners at Pixar.

s111b_28pub158The characters in MU needed to be de-aged, which meant giving Sullivan’s hair a younger “bed-hair’ style treatment. While Sulley had been rendered with hair years before – a landmark at the time – the newer Sulley would be more refined than the earlier version and only one of several very hairy primary and secondary characters.


We spoke to Christophe Hery, Global Tech and Research TD, at Pixar about the move to the newer approach and his own journey joining Pixar just in time for MU.

 Christophe Hery, Global Tech and Research TD,  Pixar
Christophe Hery, Global Tech and Research TD, Pixar

Originally the project was going to follow more traditional Pixar rendering approaches, but it shifted not long after Hery joined Pixar in June 2010. Ex-ILM, Christophe previously served as an R&D lead, supporting the facility’s shaders and providing rendering guidance. He was first hired by ILM in 1993 as a senior technical director. Hery received a 2003 Technical Achievement Award for the development of practical methods for rendering skin and other translucent materials using subsurface scattering techniques. But as he had also received a Scientific and Engineering Award for the development of point-based rendering for indirect illumination and ambient occlusion (along with Pixar’s Per Christensen), it would not be unreasonable to think when he joined Pixar he would be continuing with this work. In fact when he interviewed for a position at Pixar he had no idea it was related to Monsters University.

Watch Mike and Sulley’s first encounter.

 “I had personally never done fur…so I was surprised then when I was on Monsters University and they said OK now do fur!”

  Christophe Hery

When he was hired the team was looking at new ways to light and render MU, which included looking at point bleeding and irradiance. They were also looking at spherical harmonics (SH), along the lines of the systems being used so well at places like Weta Digital. Surprisingly then, Hery said “no, no no … now we should do ray tracing.” The film started being rendered with RenderMan 16, but finished with shots being done in RenderMan 17.  The plan was not based on making the rendering run faster, it was all focused on improving the quality of the shots, and reducing complexity. As sometimes under the previous setups, there were hundreds of lights in a shot, “it made the job of directing the shots much more complicated, it made the communication between lighters – passing around lighting setup – much more complicated and everyone had to relearn each time by simply turning lights on and off and seeing the effect they were having,” says Hery. The team wanted to simplify that process. For Hery that meant moving to physically based rendering, “and it was something that I had done at ILM before, so I was pretty confident about it.” In particular it meant that area lights would not work unless the underlying system was built with energy conservation, and textured with the correct BRDFs.

Pixar did want to try new things, but Hery did not know that when he joined. “That is why I joined Pixar at this time, the opportunity to change the way they were doing things and the people in charge of the movies were keen to do that, and I was pretty clear in the interview, we had done that on metals and surfaces, we’d done that on Iron Man in particular and in Transformers, but I had personally had never done that on fur…I was surprised then when I was on Monsters University and they said OK now do fur!”

See a clip from the film.

Key to the solutions was extending the MIS to cover the complex transparency and lighting issues around a character like Sullivan (Sulley). Actually one of the first things Hery did was explain the new shading model for hair and for surfaces that he was coming up with. “I replaced what they had for many years before that,” he comments. It took a little amount of time for the texture and lighting artists to learn the new range of values and what they meant. “It was more like they had muscle memory from what they had before, but once they got it – in a couple of weeks – they were really happy and it made more sense to them, especially in the way it would react (with light).”

One of the biggest changes involved removing the separate control for specular and reflection. Under the new physically based lighting and shading model these are linked. Before “in particular they had a Kr parameter (ed.* (ideal mirror = 1) reflection multiplier : floating point value) independent of the Ks (ed.*specular reflection coeff.) parameter – and they had a sort of blur value for reflection tracing and a roughness for specular…and I was telling them – that is the same property, you really don’t want two values there, that is when you start drifting and you won’t have materials working in a perfect ‘canonized’ environment,” that is, when you have separate overlapping controls. Of course, the Pixar look development team is some of the most highly trained and skilled in the world and “they got used to it fairly quickly and they never regretted it and never asked for it back,” Hery says.

s165_32pub.pub16.181The team did continue for a little time after Hery joined in parallel with point based approaches, and exploring options like SH, but in the end the new physically based lighting and shading replaced all the other methods. As Hery points, out SH and point based are really good for diffuse solutions, and you need to use some other method for the specular component. “Once you move to physically based frame work, once you do specular, your diffuse is almost for free,” he explains. “Because part of computing the specular samples – you have to do light sampling, which is what you have to do for diffuse”.

Key to this approach is multiple importance sampling (MIS) that provides importance sampling to both the lights and the materials. “You wanted something that almost worked right out of the box,” says Hery. This extended to simplifying the UI into light sampling. Pixar aimed to reduce complexity and provide an easier artist interface, even if it increased render times – so long as the shots were able to be better –  and yet some of these approaches proved to be significantly better and reduced render times.

While it is true the aim was not to make renders faster – the aim was to make the users experience better – a key by-product is that to get a final shot done the pipeline now requires less passes and also “less time managing them – they kinda of add up,” Hery says.  “You don’t have to do all those passes, and they add up, it is true if you render from scratch potentially a ray tracing global illumination render would take longer, by far potentially, but if you sum up all those passes it may not actually be slower at the end of the day.” This is where progressive rendering is important. To give quick feedback, progressive re-rendering allows an artist to move a light without having to compute all the samples – “you want to give quick feedback, you let it compute a nosier version first and then it updates.” The Pixar team therefore made sure to explore partial renders for feedback on MU.

s315_10gpub.pub16.769Interestingly, in an effort to hide from the artist complex numbers, the level of samples on the lights was an explicit value, but the number of samples on materials was derived from a heuristic and be part of the BRDF “based on the roughness of the material,” says Hery.

We asked Hery what sample levels were typically used on MU in this setup with such an efficient MIS implementation approximately? “In practice the specular model, given there is a difference between camera renders/rays and secondary (after the bounce you reduce the number of samples), in primary generally the specular would go between 16 and 48 samples – based on the roughness, the lights – it sort of depends on their solid angle, on the observed size from the micro polygons – so they would also have a min and max, usually from about 16 to 128.”

Hery points out that on the lighting side the team can’t predict occluders, they can’t predict shadowers so one can’t make an absolute heuristic, while it is possible to predict a value that will not have perceived noise, it can only be done in the absence of occluders – since until you have rendered that angle you cant know, in effect it is a ‘chick and egg’ problem.  While sampling according to lighting is preferable, any occluder blocks many of the directions that contribute light. Ideally, one would not pick directions that are blocked by the occluder. Unfortunately, this cannot be easily achieved for arbitrary scenes and geometry, unless you have already solved the scene.

Dean HardScrabble : the only character rendered point based
Dean HardScrabble: the only character rendered point based

The film also introduced new characters such as Dean Hardscrabble (Helen Mirren). This new character exhibits a remarkable range of textures and in particular sub-surface scattering on her wings. Interestingly this was one of the only tasks still rendered with the point based pre-based solutions. All the characters had the pre-pass done for them, explains Hery, “but all the shows since then (MU) are rendering SSS with ray tracing and potentially with the new photon beam diffusion (Link) method – it is an option for our shaders now but MU was too early to use that.”

And yes Professor Knight’s lecture hall is room A113, a reference to a room at CalArts where animation classes are taught – all Pixar films have a reference to “A113” and ex-Cheers Star John Ratzenberger voices Yeti, having also never missed a Pixar film. 

The Blue Umbrella
A scene from The Blue Umbrella.
A scene from The Blue Umbrella.

Continuing recent tradition, Monsters University is preceded by a Pixar short – The Blue Umbrella directed by Saschka Unseld. Head over to our fxguidetv interview with Christophe Hery about the film in which a live action aesthetic – and tools – helped bring the story to life.