“Katana has found a strong home as a rendering / lighting lookdev system with the people that are mainly doing photo-realistic visual effects, but also companies like Pixar with the high end animation market,” new Foundry Product Manager for Katana, Jordan Thistlewood recently told fxguide. Jordan has been with the Foundry a while and was former Head of Creative Specialists.
In talking to Thistlewood about Katana it is apparent that while it is currently a tool for high end effects houses doing volume work at the highest level, The Foundry want to see Katana move to a wider audience, and the move to port the software to Windows is a key stage in that new plan.
“The work that went into growing Katana between 1.6 and 2.0 left us with some nice strong nice technology that allows us to use Katana’s power in different ways and possibly different products.” – Jordan Thistlewood
Katana has been already moving into new areas. At FMX last year it was shown being used for research into on set workflows (Dreamspace), and at SIGGRAPH, Pixar showed a remarkable realtime GPU version they had been experimenting with. But while the tech may expand, the core product is still very much aimed at look development in visual effects and animation, most often with RenderMan or Arnold as the render engine.
It seems that The Foundry intends to make “multiple tools out of that technology”, but still very much push the high end with new releases such as Katana 2.0, which first appeared in May last year.
MPC is a major user of Katana, but only moved to 2.0 a couple of months ago, given the nature of film pipelines and not wanting to do a major upgrade mid-show. David Hirst, Global Head of Lighting at MPC based in Vancouver, has been at the forefront of their work with the product. For him the multi-threading and improved performance is a big win for MPC’s artists as well as the improvements in the OpenGL.
Internally, The Foundry has devoted new resources to Katana, it is bigger than it has been in the last two years – currently its development team “is not as big as the Nuke team but bigger than Mari, Flix and it is the right size for what we want to do with it,” says Thistlewood.
The product is popular in London, Vancouver, California but also in Sydney, Singapore, France and even China. Mostly in major effects and animation companies. At the moment most users are larger houses with serious teams developing API based plugins of their own, with Python scripts, C ++ etc, and while the new version enhances the API, the tech under the hood is very much aimed at making the tool accessible to smaller facilities with less coding and more straight pixel crunching productivity right out of the box.
Katana 2.0
There has always been two big aspects to Katana in our opinion.
- Managing large data sets – with vastly faster loading and asset/memory management
- Better more interactive lighting TD work once the file is loaded
Now file loading may not sound sexy but pre-Katana, some lighting TDs would have two computers – one would load a scene while the other allowed editing on a previously loaded setup. Artists would A – B between machines as the load time was not seconds or minutes but closer to hours.
For example, when MPC was working on Guardians of the Galaxy, Katana excelled. “When we were working on that film we had 3000 spaceships and we could manage those scenes well, we could organize the scenes logically and get send sections to PRMan without having to worry about expanding or holding any of it in memory,” explains David Hirst. “It was perhaps the first film that really showed the sort of flexibility we need on shows, which we had struggled with prior to that in just Maya with our own RIB generator pipeline – Maya is great to a point, and then it struggles.”
Interactive time means just a better job – in the end our ability to iterate defines how well we can refine anything. The new Katana provides some big obvious changes – namely a Windows implementation but the main technical improvement comes from a new scene graph engine – which is central to Katana.
Geolib3 (- dull name for a great turbo boost in performance)
At the heart of these changes is a brand new implementation of the scene graph processing engine Katana uses to build scene data: Geolib3.
Geolib3 builds on the Foundry’s production experience and the earlier version but leverages modern hardware with multiple CPU cores. The most noteworthy Geolib changes are the persistence and re-use of data while working with Katana’s node graph, and the ability to adapt Katana’s behavior to make best use of CPU resource and memory depending on the scene. Geolib3 allows for asynchronous and concurrent evaluation. It also expands the kinds of custom nodes a facility can develop themselves while providing a range of general improvements to the whole product. The previous Geolib2 did not have a persistent scene graph data model. In the original version of Katana the entire Scene Graph was reconstructed on every edit. Now with Geolib3 the OpTree is persistent allowing for re-use…which means a turbo boost for a user.
It really is a key change for an artist using Katana that the UI scene graph processing is now carried out asynchronously on another thread. This means far fewer blocks or stalls in the UI while working on a project. Which means a more responsive system.
“The work put into doing the shift to GeoLib3 is at the core of the performance experience people are seeing in the new release – when they are expanding and collapsing scene graphs, doing the nitty gritty operations, traversing the scene graph, all the operations within the program that deal with the geometry and deal with the scene graph are now faster,” outlines Thistlewood.
New Gaffer Node Governor!
A new Gaffer node type, named GafferThree, has been created to provide improved performance when dealing with large numbers of lights in Katana projects. The GafferThree implementation takes full advantage of the new scene graph processing library Geolib3.
The significance of the new GafferThree node is that an artist can now really have a hierarchical lighting plan, in controlling your scene. “You can now do a lot of work in establishing the sequence lighting, the look of a shot,” says Thistlewood, “and then import that – through a live group – as a reference. Using a GafferThree node you can then edit, as an override, the lighting that comes through that referenced file – on a parameter by parameter basis.”
This means if you want to adjust the color of the key light that comes from the light group that referenced ‘sequence lighting’ – you can do that on a shot by shot basis or on a more hierarchical basis. And if say the direction of that light has changed in the sequence lighting that flows through – it is not a blocker – it is just an override.
The legacy Gaffer node type from Katana 1.x is still present, and previously created projects should still work, but it is advisable to move to the new version. For example, for those using Arnold as your renderer, the way that Sky Dome items are implemented has changed. Instead of an ArnoldSurfaceShader of type skydome_light on the item’s Material node, materials on Sky Domes are now resolved internally in Gaffer.
Making the modifications on reference lighting from other scenarios is a big plus for Lighting TDs doing large volumes of high end work in a big modern lighting pipeline. It allows for major re-use and reworking of Katana projects over multiple shots or scenes. On one shot the software may not be faster but re-using your setups over weeks or months of work makes a vast difference.
As well as performance, GafferThree node type includes:
- Lights, rigs, and master materials can be created and managed underneath an any root location in the scene graph. This allows the creation of lights under separate branches in the scene graph.
- Adoption of lights from the incoming scene. This allows a GafferThree to edit the material, geometry, linking, and transformation parameters, and also mute and solo state of lights, rigs, and master materials created in upstream GafferThree nodes.
- Ability to add child lights, rigs, and master materials under adopted rigs.
- Soloing a light in a GafferThree node affects downstream lights created by other GafferThree nodes.
- Parameter values of multiple selected items in the GafferThree object table can now be changed at once by changing the parameter value for one of the selected items.
- Color swatches in the Color column of the GafferThree object table in the Parameters tab are shown with filmlook visualization/display transform turned on.
- The Linking tab provides fine-grained control over light and shadow linking.
For those companies that are into programming and extending Katana it is possible to really integrate Katana into a pipeline. An example of such an extension allows for crowd animation from a Massive simulation and then binding, skinning and everything else inside Katana. MPC has its own RenderMan PRMan Plugin. “The nice thing about Katana is it is very flexible about the plugin architecture,” explains Hirst. “The thing about Katana is that you can think of it as kind of a mini-pipeline inside your pipeline, so you can make your own RIB generation, and at this point you have a lot of ways you can leverage that advantage in your workflow.”
Over time MPC has evolved its workflow thanks to Katana and grown in complexity from what they had initially assumed would be a more linear straight forward pipeline approach. For example, MPC is now exploring more sequence level lighting in a hierarchical sense. Hirst is clearly a fan of the product. “It is very logical to an artist, in much the same way that a node based compositor is, artists generally just get it much easier than our old pipeline,” he adds. “The LookDev workflow has been quite an improvement for us, the ability to having edits of child materials with overrides has lead to a nice workflow where we can say create a master shader and then create child materials which then adaptions of that master shader – that has worked very well for us.”
The lighting pipeline at MPC is global so Katana is used in all of MPC’s locations/offices and in principle any scene can shared across any offices thanks to a huge investment in infrastructure that the company has made over the last few years.
Katana is now targeted for two releases a year, with the major push around SIGGRAPH (Mid-year).
The next step for Katana would be to extend it further into animation and make it the industry standard platform for render lighting pipelines in VFX and Animation. To do this the Foundry will need to provide standardization, make it easier to get going, avoiding a lot of the complex setup issues involved with putting in such systems and broaden the base. TDs and artists who know Katana well are in demand, but the product still has nowhere near the wide industry appeal of its Nuke brother. If The Foundry can lower the barriers to entry and build on this amazing scene management tool, to make it more artist friendly and ‘creative’, then the product could grow the size of its user base in the mid to lower level facility market especially in higher volume environments such as say, episodic TV effects.
For those artists wanting to break into doing high end work on Katana – we have new Katana 2.0 courses available over at fxphd this term (January 2016 term). Fxphd’s new Katana courses are designed as an update to the original KAT101 and KAT201 produced by Matt Leonard back in 2013. Matt works at MPC in Canada and has taught Katana internationally. This first course is a primer and will start by looking briefly at how to properly import data into Katana from Maya and have it organized in a way that utilizes Katana’s powerful ‘Collection Expression Language’. From there it explores building and assigning Materials to geometry, creating lights via the Gaffer, setting up the renderer and defining render passes.
The other course will cover all the aspects for the look development and lighting of a truck in Katana and RenderMan 20 RIS and integrate it in a shot. This second course is by artist Charles Chorein, who is a Lead LookDev/Lighting at MPC with 10 years of experience. He has been working in London since 2009 and has worked on various movies such as The Hobbit, Prometheus,The Hunger Games 2, Man of Steel, Harry Potter 7, Pirates of the Caribbean 4, Despicable Me and recently Terminator: Genisys. He is currently involved in the amazing new film from Disney: The Jungle Book.
