Voodoo magic

Think back to some of the most outstanding creatures in recent films, and Rhythm & Hues will come up as one of the key visual effects studios behind that work – Richard Parker in Life of Pi or perhaps Aslan in The Lion, The Witch and the Wardrobe. Hundreds of talented artists worked on these complicated creatures, a feat that was aided by Voodoo – Rhythm’s proprietary extensible toolset for rigging, animation, camera tracking, matchmoving, fur, crowds, and 2.5D work.

In fact, Voodoo has been so influential in helping to bring to life Rhythm’s character animation and visual effects that it was recognized this year with a Scientific and Technical Achievement Award from the Academy of Motion Picture Arts and Sciences. That award was presented to Peter Huang and Chris Perry for their architectural contributions to, and to Hans Rijpkema and Joe Mancewicz for the core engineering of, the Voodoo application framework.

With some upcoming presentations on Voodoo taking place at SIGGRAPH 2014, we sat down with several members of the Voodoo team to discuss the framework’s key attributes, where it’s being used in newer areas such as facial animation and whether it’s for sale. And we’ve posted several videos demo’ing Voodoo, including an exclusive look at facial authoring inside the toolset.

Watch Rhythm & Hues’ Voodoo overview video.

What is Voodoo?

On a feature film with hundreds of visual effects shots and potentially hundreds of characters, or even just one complex character, it is of course a pure necessity for multiple people in multiple locations to be working on the same shot or the same character – at the same time. What’s more, to maintain an efficient workflow, changes made to a shot or character need to be propagated throughout the pipeline as models or rigs also change to meet the film’s demands. That’s where Voodoo steps in – it allows artists to work in parallel across rigging, animation and character effects.

“It was our talking animal work that really pushed Voodoo,” explains core voodoo programmer Hans Rijpkema. “The thing is, it required lots of different people working on the same character. So the first thing that needs to be done is stuff needs to be tracked – you need to lock down the camera movement and then matchmove the face. That pushed the matchmoving and deformers. Then you need to get an animator to work on top of that. If something just doesn’t feel right, is it an error in animation or the matchmove or tracking. That’s where our pipeline and passing things back and forth and from one to the other – a lot of that work came from that. So a matchmover can still work on a rig, have their own rig and their own controls, and keep pushing out new versions of it. And then an animator, even though they already started animating, can get updates and get corrected track and matchmoves.”

Delta Mush – a Voodoo deformer that smooths arbitrary deformations applied to a polygonal mesh, without smoothing the original detail of the model.

What makes Voodoo so useful in realizing these components of visual effects creation, then, is its referencing system. “A Voodoo file is essentially a very lightweight scene description,” says Voodoo programmer Cyrus Wilson, “and this is part of why we can have other tools use what’s been set up in Voodoo, and then leverage that. Voodoo files can reference other Voodoo files. You can have subscriptions, publish new assets and people can just get them because they’re referencing the other files.”

What can you do in Voodoo?

Voodoo is now a set of tools running on Linux. It’s main components consist of:

– rigging tools
– fur creation via grooming, animation and dynamics
– crowds, including crowd rigging, manipulation and flexible body dynamics
– computer vision, including camera tracking, match moving, rig tracking and geometry reconstruction
– matte painting and environment projections (via the studio’s 2.5D tool called Rampage)

Voodoo is just part of the full toolset at Rhythm & Hues and does not cover all aspects of visual effects production. For example, Rhythm relies on its Crom tool for lighting workflow, its Icy tool for roto, paint and compositing and on its proprietary renderer Wren for rendering. Voodoo is renderer independent, and so also works with RenderMan and Arnold, for example. Says Voodoo lead Sherie Bradfute: “We basically made a headless program that can query Voodoo for various details that the renderer cares about. The renderer itself looks to that to determine what it needs.”

Rigging multiple characters is one of Voodoo’s strengths, especially applying already developed rigs to new models. “We are not afraid to take in a different model once we have rigged this whole thing,” notes Rijpkema. “There’s a couple of very central components but our rigs are almost completely model independent. We might rig a character and then if you have a model that has a completely different topology but is roughly the same shape, it will just work. Traditionally in a lot of systems, people have to lock off a model or jump through hoops when there’s a new model – we just say point it to a different model and it’s there.”

The rigged skeleton is then used by the animation department, but there will also be a tech rig built in Voodoo for ‘tech anim’. “That’s got muscles and cloth,” says Wilson. “We use our cloth sim for skin and that’s set up to be driven by the original skeletal rig. But that doesn’t have to be driven by the animation curves and the rig itself. It can be driven by a cache that’s exported by the original animation.”

Rigging demo with hoofed character.

A layered system in Voodoo is used for muscle and skin movement. “We generate this feel of muscle over skin over fat, as well as loose skin,” describes studio rigging supervisor Matt Derksen. “So we utilize our system of layering deformation, how we can actually first bind the skin directly to the muscles and those muscles can actually be generated either by a modeler or by our package, and we bind those muscles to a skeleton and then we bind that skin to our muscles.”

“From there,” continues Derksen, “we can actually solve on top of that using our cloth solver that allows us to shrinkwrap the skin to those muscles, allowing for an off-set from those muscles, which is the fat layer. From there we do a secondary solve to create loose skin. Because we never want to actually generate or try to solve all of that in one solve, because in a big production environment it doesn’t actually work very well – and they have to re-solve one giant cloth solve. So our solve is a shrinkwrap of the skin to our muscles and then we do a secondary solve that allows the skin to fall away from our muscles. We can actually determine where the skin falls away from those muscles using our ‘home’ influences.”

Body model swapping.

This reference to ‘homes’ is, according to Wilson, “an abstraction where you can have different ways of weighting things procedurally – it can be a spatial fall-off. You could if you really wanted to paint weights but no one does because why would you if you don’t have to.”

Simulating hair and fur is another principal Voodoo component. “In 1999 we started doing long hair for Cats & Dogs and using the concept of guide hairs,” says Rijpkema. “We have always seen the guides as just the guides for the shape – they do not necessarily determine where hair grows or clumps. Guides, clumps and placement can be controlled independently of one another.”

Character authoring in Voodoo.

“It has a layered approach in terms of how you control it,” adds Rijpkema. “You can have shapes coming in in Voodoo. You can put blend shapes on guide hairs if you wanted to. All the controls on the hair – how wrinkly it is, how thick it is, can be controlled – you can paint it, animate it with home influences. If hair has to grow from the front to the back we can just animate a home over it. Noise levels. Strength. There’s lots of parameters.”

For 2.5D matte painting and environment creation, Rhythm’s Rampage tool acts as a plugin to Voodoo. It features a layers-based experienced and takes advantage of GPU accelerated shaders to visualize complex environ set-ups. Rampage’s development came from early Rhythm work on projected textures for talking animals. “One of the themes with Voodoo is having these very modular building blocks that you can re-use in other places,” says Wilson. “Some of that stuff found a new life in our matte painting pipeline via Rampage.”

A demo of Rampage.

In addition to these and other areas of the VFX pipe, Voodoo has been adapted to carry out extensive facial rigging and re-targeting. “One of the big things we’re getting into is how do animators actually interact back with facial, as far as developing systems that an animator can just click on part of the face and it drives a muscular deformation to actually pose that side of the face,” outlines Derksen.

“Another of the big things we developed here,” adds Derksen, “is our re-targeting system going from an actor onto a character’s face. That’s one of the biggest things you have to deal with – how do you re-target onto an imaginary character? As long as you can match mesh to mesh you can re-target one to the other. As long as you have that you can re-map it onto your art-directed character controls and drive those.”

Watch an EXCLUSIVE Voodoo video on facial authoring.

Among Rhythm’s tracking and match moving and computer vision technologies in Voodoo is the ability to take plates or simultaneous views from multiple cameras and do facial mocap. “We can then capture the performance directly onto a rig,” explains Wilson. “In one case we did it where there was a partial face replacement where there was a character who was a cyclops and we needed to track his original performance and then replace part of his face with a single eye that was doing something like what his eye was doing.”

“We also do take mocap from other places,” adds Wilson. “We can take in curve data that’s targeted for simply rigged skeletons. Then it’s about re-targeting that and it’s usually handled by the rigging department – they set up rigs that can take that original curved data and re-map that to curved data for the rigs they’re developing, so when they hand off re-targeted data to the animators, then the animators have curves they can easily augment and change. We pride ourselves on our ability to do key-frame animation, so if we are bringing in mocap data, that we can completely do whatever we want with it.”

Is Voodoo for sale?

Currently, Rhythm’s sister company Prana Studios has adopted Voodoo (but not Wren) for use in production. fxguide put the question to Rhythm & Hues – are they selling Voodoo? The answer is yes, sort of. “Licensing is the direction we are definitely interested in and pursuing,” says Bradfute.

“If somebody is interested in using it, we are interested in helping them,” adds Rijpkema. “It’s not a secret.”

In fact, Rhythm is encouraging anyone interested in adopting the studio’s technologies to email this address – [email protected].

And although Rhythm & Hues is a far different company now than only a few years ago, there remains a great passion for continuing to make realistic creatures and using Voodoo for the task. “Voodoo is our love child,” offers Derksen, “and it’s something we’d love to see go on.”