Beowulf: Breaking Ground with Mocap

Beowulf once again raises the bar for films based on motion capture, extending the work from Polar Express and Monster House. Fxguide spoke to Doug Chiang at Siggraph about the film and we now follow that up with Sony Pictures Imageworks’ Visual Effects Supervisor Jerome Chen and CG Supervisor Theo Bialak. (Updated 30th Nov)

Beowulf is the latest film in a series by essentially the same creative team headed by director Robert Zemeckis. In the film, the mighty warrior Beowulf slays the demon Grendel and incurs the wrath of its monstrous yet seductive mother in a conflict that transforms a king into a legend. It has all the complexity of a mythical tale combined with the technical complexity of the being the most advanced motion capture film attempted and it is being offered widely in 3D stereoscopic vision.

3D Distribution

The film is being released in traditional 35mm release print form and in several others options, Jerome Chen explained to fxguide that the film is also being shown in IMAX 3D, Digital Cinema, Real 3D, and Dolby 3D.

In the stereoscopic versions, the left eye is most of the original ‘2D’ footage or renderings and the right eye has been redone. The right eye is created by an additional render pass, “and where you place that right eye (camera) really determines your convergence point, in other words how much depth there is in your scene. Different directors have different creative preference for how much dimensionality is created, some like to create a window which you look into, a third party watching the scene, others directors want the 3D to come out at you, so it is more of a spectacle , and Zemeckis prefers the 3D basically come at you as much as possible,… when I first started this did not realize there were so many different 3D aesthetic approaches in Stereoscopic”

3D has been growing over the last few years, when Polar Express was released there were only about 50 theatres able to show the Stereoscopic version of the film. Monster House had closer to a couple of hundred Real 3D theaters, for Beowulf the number jumps to well over a 1000.

Motion Performance Capture

Massive multi-person performance capture

The base performances of the film are built on complex motion capture. The process is staged very differently from a traditional shoot, as the director, crew and even the camera do not need to technically interact with the actors nearly as much as during traditional filming, allowing for much longer multi-actor takes. But this is at a trade off of having all the actors in complex dot suits walking around on set make of chicken wire, being filmed by hundreds of motion capture (mocap) cameras. While the body suits were covered in dots or markers as in previous productions, for Beowulf the captures space the actors had to work in was expanded to a huge 7.5m x 7.5m x 4.3m stage, which was large enough for even two horses to be ridden.

John Malkovich (Unferth) commented during a recent promotional podcast that he was struck by the freedom that mocap enabled and how much the acting process resembled theater performances much more than a typical film shoot. The mocap approach allowed them to actually act for most of the day, unlike what they typically do on a film set, which is to wait. In effect the entire performance is virtually filmed from every angle and at a later stage in post the director positions the camera to capture the actual performances from the angle he desires. This means that actors are not waiting on reloads, camera or lighting setups and are free to perform in almost the same way that a theatre in the round is presented.

To allow the mocap cameras to see the most amount of trackers on every actor, the props used by the actors were made of wire, and thus effectively just frames or outlines of the later 3D rendered tables, chairs and other items the actors needed to interact with. Because Malkovich would usually spend time in the makeup trailer, he commented that “the time spent getting into the mocap suit wasn’t an unusual inconvenience”. The actual suits not only captured the actors gross body movements but their subtle facial and even eye movement. The delicate eye movement was new to this film and solved by having sensors on the actors faces that recorded muscle movements. This allowed the software to accurately work out where the eyes should be looking without explicit eye ball markers such asspecial contact lenses. The actors eyes could be tracked and recorded along with their gross motor functions. The rest of the face had over 150 individual stick on tracking markers.

A much discussed concept since Polar Express is the question of human realism and people’s response to the near photo-real look of the rendered characters. A term much discussed since Polar Express is the “Uncanny Valley”. The “Uncanny Valley” is a hypothesis about robotics concerning the emotional response of humans to robots and other non-human entities, introduced by Japanese roboticist Masahiro Mori in 1970. Mori’s hypothesis states that as a robot is made more human-like in its appearance and motion, the emotional response from a human to the robot will become increasingly positive and empathic, until a point is reached beyond which the response quickly becomes that of strong repulsion. However, as the appearance and motion continue to become less distinguishable from a human being, the emotional response becomes positive once more and approaches human-to-human empathy levels (Wikipedia)

Beowulf prepares to slay a horrible dragon fire plays a large role in the effects animation of the film

Jerome Chen, (Polar Express, Stuart Little, Godzilla, Contact) senior visual effects supervisor on Beowulf, does not believe that the Uncanny Valley hypothesis really applies to films as they are entertainment. According to Chen, movies are artistic and commercial projects and clearly audiences are connecting with the characters, since even with the response to Polar Express, “no matter what the critics have said….I would not be able to still be making these films”. Regardless of critics’ debates over a 40 year old Japanese hypothesis, Beowulf has lifted the bar tremendously in terms of realism and model detail and accuracy.

Doug Chiang, the film’s Art Director, first worked with Director Zemeckis as a freelance conceptual designer for Back to the Future II, did a lot of very realistic drawings of the characters in Beowulf. At this early stage the drawing were very detailed but with still an amount of stylized representation. From these concept drawing to the final films characters the process was very evolutionary. Chen comments that he found “it ended up being more realistic than I expected, or rather more detailed… otherwise we would miss parts things.”

Animation begins while using the performance capture data as reference
Character animation controls allow the animator to refine the motion captured on stage

Final animation prior to the final rendering of the scene or characters in the film
The final shot of Wiglaf as he speeds to the support of Beowulf who is prepared to battle the fierce dragon guarding the entrance to the cave that houses Grendels mother

The team found they needed high levels of detail to capture the subtle performances of the actors. “The more detail we put into the characters, the more we were engaged by the performance – even among ourselves,” says Chen, “we got to the level that we said you know he should have hair coming out of his ears, he should hair out of his nostrils. In fact he should have peach fuzz all over his face… and that was the level that we thought was interesting. And the great thing is…we are at the point that we can do that, technically and creatively the tools and the hardware are at a point that you can do that…and you can render it in a production environment.”

After the characters have been designed and the performances have been captured, a certain amount of retargeting is required. This which is the process of having the actors dimensions adjusted to the desired size of the characters the actor is actually playing.

The digital characters are then clothed with various cloth simulations and Hair simulations. As cloth simulations can be very expensive and time consuming a library of referenced simulations was developed. Only the Chain mail was done procedural . The Cloth sims libraries and faster variations without the need for running new full simulations, but to further reduce the project complexity items such as shoes for the male characters were : “One size fits all” – and each male character wore effectively the same boot, with the addition or subtraction of straps, buckles and of course the color changes. This process was applied to all the models and characters allowing 46 pure body types to be built out to over 300 variations.

Water, Fire and Snow

Grendels seductive mother makes a tantalizing proposition to the Viking hero Beowulf confronts the monster note the fine body hair and water wet dry body mapping

The system was based on the system built from Ghost rider and built into Sphere Sony’s Volume Renderer. As there was so much fire they built libraries allowing artists to select from a literal A to Z of fire or burning torches – with varying amounts of movement and flame.

In one scene where the entire scene was lit by the single central fire light source, the team came up with some clever techniques to speed up production. While the blueish gases flames would act as a multi-point set of light sources and could be reduced to a constantly flickering point cloud of twisting point courses, ray tracking such a multi-point point cloud is extremely complex.

The team built a script that would average the point cloud to a moving ‘pill’ shaped floating light source. The position, orientation and size of the single light source is calculated by an average of the point cloud points so it still darts around in the region of the flames, but it reduced the problem to one darting light instead of dozens of individual lights. This reduced a volumetric complex problem to a single procedurally animated and varied “jumpy” light.

The team was also aware of the need to control what could be explicitly seen and still maintain the film’s delicate PG rating. In one scene, a character is ripped in half and the team needed to make sure that was played out in shadow and not actually seen. This also affected the amount of blood in many scenes. Tt Digital Media World in Sydney it was hinted that the film may have a less restricted version available on DVD with added or restored scenes. The PG rating also affected the blocking of the action as our hero Beowulf is naked for several key scenes.

Ice, water and rain were often simulated in Houdini and then rendered in Renderman. To address the huge computational demands of rendering reflections, the environment surrounding of a pool of water would be rendered once and then re mapped back on lower polygonal count geometry. This allowed the water sim to exist in a lower poly count final environments such as the Treasure Room.

Interesting work was done to produce bioluminescent water. Once again point clouds were used to get the eery blue water glows, with complex black and white surface maps used to control the water droplets and skin moisture gleams as wet actors interacted with each other. The wet to dry maps would allow the artist to control all aspects of the virtual actors skin surface. Beowulf not only has a skin surface with pores but also micro “peach fuzz” hairs all over his body.

Much of the movie’s environments have clouds, smoke, raind and even now. To get accurate interaction between objects in the scene and the environment, the team developed an powerful “rotowash” 3D vector field technique. In one scene in which this was used, the dragon flies through a snowstorm with the movements of its wings displacing the snow. In order to speed renders, a point cloud of force vectors could be calculated for the dragon’s wings allowing the various air current forces to be translated to things such as smoke and rain. The effect was to produce subtle and beautiful eddies and currents in the wet or snow filled air as the dragons wings fanned ‘virtual’ air currents.

To help the compositors, these environmental vortex snow elements would then be rendered in R, G and B depending on distance from camera, allowing the artists greater control on blending the final white snow into the complex composition of the shot.

Dragon Battle

The Viking hero Beowulf tries to slay a horrible dragon
There was enormous animation beyond motion capture

The Dragon was a finely detailed model, with particularly complex aging on the golden textures of its skin. As reference the team built the gold creature as an aged version of a part alligator part crocodile. This real world reference was invaluable to getting the worn scales and skin textures.

All images copyright © 2007 Paramount Pictures

1 thought on “Beowulf: Breaking Ground with Mocap”

  1. Pingback: The Decline of 2D animated films | 50162736kieran

Comments are closed.