Tarzan creature & FX pipelines

David Yates (Dir. of the last 4 Harry Potter films, plus the upcoming Fantastic Beasts and Where to Find Them) directed The Legend of Tarzan starring Alexander Skarsgård (HBO’s True Blood) as the legendary character created by Edgar Rice Burroughs.  The film also stars Oscar nominee Samuel L. Jackson (Pulp Fiction), Margot Robbie (The Wolf of Wall Street) and two-time Oscar winner Christoph Waltz (Inglourious Basterds, Django Unchained). It has been years since the man once known as Tarzan (Skarsgård) left the jungles of Africa behind for a gentrified life as John Clayton, Lord Greystoke, with his beloved wife, Jane (Robbie) at his side. Now, he has been invited back to the Congo to serve as a trade emissary of Parliament, unaware that he is a pawn in a deadly convergence of greed, revenge and slavery masterminded by the Belgian, Leon Rom (Waltz).

The film’s story required the VFX team to have Tarzan interact, fight and even hug a wide range of animals. All of which were CGI and most of the jungle was actually a soundstage in England. Visual effects was therefore a key component in providing all the animals in the film and making the jungle, and other sets, even more vast by extending them. Visual effects supervisor on the film was Oscar winner Tim Burke, and a number of VFX houses, including Framestore, MPC, and Rodeo FX, produced the assorted menagerie of gorillas, lions, wildebeests, and rhinos for the film.

 

 

Production on The Legend of Tarzan involved a fusion of design, aerial photography and state-of-the-art visual effects, all merging to present the spectacular landscapes and wild inhabitants of Africa, while shooting almost entirely on the stages and backlot of Leavesden in England. And though the human characters interact with a variety of African species, no real animals were used in the making of the film. All of the creatures—from gorillas to lions to elephants and more—were entirely brought to life through VFX.

During pre-production, Film Advisor Josh Ponte, who spent the last 15 years striving to preserve the wildlife and natural resources of the African nation of Gabon, arranged for a military helicopter to show David Yates the splendor of the country’s forests, cliffs, rivers and waterfalls. The director spent four days marveling at the terrain. Those remote landscapes were captured in a six-week location shoot following filming at Leavesden. They provided the wealth of geographically diverse backdrops that seamlessly composited with Stuart Craig’s sets. After the first scouting trip to Gabon, Ponte went on to serve as the project’s African technical advisor, becoming an invaluable consultant for virtually every department. “It was incredible having Josh with us,” confirms Barron. “He has spent a huge amount of time working in Africa, and Gabon in particular, and is extremely knowledgeable about everything from the historical context of our story to the village culture to the animals.”

To bring the animals to life, the different teams began by watching documentary footage, studying the behaviors of the various animals in the wild. Some also took trips to zoos to observe them in person, noticing they behaved quite differently in captivity versus their natural habitats. Based on those reference materials, the animators re-created the native fauna seen in the film, including gorillas, lions, elephants, gazelles, zebras, hippopotamuses, ostriches, wildebeest, crocodiles and more.

The variety of species upped the ante for the visual effects teams. Burke explains, “Techniques have been developed to create fur, feathers and skin, but it gets very complex when you’re dealing with something on this scale—just being able to render the huge number of elements in so many shots. It’s only in the last few years that you would attempt something of this size and know you could realize it. The bar is constantly being raised, and we just have to keep chasing it.”

The bar was raised even higher in instances where the animals interacted directly with Tarzan, whether he’s nuzzling with a lioness, marveling at the nobility of elephants or reconnecting with the Mangani gorillas he once called family. “One of the things I love about Tarzan is his ability to communicate with the animals,” says Yates, “I think it’s one of the more magical aspects of the film.” In those scenes, a stunt performer in a gray suit became a stand-in for the animal, allowing both Skarsgård and the VFX team to have the proper reference points for the action.

Tarzan - Gorilla fight clip from the film.

A prime example was the fight between Tarzan and his gorilla brother, Akut. Burke recounts, “We designed a big padded suit as well as a transparent helmet that gave the stuntman the overall dimensions of Akut. It was very important to give Alex something to react to that was the same size and shape as Akut; otherwise, we’d later get into all sorts of problems with Alex putting his arms through Akut’s body or intersecting with bits of him he shouldn’t be intersecting with”. The sequence was keyframe animated by Framestore, and not motion captured. In all Framestore delivered some 700 shots which included all the gorilla sequences.

The gorilla sequences were complex as they involved both close proximity fighting and also extensive interaction with FX simulations. Framestore’s visual effects supervisor was Alex Pejic, who worked alongside Andy Kind and Ivan Moran. The team worked on the film for almost 2 years, although Pejic joked that “it was around in London for a few years”. The work grew for Framestore from the initial brief with the gorillas to also include elephants, leopards and other animals. The work was split between Framestore London and Montreal. As the shot count doubled for Framestore, Andy Kind joined Pejic in London, where the majority of the assets were built. Moran’s Montreal lead on the sequences with the water and rain (and the Elephant sequence), and London lead with the majority of the gorilla fights and flashback scenes. Pejic believes the tools the team have in place makes the division very seamless between the two teams.

There was a 1:1 maquette of Akut on set for stand-in and lighting reference, but it was only the upper torso, shoulder and head. It was fully groomed and after the shoot the maquette was scanned for CG reference.

tarzan2

Each of the fight sequences was shot with the stunt performer and Skarsgård then the production would shoot a clean plate with only Skarsgård. Following this the Framestore team would supervise plates shot with the maquette for lighting reference plus “grey ball, chrome ball and thousands of other reference stils!” explained Pejic. Framestore would normally need to digitally remove the stunt double from the hero take and then both extend the set and add in the creature animation.  But even with all this work in some shots it was decided that the motion of the stunt performer was just wrong in terms of gorilla physiology. “Finding the action itself, the interaction and the interaction with the ground… creature wise the look was relatively easy in terms of the look, but the animation could be really challenging.”

One of the most complex sequences was the touching flashback scenes with Tarzan and his adopted mother gorilla, Carla. There were complex both in performance from the creatures and the interaction between the young Tarzan and the fur on the CG gorilla. Some of these shots were so complex they required digitally replacing the actors arm with a full CG arm so the entire shot could be CG and the fur could interact correctly. The interaction of the younger Tarzan and Carla were the last shot turned over to Framestore. “There was a stunt guy on set to lift the little tarzan off the ground, but we fully replaced little Tarzan’s arm with CG.. there were shots where we only kept the young actors feet and hair”.

The team used a new muscle system for the film. The new muscle skin system is called flesh and flex. “It uses the real life muscle structure to give you realistic jiggles, motion and skin sliding.. it works by taking the animation into account, applies dynamic and it adds all the additional level of detail that makes the animation more real”. The system allows animators to ‘fire muscles’ and control how the creatures flex or fire, prior to moving or jumping.

They also developed a new tool for the jungle growth. The system populates the jungle based on various rules, including how much light would get through the tree canopy, etc. All of this adds realism in producing a ‘sensible’ forest and jungle floor.

TAR-T1-067r

The stunt team, led by stunt coordinator Buster Reeves, collaborated with movement choreographer Wayne McGregor. “They didn’t have to move exactly like a gorilla,” McGregor clarifies, “but they had to embody the essence of being a gorilla. The discipline required not thinking like a human, so we did several workshops to explore what that meant. What happens to your carriage considering you’re nowhere near the same weight? How is it to walk on all fours and then shift to bipedal? What happens to your head, shoulders, arms and legs? What do the different gestures signify?” Reeves adds, “Wayne and I really worked hand-in-hand to get the best out of the action. The hardest thing for my team was not reverting back to acting like a person because the natural inclination, especially in a fight scene, is to move as a human would. To stay within the animal realm was the biggest challenge.”

Mixed in with the natural plants, the art department fabricated a variety of trees, “designing them as pieces of sculpture,” says Craig. “As we went along, we learned the trick was not to bury the trees in the foliage, but to put the foliage behind them to throw the trees into silhouette. It’s like flower arranging, but on a massive scale,” he laughs.

On the stage the extensive sets were well planned but the five major sets were only about 30 x 30 m in area. “You walk into the set and it looks like a real jungle “comments Pejic. But even with the large sets, the jungle still needed to be extended up and back into the distance. “The set was build roughly up 10m in height, everything from there was digitally extended, so there were no tree tops, or anything else. But what we realised fairly early is that even with a 30 x 30 m set, you still shoot off it, so we LIDAR-ed it, photographed it and then our R&D guys wrote a script to procedurally populate the mid ground and background.”

One of the critical sets was where Djimon Hounsou’s character, Chief Mbonga, confronts Tarzan and the gorillas arrive to even the score. The set required the actors and apes to be in shallow water. This meant the CG gorillas needed to be wet from spray and rain and splash through and stand in a foot of water. On set there was nothing in shot to interact with the water so all of the gorillas water interactions are CG water fluid simulations blended into the plate. “It just wasn’t easy, just matching the plate with CG water is a hell of a job” adds Pejic.

c1

Andy Hayes, Head of FX for Framestore Film, was responsible for overseeing all the visual effects animation such as the water sims, but also smoke, mist and rain. The rain alone in the jungle is much more complex than one might suspect. The sim of rain itself would normally be simple, but as the rain had to hit literally millions of CG leaves on CG trees, just getting the vast array of geometry into the sim engine was no small task.

The team primarily uses its own in house fluid simulation tool Flush for water sequences, mist, clouds and smoke. The tool has its own spare volume format since 2010, which precedes the OpenVDB format. The Framestore solution fVox is similar, with dynamically created spaces which do not have to be uniform in shape – “they can be thin in some areas, thick in others as you need it. It dynamically grows as you need it… it is not only for storage but we used it rendering, so not everything has to be loaded in, so we can render large volumes very effectively – plus it has good data compression” explains Hayes. fVox also works with the fluid simulator so if there is a “small piece of water running along rocks into a river – it captures that effectively, .. many programs on the market still produce giant grids and are very inefficient”. Houdini was used for some of the dirt shots, on the floor of the jungle. “In about half a dozen shots the camera is so close to the ground we had to simulate grains of dirt that the animals are walking and sliding through… and this worked with our fBounce tool for leaves and twigs on top of that,” explains Hayes.

The scenes were all rendered in Arnold. The approach at Framestore is to build multiple sims in hierarchy and then render, as much as possible, in a single pass or as few as possible. This means rather than doing a volumetric FX pipeline with deep compositing, or compositing layers of sim renders together, the team carefully schedules the pipeline in Shotgun and then compiles the various sim caches and animation into one giant render. Their custom Arnold shaders have bi-directional path tracing since Jupiter Ascending . “We try and put everything into one render and just render that at the same time… so if you have a fire simulation or a water simulation or smoke… it all goes into the same render as the characters and their fur. That way everything interacts and is physically correct in terms of the lights and we get more believable images,” explains Hayes, “I think one of the big advantages at Framestore is that our images look pretty and that is because a lot of work goes into everything we do, there is a lot of stuff we build out from and add to… we have done a lot of work on instancing for example… the idea of just doing big vast render is something we just can do,” he adds. Lookdev and lighting is done in Fribgen, which is a nodal and rules based lighting system. This custom tool is highly integrated into the Render pipeline and allows for very quick lighting setups even on the vast data sets the team had to tackle in Tarzan.

For a typical shot the sequencing is important. The team focuses on getting the animation correct first and this then drives Framestore’s rigid body simulation fBounce for things falling around the creatures or plants being moved by the hero creatures. All the plants were designed to be reused and instanced, but having all the plants dynamic meant that the creatures would really sit in their environments. If the creatures were interacting with water, these elements would drive the water sim, which in turn would inform and drive the fur simulations. On top of this there would be atmospheric volumetrics.

Hayes is proud of the solid R&D that the Framestore FX team has developed over the last few years.

TAR-FP-0140r

The backlot at Leavesden became the site for a number of major sets, including the Kuba village, with its thatched roof huts, and Mbonga’s lair, surrounded by steep, jagged cliffs. The giant rock formations were crafted from large molds made from impressions taken at a slate quarry in Wales. The mountainous terrain forming the backdrop of the setting was captured in the Dolomites in northern Italy.

The production broke new ground in the use of the Weapon RED 6K cameras to film the landscapes of Gabon, which would serve as the primary setting for the story. Months prior to the start of principal photography, Josh Ponte arranged to borrow the helicopter belonging to the President of Gabon to take Yates on a scouting flight. It would prove to be logistically unfeasible and environmentally unsound to drop an entire film company into the wilds of Gabon to shoot the bulk of the movie. Instead the filmmakers devised a way to lens Gabon from the air. They commissioned the manufacture of a six-camera housing, which was fitted onto a mount built by Shotover, a company that specializes in aerial camera systems for cinema. The camera array, in two rows of three, was then mounted on a helicopter, which allowed director of photography Henry Braham to capture the different terrains of Gabon in stunning detail.

Braham relates, “It made a huge difference for me to shoot this material myself because I knew exactly what light conditions and landscape I needed for each scene and could shoot the backgrounds to match exactly, which was challenging in a fast-changing tropical environment. We also shot additional landscape material to lend more visual scale to the movie.”

For certain scenes, like traveling down rivers or inside the jungle, they used a long-line technique: suspending the array, or a conventional camera from a 50-foot wire, allowing the cameras to shoot in places inaccessible from the ground.

The six-week location shoot provided Burke’s VFX team with overlapping tiled background plates that, together, formed a larger moving image. Burke elucidates, “The six cameras provided an almost 180-degree field of view, but we then did a pass looking forward and another looking backward and then sideways, which effectively gave us a moving 360-degree tile that was photographically real and could be mapped into any shot. The results are great because you’ve got live action combined with live action, so it all feels real and seamless. I would say we definitely pushed the limits of plate shooting. I know I have never done anything as advanced or as adventurous as that before.”

During principal photography, Braham also utilized the same cameras in what he describes as “the opposite extreme from the helicopter shoot.” He explains, “For the visual language of the film, David wanted to employ a lot of close camerawork to counter the wider cinematic scale of the picture and to gain a more emotionally intimate perspective, which we developed throughout the shooting of the movie. To achieve the versatility we needed, we worked with RED to make their newly developed 6K camera suitable for these challenges. The combination of lightweight, compact technology and large negative area helped produce astounding images. The Legend of Tarzan was one of the first movies to be shot with the Weapon 6K Reds.”

MPC handled the end stampede as well as the first encounter that Tarzan has with the lions upon reaching Africa. While MPC did most of the creatures in this sequence, they worked in partnership with Framestore who provided the digital Akut seen especially in the beginning of this sequence.

A number of scenes take place on Leon Rom’s steam paddle boat, which was an actual floating craft. Art director Christian Huband, who is a boat specialist, expands, “It was built on military-style floating pontoons. We created a superstructure to carry the decks, cabins and working engines that appear to power the paddlewheel. The boat was actually controlled by outboard motors, operated from hidden positions around the vessel. There was no hull, as such, but it had sides that extended just a few inches below the water line.” The boat was first used on Virginia Water Lake in Windsor Great Park, then dismantled and trucked to Leavesden, where it was rebuilt in the tank.