Newt Scamander, British Magizoologist, expelled from the British Wizarding school Hogwarts, arrives in New York and so begins Fantastical Beasts and Where to Find Them.
Central to the film’s success was plausible and yet fantastic creatures, “we made literally hundreds of designs and in the end there are about 15 main types of animals, many with variations – babies etc” Christian Manz told us. Manz and Tim Burke were the vfx supervisors.
The character work was roughly shared between the key facilities as follows:
- Erumpent, Niffer, Bowtruckle + Gnarlack (The goblin gangster): Framestore
- Thunderbird and the Obscurial: Double Negative (DNeg)
- Demiguise, Occamy, Billywig: MPC
- Graphorns, Swooping Evil: Image Engine
For the marvelous “Inside the case” sequence, Rodeo FX and Image Engine did a lot of work. But “some shots had up to five companies working on them ” explained Manz.
Our Special Exclusive WIRED Design fx piece with our media partners WIRED Magazine: Erumpent Mating Dances!
The film not only had previz, but the entire film was also post-vized, by an inhouse team of some 50 artists. Both Third Floor and Proof had previz teams on the film along with teams from Framestore and DNeg, “we had a mini-facility by the end”, commented Manz, “and by the time we presented the Director’s cut, we had filled every single effects shot, New York was in there and all the creatures – so we had a really solid blueprint with assets worked on for everything”. Burke adds “We ended up keeping the previz team right through production until the end of the film, we were designing shots in house , and then turning them over to vendors- as you always do with these sorts of films – right up until the end.” For example the director crafted the ending, “the film you see is not the one we shot, David (Yates) works in a very considered, fluid way. For example the transformation with Jonny Depp at the end was not originally in there. When we were shooting that end scene, he wasn’t turning into Grindelwald,” adds Burke. “that wasn’t even in the script, it was the day we were shooting, that David turned and said – ‘Yeah he’s going to turn into someone else now!’ – and we were asking who? And he said – well we don’t know yet!”.
There were three additionals shoots from April until the end of July to provide additional photography to craft the story of the film “all for the benefit of the film” says Burke. For Manz, this is the nature of modern filming “we always knew we were going to go back”. Tim Burke points out that hopefully, on the DVD, audiences will see quite a few additional scenes “with quite advanced animation and with other creatures”.
Officially, the work on the film started in January last year and the film was shot in August. Director David Yates gave the team a brief to make animals not characters, and “fantastical but not fantasy” according to Manz. The team started with concept art and was led “in house by Pablo Grillo who was our animation supervisor”, he adds. Grillo who is Senior Animation Supervisor at Framestore worked with the team in house on creature animation development.
Framestore was the lead facility on the project, although many other (mainly London based) houses also contributed, including, MPC, DNeg, Cinesite and Method amongst others. Grillo and the team were given the chance to build test scenes or ‘skits’ with the films magical beasts before principle production, to explore the characters and their styles of mobility. “Almost divorced from the script,..we were coming up with ideas for the character which would then flow back to design changes, more drawings or further exploration, it was a real circular process that went on for months. For Picket the Bowtruckle – we had over 200 designs before we settled on what we finally had. David was very keen, even if we only saw the creatures for a moment, – that we had a quick read and a story for each one”.
This led to a different pipeline than the other Potter films. On those films there would be extensive design work perhaps resulting in a maquette, that would be turned over to the visual effects team much later. Previously the job of animating the creature was solved based on what was production designed and then how principle photography was staged. For the new film, the animation team explored the characters moving before principle photography and this taught us lessons that could be incorporated in both design and on set photography. “We worked it out before the film was shot, which meant by the time we shot we had our digital cast, and we could show David and our actual human cast how they would work, and that was a really great and creative process”, explains Manz.
Tim Burke started on the second Potter film in 2002 when he was a Supervisor at Mill Film. He then joined the production on the third film. He worked on seven films in 11 years, “and it has been 5 years since that last Potter film was released so that is quite a chunk of time really… but in saying that it wasn’t that long, we did so much work in such a relatively short about of time, that’s the scary thing”. What Burke enjoyed was returning to the new film and seeing so many people from the earlier Potter films.
Burke pointed out that the creatures in the film heavily reference real world animal behaviour, “everything had to be very grounded, it was not about creating fantasy creatures but more creatures that you really believe exist in this world and perhaps you maybe might have even seen, but mistook for something else.” For example the team referenced the Honey Badger for the Niffler. As it is a creature with a long snout and a coat of black, fluffy fur. The niffer are attracted to shiny things and have a pouch on their bellies which holds far more than at first seems possible, reminiscent of an undetectable extension charm on a container. Burke points out while they did extensive work before the shoot, they did continue to explore some of the designs right through the process.
The huge Erumpent magical beast ended up in the film looking fairly similar to where the team started the process. It’s initial look, that would become the final look, was put to one side – “it sat on the shelf for months” as the team looked at trying more complex motion and animation studies, with increasingly more complex versions of the Erumpent and what the horn might look like. “We spent weeks and months doing lots of different designs of what the horn might be like, and changing it.. and it looked like quite a different thing, with a very elaborate horn. We shot thinking that is what we were going to use,” he explains. “We put it in the scenes and showed David (the director)… and he immediately pointed out it did not look real, – it was too fantasy”. The team stripped it back to the version audiences see today and put that simpler version in the same plate photography, “and the director said – yeah I believe that now – that looks real”.
This realism and grounded natural animal references were then balanced with the charming personalities of these creatures in the story. The Erumpent, for example, “just has this wonderful character when you see her sitting looking up at the tree, she just really wants Jacob, because he was covered in musk”, comments Manz. This combination of exploration and experimentation was clearly very rewarding for the visual effects artists involved. “We had the joy of trying something out and saying “oh actually that works, that is really cool’, and then the amazing feeling of having some of those things turning up in the next draft of the script”.
The Erumpent is a huge African magical beast resembling a rhinoceros. It is a powerful creature, with a thick hide capable of repelling most curses and charms, a single long horn, and a thick tail. The actual Erumpent in the film is a horny rhino beast on heat. The beast will attack if provoked by ejecting an explosive fluid from its magical horn.
“Pablo put together a very strong team of animators, with a core of 4 or 5 character animators who designed the functional aspects of the creatures applying the knowledge of what they would be required to do in the story” explained Burke. The team did not have to make ‘general purpose’ creatures, as the script was well developed and the process was not reactionary to changing whims. “As that was all worked out ahead of time, we could work with Eddie (Redmayne), to design props and devices for him to use on set that would help him know what was going to happen and how it was going to move, – a sense of timing and interaction. As a result, one of the things that I think works in this film is that you believe the creatures are there as the actors believe they are there, – they occupy the space. Eddie’s desire to have a relationship with these creatures early in production.. [meant] we worked very hard and went to extreme lengths to create puppets including the Erumpent”. The “puppet” for the Erumpent was a 17 ft. carbon fibre wireframe Handspring Puppet. The South African company that created the Erumpent puppet is the team behind the incredible ‘War Horse’ stage puppetry. The Erumpent was on set to provide eyeline, performance and framing reference and was controlled by a team of four people. “It was a very light weight frame but enormous, 17 feet high and 20 feet long, we not only had them on set, but they also rehearsed with Eddie beforehand” explained Burke.
The mating dance is one of the highlights of the film. It was choreographed by Eddie Redmayne with Alexander Reynolds, the choreographe rRedmayne had worked with on previous films (The Theory of Everything and The Danish Girl). “She and I watched birds do mating dances, and I then did one, I videoed myself. Or no, she videoed me doing about 10 different equally humiliating mating dances.” Redmayne explained. A set of these were then sent to the Director. From this set of selected moves, the animation director Pablo Grillo and his team animated a version of the Erumpent, which the puppet team learnt so they could rehearse directly with Redmayne before filming. “The puppet would come on set, we would be able to design lovely poetic shots around the choreographed puppet, we would then shoot at least one take with the puppet in, .. and then we would shoot a clean plate as Eddie or any of the other actors would have a really clear idea of what was happening,.. we did that basically for all the creatures,.. we had some really skilled puppeteers”, explains Burke.
“There were fairly simple puppets but they could make them so expressive” said Robin Guiver, the lead puppeteer. Avye Leventis was the puppeteer for the Niffler and the Bowtruckle, Picket. The puppet of the Bowtruckle proved extremely useful in one shot in particular where he picks the handcuff lock of Newt. One of the actors was needing clarification in the death cell scene. “When he was meant to jump on one of the executioners – the actress was having trouble imagining what this would be like so I got Avye to come over and put Picket on her and run her up and down her arm, and suddenly the actress was like “oh gosh it’s like having a spider on you!’. It was really important – she really got that performance – the puppeting was really important to help make you believe the characters were really there”.
Framestore
In addition to all the work Framestore did as described above, they also did Gnarlack, the nightclub goblin gangster. The character was played by Ron Perlman.
‘Gnarlack is one of the best digital humanoids yet,’ says Pablo Grillo, ‘And a testament to the talent that we have here in Framestore.’ A cigar-chewing, wise-cracking goblin who stalks around his speakeasy, the character of Gnarlack was heavily influenced by the actor Ron Perlman. Perlman was shot at the main studio in Leavesden for the motion-capture with the Capture Lab team.
On the primary set in the studio, another actor stood in for eyeline, as Perlman is considerably taller than the character Gnarlack he was playing. “We had Ray- who was a great double. He had a full Gnarlack costume, -he was very good actually, he had a lot of character – he was really useful for lighting and eyeline” explained Manz. Perlman actually turned up on set with his own costume! “Ron was lovely to work with,…Ron ran all his lines just hunching down in the chair so everyone could get a feel for the performance – then we ran it with Ray for blocking – and with Ron delivering the lines off to the side of set. Then we moved to another set on the other half of the stage where we had constructed a scaled version of the set. The Art Department built it with blocks that were the correct size for Ron being only 4 foot high. All the other actors sat up on the blocks, so they were the right height to look down at Ron – and that is when Ron put on a costume he had brought of his own. So we marked up Ron’s face and put on the head set, and got all his performance capture,- just an hour after shooting the actual scene”.
‘Ron has a strong feature set: his eyes are hooded, so it made tracking around the eye especially difficult,’ explains Richard Graham, Capture Lab Studio Manager. ‘Plus his character was constantly smoking a cigar and drinking whisky, so we had to work around that.’ Perlman wore a Vicon Cara 3D facial motion capture headset, surrounded by four witness cameras.
More cameras were needed to capture the subtle details of his face. ‘To assist the facial reconstruction of the character, we had him sit before 98 cameras for a facial coding shape session,’ explains Andy Kind, VFX Supervisor. This allowed the team to sculpt the face directly in 3D, refining the performance in keyframe animation. For his facial FACS capture the team did an extensive 90 expression FACS set.
For his performance capture on the special set, the team used the Cara Vicon facial system which has 4 high resolution, high speed cameras capturing every nuance, complete with a custom built on-board lighting setup. His face was also marked with tracking dots. “Framestore evaluated a bunch of systems and that is the one they liked”, explains Manz. “They used it on Jungle Book and Paddington. It got the most resolution and the best accuracy . It is a point based system, but with the 4 cameras it solved the depth really well”. The Framestore team always knew they would not be doing a straight retargeting. “It did what we intended which is give us great reference and drove our post-viz to very quickly give us walking talking Gnarlack for the Director’s cut”.
There were also 4 high resolution witness cameras to provide the animators with a package of data and reference that allowed the highly skilled Framestore animators to produce such a nuanced performance.
The final characters were rendered in Arnold with the Flex and flex muscle Framestore custom software. The team has been tracking the USC ICT high resolution anisotropic skin stretching, with its effect on the specular lobes. They also did detailed bloodflow work. “We did basic version of this back on the Potter films but what Manz saw on Gnarlack at Framestore was “wow, we are seeing a really cool looking rendered creature – very quickly – we were getting 90% very quickly – allowing time to craft the last 10%”.
Within the Blind Pig speakeasy scene, Framestore also worked on a goblin singer (Aretha Ayeh, who performed on set and Emmi who provided the vocals) as well as a medley of magical creatures. ‘We created CG characters for the giants, elves and band ensemble,’ adds Kind. ‘Then we gave them crazy instruments including a sousaphone/trumpet concoction. Pablo and the team then developed the keyframe animation.’
MPC
MPC’s Ferran Domenech supervised more than 220 shots for the film. MPC had previously worked on all the the Harry Potter movies and they are very versed in creating magical effects for JK Rowling’s world. For Fantastic Beast’s the team’s scope of work ranged from environment extensions, a full 3D environment of prohibition era Manhattan and magic effects.
The environment work was critical for recreating New York. In general over 1000 people were scanned through the productions Clear Angle Studio system comprising 100 DSLR cameras (Canon’s 5Ds mainly) for high detail full body and facial 3D scanning. This data set was available to MPC and the other vendors. Manz believe’s Clear Angle Studio system is the best in the world for high speed, detail full body scanning for work such as Fantastic Beasts. 4DMax was responsible for the 3D Lidar & Photogrammetry of all the sets, location & prop scanning. “We wanted to make sure we had everything, every set, all the extras, we wanted to have absolutely everything.. for the film and even VR – we have become the hub for everything”.
MPC used this data for digital double’s for Newt in the end sequence. MPC’s work started with the set and crowd extensions for the arrival of Newt to New York and the creation of a detailed transatlantic boat. The most challenging work in the opening sequence was recreating New York’s lower Manhattan, fully modeled and rendered for a flyby establishing shot. MPC’s team had to conduct in depth research into Manhattan of the period as many of the buildings had changed, in order to recreate the area in full CG.
However, the majority of MPC’s work was largely focused on the creation of three of the film’s beasts in a major set piece.
In one of MPC’s big action pieces, two creatures; the Demiguise and Occamy completely destroy the attic of a department store where they are hiding. MPC’s VFX Supervisor Ferran Deomenech said “The first creature we began work on was the Demiguise, a small ape-like magical creature covered in long silver hair that can turn invisible and read the future. To create the challenging long flowing hair of the Demiguise, we used our proprietary Furtility groom technology, controlling it using strands of simulated cloth geometry. This method allows for the hair to move naturally and interact with the creature’s arms, legs and the environment around it.” To create the Demiguise’s invisibilty effect, MPC developed a new texture projection tool for Furtility to be able to paint the background image over the fur and have it move realistically. To animate the Demiguise the team used keyframe animation for the body and facial performance.
The most challenging creature to realize was the Occamy; a gigantic feathered snake with the wings of a bird and dragon-like facial features. Domenech said “To build this magical beast, the asset team at MPC had to create five different variations of it’s body to be able to fill the space in the attic in which it resides. This creature had to perch and coil around the beams of the attic and look almost impossibly long, therefore splitting the body into different pieces was a must to be able to fill the space and match the carefully designed compositions of each shot.” The snake rigging technology that was previously developed for the Harry Potter films was refined and put to good use for the Occamy with the added challenge of covering the entire body in feathers using MPC’s Furtility tool.
In the sequence, the Occamy is startled by Newt and it’s serpentine body gets tangled on the structure of the roof, sliding and shaking it apart as it tries to free itself. MPC built a detailed CG set extension of the attic which included structural beams, bolts and nails, wood paneling and an outside layer of shingles. MPC’s FX team then used it’s proprietary Kali destruction technology to destroy the attic. The action comes to an end when the Occamy is trapped inside a tea pot using a CG cockroach as bait. One of the magical attributes of the Occamy is it’s ability to fill the space where it resides, so the fully feathered creature had to be able to scale from the size of a building to the size of a mouse at will. To create this complex transformation, MPC’s Furtility tool was adapted so it could scale interactively.
Finally, MPC worked on the Billiwig, a small lightning fast irridescent magical bug that can change it’s flight mode from dragonfly to propeller plane at will. “This little creature has lots of character and was great fun to bring to life. The Billiwig makes appearances throughout the film, in the foreground for some shots and appearing in the background for others, always curious about what is happening around it.” said Domench.
Cinesite
Andy Morley was the visual effects supervisor for Cinesite. Morley has worked around the world on a variety of film, TV and commercial projects, and has over 16 years of industry experience. Andy’s huge list of film credits include Avatar, Hellboy2, Transformers. Andy was previously a VFX Supervisor at The Imaginarium Studios.
Cinesite handled the magic meal for Jacob and Newt that the sisters cooked. The entire room was filled with Magical items from an old school wooden cloths horse with cloths hanging on it, to an entire strudel which is created and cooked as it moves from the kitchen to the table.
The set had an elaborate set of rigs, and while some worked well, some such as the cloths on the clothes horse were all digitally removed and then replaced with a new cloth sim. Things such as this and the background magical ironing required some juggling, as cloth is produced by cloth sims which naturally assume gravity and the real world. The cloth needed to look real but also have some character and relaxed self governance.
The strudel posed a slightly different problem in that a cooking strudel does not move, it just browns so the team had to invest flourishes to animate with and on the food as it not only moved to the table but also cooked. “This and say apples peeling themselves -all involved some quite interesting and intricate rigging problems for the team, that you just don’t normally get to do” comments Morley “and it was really good fun!”. The strudel ended up being a series of about 6 shots, each one of which “pretty much had its own strudel model- different asset, with various components such as sugar, raisins – and then folding in a certain way.. lots of sculpts and builds” he adds.
Cinesite’s Making of
Morley used the same team on all the shots so they could provide a continuity and he found a small team works extremely well on a contained sequence, “we worked in what I call a ‘narrative mode’, by that we were not too hung up on individual shots, we treated it all more as one thing so if the edit changed it didn’t matter – it is just more flexible” he explained.
Below is the full clip.
As the table was empty on the day of the shoot, and all the elements were added digitally by Cinesite. This had a few unexpected benefits, lighting food can be tricky and making the food look magnificent and in no way plastic is a skill even for real food photographers. As Cinesite had nothing on the table indicating the real lighting, the team was free to skillfully relight the table to make the food look its best.
Method
Deluxe’s Method studio worked on many sequences and in particular reimagined the look of the distinct “apparate” and “disapparate” effects from the Harry Potter films through which a wizard magically teleports into or out of a scene. “We had sort of a disapparate shoot out between various companies, to find the right look” commented Manz.
Andrew Hellen, VFX Supervisor, Method Studios, said, “Production wanted the ‘apparate’/’disapparate’ effect to have more a of a 3D feel than in past films, so they asked some of the vendors to redesign the effect, retaining the essence of the original but with even more dynamic movement. We came up with a rhythmic particulate effect that follows a bit of a corkscrew pattern, almost like a whirlpooling Picasso painting that snaps together, and our concept was ultimately selected. It looks completely different from every angle, adds a lot of dimension to scenes and was a blast to devise.”
Method employed the ‘apparate’ effect in a full-CG point-of-view fly-through shot, covering several blocks of a grungy Manhattan tenement neighborhood, with the character finally appearing near a recreation of the historic Woolworth Building. Artists peppered the sequence with digital crowds and various CG elements like flapping laundry, using period photographs for reference and tonal accuracy. “We referenced a lot of old maps from New York circa 1920. It’s such an iconic area, we wanted to retain that authenticity, inject the appropriate amount of chaos while making sure any creative historical deviations didn’t distract from the story,” Hellen explained.
On the creature’s side, Method handled the shot of Newt bottle-feeding a young glowing creature – a tentacled cross between a dust mite and a squid with a transparent body. Production provided live action plates with the actor, and Method artists added the CG creature to the shot, digitally wrapping 10-foot long tentacles around Newt and creating an underwater feel.
Newt transports his magical menagerie in an unassuming suitcase that he’s able to walk down into, entering a ‘zoo,’ with each creature housed in a different environment. The backgrounds surrounding each pen appear to be realistically painted canvases, but with a moving, magical quality. Method’s team created a mangrove swamp within the suitcase environment, and incorporated creatures from four other VFX studios into the final composite. Method also created the snowy environment that is supposed to contain the Obscurus – a particularly evil creature that has escaped. Largely shot on greenscreen, artists added mountains, trees and a frozen lake, giving them a brush-stroked feel and layering them with animated snow.
“We spent a lot of time going back and forth with production and other VFX studios involved to figure out who was doing what, and the entire process worked surprisingly well. Everyone was very open about sharing materials and doing whatever was needed to complete a shot in which a flying Billywig gets eaten by a Doxy which in turn gets eaten by a Fwooper; it turned out beautifully,” said Hamish Schumacher, VFX Supervisor, Method Studios.
Framestore also produced a VR experience (above) using the same assets with textures by DNeg.
Inspired by the narrative of the J.K. Rowling’s film, Framestore VR Studio scripted the Fantastic Beasts experience for the new Google Daydream to make the user truly feel like the protagonist: a wizard with a wand in their hand, able to cast spells and control their environment, whilst tending to their Graphorn, Erumpent and Thunderbird beasts!
Amazing. Demonstrates what talent can do when working conditions are harmonious and the creature look, feel and movements are worked out prior to live shooting.