The tricky tracking task of Fungus the Bogeyman

Visual effects practitioners are often asked to solve complex problems – the team at DNeg TV had a major one on their hands for the UK Sky series Fungus the Bogeyman. Here, the children’s picture book character, notable for his vast coat, large ogre-shaped head and green skin would have to come to life in the most believable way on screen.

When it was determined that Fungus and his friends were to be portrayed by human actors wearing coat costumes with CG heads, DNeg TV quickly had a tracking problem to solve – how could the heads be tracked onto their live action counterparts convincingly, efficiently and on a TV budget? The studio drew on a strong history of R&D in 3D object tracking and photogrammetry, plus some ingenious on-set head tracking rigs, as well as new techniques in rendering, to make the show possible – completing more than 900 visual effects in three months for the mini-series in the process.

fxguide breaks down DNeg TV’s technical and problem-solving process behind the daunting task.

Step 1: The look of the bogey people

Fungus was produced by Andy Serkis’ performance capture outfit The Imaginarium Studios in partnership with DNeg TV, which got to work on the look of the characters. “We ended up doing all the concept design in-house at DNeg,” says visual effects supervisor Hayden Jones. “There’d been some concepts done but we took it back to the original book by Raymond Briggs and thought about what makes Fungus look like Fungus.”

Although The Imaginarium was behind the show, the production did not rely on a motion capture solution for the bogey characters. Ultimately, the bodies would be live action and the faces all-CG, realized with keyframe animation. “Fungus has such a wide face shape and has such definition in the soft tissue,” notes Jones, “with lots of jowls and sags around the mouth. We were worried that unless you had animators making conscious choices about how much sag to put in, it can sometimes look a little rubberized.”

For the same reason, a prosthetic or puppeteered solution for the bogey people heads was also not taken. However, prosthetics were made in the hope they could be used for background characters. “The thing is, the CG turned out so nicely that there was a kind of subtlety that the prosthetics didn’t really have,” says Jones. “Also, they were quite bulky, but they did help us on deep background characters and were great for publicity purposes too because you could have Fungus turn up to events. But once we proved we could get a performance out of the CG, then it was that way, all the way.”

Step 2: Tracking tests

Before the show was greenlit, production sought a suitable method for shooting scenes to be inhabited by the bogey people (ie. pretty much all of them). A proof of concept test was devised and made possible with some Sky development money. “We were able to use that money to do a five shot test where Fungus is walking around a house and accidentally wakes up the inhabitants and they see him and he flies out,” states Jones.

That test was also the first tryout of a rigid head tracking rig to be worn by the actor. “We thought we could get away with some form of LED tracking markers just around the neck area with a ‘hat rig’ above the head as a general outline for Fungus’ head to help with framing,” explains Jones. “But what we found out during that test was that the clean-up was way more complex and expensive than the actual benefit of having these rigs.”

The problem with that initial head tracking rig too, says Jones, was that as the actor moved around the points from which DNeg could track kept getting lost. “Because all the lights of this first rig were recessed into his neck area, as soon as the body actor turned or did a large movement, we would go from having virtually all the tracking marks on display to none. And that was a really problematic for our tracking tools. In the end, for the test, we essentially just body tracked it.”

Body tracking clearly was going to be too time consuming and cumbersome for the hundreds of shots required. DNeg needed to find a more autonomous or at least semi-autonomous solution for 3D object tracking. So, a new head rig was devised that, when used in conjunction with the tracking tools – mostly 3DEqualizer – would allow for more flexibility in the track and let the actors move more freely, while removing most of the need for clean-up and roto work.

“What we came up with,” explains Jones, “was a two-ringed array. “It was like a little metal cage that sat over the actor’s head and was bolted to the costume. The lower ring and the upper ring had LED tracking markers and they were designed specifically so they always sat within the profile of whichever character they were going to eventually be. We knew then that, for 95 per cent of the time, as soon as we put the CG head over the top and composited it on, we would never see a tracking marker or any of the rig at all. So there was no clean up for most of the shots. On TV budgets that is absolutely essentially.”

The new head rig was covered in LED tracking markers owing to the dark cave environments that the shoot was to take place in. “We also did put on some gridded bits of colored tape,” adds Jones. “These were just as a secondary back-up that, since we were using photogrammetry to survey the head rigs, worked better to provide clear markers and give a more accurate solve.”

For tracking, DNeg relied on 3DEqualizer and hand-tracking in Maya, greatly aided by the ringed tracking rig which enabled the presence of many tracking markers. That rig was also photogrammetry’d on set. “Photogrammetry enables us to essentially do 3D scans and texture reference at the same time and incredibly quickly on set,” says Jones. “One of the things that always happens on a TV shoot is that there’s never enough time – you never have the access you need, everyone is so rushed into getting what shots they need for the day done, that there’s no time to stop and let us take the actor away for 40 minutes and do a full scan. You normally get two minutes and if we can quickly grab a little photogrammetry set of pictures, it’s so important.”

The photogrammetry process involved taking a 360 degree array of still photos with a Canon 5D equipped with a prime lens. “I like to sit the person in a chair,” outlines Jones, “then get them to face forward and if we can circle them about three or four times taking a photograph say every 10 degrees and get them low, mid and high angle. Then if your actor can stay still for about a minute, you get not a bad scan. We use Agisoft’s PhotoScan to produce the 3D data.”

Step 3: Bringing CG heads to life

Green skin has traditionally been very hard to do on CG animated characters. DNeg approached the bogey heads by spending significant time on modeling, rigging and texturing. “We knew we were going to have to get really close to Fungus’ face,” says Jones, “so we put in a lot of time and effort into texturing. There were a lot of levels of skin texture – micro-bump as well as just normal pore detail. But the main thing was in the rigging. We had to make sure the way the character was sculpted and rigged gave the animators enough flexibility to get those really beautiful natural small movements in the face.”

The voice recordings of principal actors Timothy Spall and Joanna Scanlan, playing members of the Bogey family, were filmed and referenced by the animators. Interestingly, one adaptation the animators had to make for the CG heads was related to the head rig worn by the on-set performers. “The actors were quite free to move their heads around within the head rig,” notes Jones. “What would happen is you’d have two actors on set and they’d turn their heads to look at each other but they wouldn’t turn their bodies. We had to enable the CG heads to turn without having too much body movement – it was a huge challenge.”

After animation, the CG heads and other effects in the scenes were rendered using Clarisse iFX from Isotropix. DNeg has launched into a major partnership with the software company to use Clarisse for lookdev and rendering. “What Clarisse really gives us was a beautiful lighting setup,” suggests Jones. “We did surveys on set to have accurate lighting positions of the flame bars and on-set lights. That, coupled with the multi-layer subsurface skin shading, really gave it this amazing look.”

That on-set lighting survey made use of a Spheron camera as the primary capture medium for HDRIs. “Andy Hargreaves, our lead CG artist, then took those lighting spheres and dissected them, took out some of the lights, recreated them back into 3D space,” explains Jones. “So with things like the flame bars, we ended up using animated flame elements to give you that really accurate lighting and shadowing. The shadows don’t just pulse, they really change position as a character walks past a flame bar – all those little things gave you that sense of reality.”

But DNeg’s work was still not done. In compositing, it was still necessary to add some finishing touches to the way the CG heads were married to the live action costumes, especially for the neck connections. “Our lead compositor Dave Bannister developed a little warping tool that enabled us to take some locators that were output by the 3D department and do little post warps on the costume,” says Jones.

“Warping the characters’ neck line, albeit very subtly, helped marry the animation performance to the plate photography,” adds Bannister. “We needed this to be as automagical as possible since we had to minimize the tinker time per shot as our workload was challenging. We were able to generate motion vectors from pairs of locators, the first following the match moved head and a corresponding locator attached to the animated head. These were then used as the origin and destination areas for the NUKE warp tool. The effect could then be dialed in at any level to achieve the best result for the shot.”

Jones tells fxguide that the show was a real technical triumph at DNeg TV, but one that the facility was well set up to handle. “I don’t think Fungus would have been possible at a smaller facility without having the access to the broad base of bespoke tools and R&D that Dneg has. When you’re doing 900 shots in three months, the speed you need to turn those shots around is phenomenal. DNeg really gives you the backup of a big facility, but Dneg TV means we still feel like a TV team.”