The SpongeBob Movie: Sponge Out of Water has already raced to become the second highest box office performer for films released in 2015, behind only Fifty Shades of Grey (!!!). That stunning result speaks volumes about SpongeBob’s existing fan base, but also new converts who gravitated to the hybrid 2D/CGI film. Helping to realize director Paul Tibbitt’s vision for the film was Iloura, who combined 3D animated characters with live action backgrounds. We delve into the making of the CG characters and live action sequences with Iloura visual effects supervisor Glenn Melenhorst, with a special look behind the scenes of the Patrick Star ice-cream sequence.
Watch a clip from the film when SpongeBob and his friends find themselves on a beach.
Building the characters
Bringing traditional 2D-animated characters into the 3D world had its challenges. But Iloura, wanting to stay true to their 2D forms, still had to integrate them believably into the real world. That meant, according to Melenhorst, that “they had to have a slight texture to make them feel less digital, and we need them to not look like PVC toys. We added small textural finishes to each character like pores in SpongeBob’s sponge, tiny bumps on Patrick, weave and stitches on socks and shoes et cetera to visually bridge the 2D to 3D worlds. When the characters were seen in mid to distant shots that detail all but vanished leaving the characters as simple, clean silhouettes.”
“We also used varying amounts of sub-surface scattering,” adds Melenhorst, “to bring a fleshiness to the characters, predominately when they were tiny. This backscattering translucence on Patrick’s skin and SpongeBob’s sponge and Plankton’s limbs not only helped sell the characters as tiny, it gave a warmth of tone and subtle complexity to the characters.”
Live action shoot
Halon had previs’d sequences prior to Iloura coming on board. lloura took this work, supervised the live action shoot and also postvis’d sequences, along with re-shoots and pick-up shots. To aid in integration, lighting and actor performance on set, maquettes of CG Iloura sculpts were 3D printed prior to the shoot by Gentle Giant. “They were fixed to metal plates and poles for puppeteering as needed,” says Melenhorst.
Watch part of the live action shoot with Antonio Banderas.
Later in the film the characters are featured as six foot superhero versions of themselves. “For the full sized stand-ins we used people dressed in simple frameworks and props to help with framing and eye line,” adds Melenhorst. “We plotted the size of each character and made sure the stand-in eyeballs were at the correct height for each character. Plankton was much taller still and a plywood board (with holes cut to protect against wind) was created. This was either walked through frame or fixed in place on set for reference.”
For each scene, LIDAR was taken, as well as HDRIs, with Iloura’s lighting team using them as ‘jumping off points’, as Melenhorst explains: “Much of the integration of character in scene is invisible to the audience but crucial to selling the characters as ‘real’. For instance, for every shot the environment was built by our modeling and lighting team to cast the correct reflections and ambient light back onto the characters. The whole of Savannah, used for the chase scene, was recreated digitally so that it would reflect correctly into Krab’s metal finish, or into the chrome of a bike, or even the sheen in Squidward’s costume.”
Animation
Animation from the 2D sequences to what would be seen in 3D had to be consistent. “We were acutely aware that people had spent many years loving these characters,” says Melenhorst. “Not just the show’s creators, but the audience as well. There was a huge responsibility to capture the performance of SpongeBob and the others in an authentic way. Compounding this was the fact that the audience will have watched 2/3 of a film with classically drawn characters before ours burst onto the screen. Our animation played back to back with the real thing.”
Watch the superheroes clip.
So Iloura spent significant R&D time looking at rigging, rendering and animation techniques for the characters to match their 2D counterparts. Melenshurst suggests that two elements were crucial to pulling it off: timing and staging. “Unlike many hybrid films (Smurfs, Chipmunks et al),” he says, “SpongeBob had a very snappy, distilled style of animation that might not at first lend itself to CG. Many people believe that CG can never stand still. The result is many films where the principal actors are standing around talking quietly and the CG actor is fidgety and weaving about. We studied as much SpongeBob as we could understanding their beats, both of acting and stillness, understanding how long to transition from pose to pose and how much to overlap the action. One thing we noticed was that SpongeBob seldom rotated smoothly in cel animation. As soon as his rectangular head rotated digitally he looked very wrong, so taking cues from the 2D animators, we turned him snappily, often with a compression of his body or animated accent.”
During animation, Iloura would guide each shot through a ‘post sculpt’ phase designed to provide for strong silhouttes (again a feature of their 2D world). “Here,” describes Melenshurst, “the key frame animation was studied and drawn over in discussion with Nick Tripodi, our animation supervisor, Dean Elliot one of our animation leads and Samuel Jensen our sculpt lead. We determined what frames or poses need to be strengthened with a custom sculpture to strengthen the character’s silhouette. This technique of sculpting on top of the key framing also allowed us to seriously break the characters for extreme poses and face shapes. We could also sculpt on top of higher density versions of the animation for face wrinkles and pulls in fabric et cetera.”
Effects
At one point Patrick Star is flung through the air and lands in an ice-cream cone. Iloura utilized effects simulation for the shots. “We built super viscous fluids in the shape of ice-cream that pushed and expanded when Patrick interacted with it,” explains Melenhorst. “Houdini was used for most of this work. After a while we began to see that the straight simulation wasn’t giving us the aesthetic we were looking for so Paul Buckley (FX supervisor) and I decided to default to what we knew would work and very simply built the effect out of morph targets in Max. Sometimes the low tech approach is still the best.”
For ice-cream that ends up all over Patrick’s face and body, Iloura created a series of wet maps. “We applied these maps as displacements to a copy of his body shaded like the ice-cream,” says Melenhorst. “We also made the ice-cream more glossy and wet as he mushed around in it.”
