The Mill for Coke: Super Bowl Revisited

In a broadcast filled of commercials with talking animals, dancing animals, and even animals with a point to prove, the Coke spot “It’s Mine” stood out from the crowd. We speak with The Mill’s creative director Angus Kneale and Senior CG artist Ben Smith about the work that went into making one of the most popular spots from the big show.

There are two video downloads which accompany this article. The first is

a 720P version of the final spot which you can download. The other is The Mill’s making of video for the commercial, which is part of their video podcast.

08Mar/themill/dogThe original plan for the production of the spot was to use practical baloons for approximately the first third of the commercial, up to the point where they start to fly away. At that point, CG balloons would take over and the animation would be in the hands of The Mill. “I’m always a big fan of getting the effect in frame,” says The Mill creative director Angus Kneale. “I’ve always liked that as approach because it’s a very filmic approach…you can see what you’re shooting and frame it nicely. You get good lighting reference, texture reference and it’s real so it gives us a good anchor for the 3D work.”

Director Nicolai Fuglsig is also a huge fan of getting as much in camera, according to Kneale. The production had a great location in New Jersey which looked very much like Central Park West in New York City. However there was a snowstorm on the day of the shoot and the filming needed to be moved to Paramount Studios in Los Angeles. An existing 4 block city set was used in hopes of integrating it with the rest of the spot.

Once they got in post, they felt the footage looked fairly unconvincing. “The scale just didn’t translate very well,” says Kneale. “It felt very crowded with the ballons there and it seemed very small and constricted (on camera). After we started using the 3D balloons and putting them into the plates we shot at the actual parade this year we suddenly realized we couldn’t put the two worlds together because the live action stuff would actually bring down the CG stuff.”

Another concern leading to a CG-only solution was the fact that some minor changes needed to be made to the practical baloons. The turnaround for building the practical baloons was over six weeks and with the shoot three weeks away, word came that the design of the baloons might need to be modified. Since it was impossible to stop the phsyical production of the balloons, the creatives approached The Mill about augmenting and changing the practical baloons. This wasn’t really a very exciting option for the cgi crew – and could be a very difficult process to modify the in-camera heroes. So in the end, all an CG-solution for the hero characters was ideal.

In the middle of December, The Mill started to show their clients tests of fairly final balloon renders. According to Senior 3D artist Ben Smith, “everyone was blown away by them…really loved it and kind of inspired by that.” Most of the plates were shots that director Nicolai Fuglsig filmed on the day of the NYC shoot, including helicopter shots of the city. However, based upon having to address redoing the scenes shot in LA, the Mill needed to recreate the scene by photographing stills and using camera mapping to rebuild the environement.

08Mar/themill/previs
The Mill did extensive previs to determine the speed and pace of the spot and animation

Previs and the Edit

The Mill did a comprehensive previsualization of the spot, which was critical for creating baloon movement which was believeable. “It was really a fine line to nail it,” says Smith, “in that too fast and they lose weight and look unbelievable immediately. Too slow and you had real edit problems because you had to convey a certain amount of action to carry the story along and keep it interesting, but you couldn’t end up with five or second shots.”

The aid the offline editing process, the Mill provided editor Russel Icke of Whitehouse LA with a series of stills, including background plates and various angles of the baloons to animate in 2D. The process of creating a roughcut for such a project isn’t easy, as “it’s quite difficult to imagine something that isn’t there,” says Smith. Several different versions of the cut were provide to the team at The Mill.

This provided a framework for the narrative, but none of the versions worked particuarly well in terms of the animation. Working with the editor and creatives, The Mill was given a certain amount of creative control to tweak the edit and create an animation-led series of plates that worked to tell the narrative. With the tests that the Mill had done, they knew what would work and what wouldn’t, so influencing the edit was crticial to end up with an effective spot in the end.


08Mar/themill/anim_grab

During this process, as they tweaked the edit, they were also roughing in placeholder animation of characters in the scenes to make sure the timings were right. By the end of the roughtcut process, they had a really good blocked out guide to work from. “When it came to animating the real shots, we knew exactly the start frame and the end frame positions and what we needed to do in between time. At that point it was a case of just really finessing the animation to make it look believeable.”


On the collision shots where the baloons either bounced off each other or off something in the environment, the Mill used a combination of traditional animation and Maya’s nCloth system. They built a bunch of presets from scratch to define how inflated they were, how bouncy they were, how quickly they’d compress and decompress. “It was really a case for the big collision shots…like the shot where they first collide into the Coke bottle…of trying many different versions and making tweaks and ending up with a version where everyone is pleased with”

For most of the wide shots, it was fairly traditional enveloped character where the balloon was subtly animated to give the impression of scale and how the wind would move them.

The Modeling

The modeling of the baloons was a fairly standard process, starting with references from the actual Macy’s Thanksgiving Day Parade, creating drawings, modeling, and then creating turntable renders to have the models approved by the client.

While the modeling was quite standard, a tremendous amount of work was put into the texturing of the characters. “In fact,” says Smith, “there was one guy whose job it was to do all the creases for all the characters and he spent the entire duration of the project just doing that. He was our crease master.” It was the first time The Mill had used Mudbox as a sculpt tool to create normal maps. These were then applied to the characters and rendered through xsi in mental ray. “We built lots of custom brushes in Mudbox based on reference footage of balloons which were then used to create the seems and creases. All of the creases and folds for the balloons were created in this way,” says Smith.

08Mar/themill/passes
A sample of the passes provided
08Mar/themill/beauty_pass
The final beauty render

The Pipeline

The spot was finished in high definition at 1920×1080 24P. The SuperBowl was broadcast on FOX, which is 720P, but 1080P format provided more flexibility for other networks as well possible theatrical release.

The Mill’s technical staff ahve written several different proprietary tools to transfer 3D geometry between applications, allowing artists to use their preferred application. In the case of this commercial, the lighting, shading and rendering was done using Mental Ray in Softimage and all of the animation was done in Maya. The custom tool allows effective interchange between these apps. “The animator can be animating away on their shot and pass a version of the geometry on to the renderer,” says Smith. “Then the renderer guy can get all his shaders and render passes and lighting all worker. Whenever the animator publishes a new version of his animation, it updates straight away into the renderer’s scene. He doesn’t have to go and manually import or update anything — it all happens automatically.” The tight deadlines of a commercial project such as this are made much easier by the ability to work concurrently on scenes.

Six or seven standard passes are output by the CGI team for final compositing — passes such as reflection, diffuse, specular, etc. In most casses, the passes are then pre-comped in Shake in the 3D department and then those pre comps are taken to flame with any additional needed passes and really finished off and polished. The passes are rendered and composited 8-bit for commercial projects such as this.

“A lot of times places just do loads and loads of passes…and then they just hand it off to a separate Shake compositor to put it all together,” says Kneale. “We’ve found it’s very productive to have the actual person who is rendering to put together a rough composite of what they are trying to do. In a way you are trying to see what is in their minds, and the guys we used actually took it pretty far.” These aren’t final composites, however, but blendings of the CGI layers over black which are then handed over to the flame artists for final tweaks to take the composite further.

The Mill has developed custom tools over the last three or four years to allow the various flame and smoke systems to work together on a project. Setups from all users are available to all systems at any time, and there is a system of setup and version notes which helps artists exchange information between each other.

The Work

From a vfx standpoint, one of the most complex scenes in the spot is the third scene in which we first see the hero balloons up close. “Because the background for that was this mockup New York street scene from the backlot, it just edit-wise jumped out and looked really, really wrong,” says Smith. “So for this shot we basically took the foreground people, keyed them off — the flame artists did a great job of this — and composited them onto a CG rendered street of the actual location of Central Park West.”

They tracked the camera move from the original plate photography and then applied this move to the CG scene so that they could composite the foreground people correctly. The crowd in the street which runs perpindicular to the main street was generated using Massive software. On top of this, the CG rendered balloons and ropes were added, creating the final scene.

A scene which looks uncomplicated but really isn’t is the scene in which Stewie head butts Underdog. By the time the original plate was photographed, the day had clouded over and the scene was completely flat and overcast. “The flame boys did a great job of adding in directional sunlight to match the shots around it,” says Smith. “Another thing on that shot is that we had an issue of scale in the sense where all of the sudden the characters felt quite small in that shot. So in order to augment the scale what they did was resized all of the windows (smaller) and added in extra windows to make the buildings feel bigger and thus the characters feel bigger as well.”

08Mar/themill/windowFor exterior shots, the CGI crew at The Mill would rebuild the building geometry in the scene to have the shadows and reflections automatically cast when rendering. For the interior shots, there was a great deal of work that was done to finesse the interior scenes where the characters passed by and cast shadows in those environments.

Late in the spot there is a scene where Stewie and Underdog crash into the window where a janitor is working in an office. Blocking the windows would of course block the ambient light entering the office — yet a plate for this would be difficult to acquire. So in the end, there was an incredible amount of 2D paint work on the scene that required all the highlights and pings and reflections to be painted out of the scene. The flame artists could then use mattes from CGI and masking to animate between the daylight scene and the painted dark scene.

“There was a huge amount of work done for the interior scenes,” says Kneale. “We actually went as far as pretty much painting out all the reflections, all the highlights and specularity of the objects in the scenes. We ended up with effectively a flat pass and then we recreated the highlights and tried to simulate the animation of the light source as the balloons passed by the windows. There was an enormous amount of time spent on the three interiors.”

There was about two months from preproduction to post, with about six weeks available to do the CG animation and rendering. In order to help with deadlines and possible changes, 2D prep work was also being done to the scenes before the final renders were complete. The final 2D compositing inolving integration of 3D elements took place over a week.

Credit List

Agency : Wieden & Kennedy

Creatives: Hal Curtis, Sheena Brady

Producer: Matt Hunnicutt

Production Company: MJZ

Director: Nicolai Fuglsig

Producer: Emma Wilcockson

DP: Ellen Kuras

Editing Company: Whitehouse LA

Editor: Russel Icke

Assistant Editor: Joanna Manning

Editorial Produce: Justin Kumpata

Post Production, VFX & Animation: The Mill New York

Creative Director: Angus Kneale

VFX Producer: Bethan Thomas

VFX Supervisors: Angus Kneale, Andrew Proctor, Asher Edwards

Lead Flame Artists: Angus Kneale, Dan Williams

Flame Artists: Pheng Sisopha, Randy McEntee

Smoke Artist: Jeff Robbins

Support Artists: Greg Gilpatrick, Anu Nagaraj, Suzanne Dyer

Senior CG Artist: Ben Smith

CG Artists: Andrew Proctor, JongJin Choi, Keith Kim, Glen Swetez, Rob Petrie
Kevin Ives , Justin Zurrow, Wyatt Savarese, Mike Panov, Joshua Merck, Dylan Maxwell, Douglas Luka, Vince Baertsoen, Yann Mabille, Tomas Salles, Yorie Kumalasari

CG Producer: Asher Edwards

Shake Compositors: Andrew Proctor, JongJin choi

Telecine Artist: Fergus McCall

Asst Telecine Artist: Alex Maxwell, Sal Malfitano

Telecine Producer: Angela Botta