In this first of four spots that The Mill has created for Sky with agency Brothers & Sisters, a TV set transforms into a spray of color, in a sense the television liquefies and drifts through a real space. Each spot has a unique set of droplets that aesthetically reflect the footage they contain and have their own subtle behavioral differences. In this first spot we see the classic red white and blue tones of footage from Marvel’s Avengers and in particular, Captain America, travel the length of a house. It’s one of the most beautiful spots of the year.
The Mill’s Dan Williams, Lead 2D Artist and Francois Roisin, Lead 3D Artist were joint Creative Directors on Sky Q spot. The piece was built by a team of 52 VFX artists over the course of 12 months. The project spent three months in R&D and intensive pre-production before principal photography. The shoot itself took place over three weeks. This was followed by 9 months combined of post-production work on the four spots. By the time the team at The Mill were finished they had made over 105 liquid VFX shots and over 4000 individual simulations with over 45,000 hours of rendering.
While there is a live action set for the action to play out in, all these sets were also recreated in 3d in order to have proxy or stand in dummy objects to catch rendered shadows, light or any other type of interaction with the CG fluids. A few shots ended up being entirely CGI due to the amount of interaction (breakout moment, driver in the toy car, first droplet splashing on wooden floor, etc) or simply because it was simply not captured on location. “Because the journey of these fluids were dictating our edit, we, on a few occasions, realized that we needed an extra shot that did not exist,” explained Roisin.
Williams commented that he felt that the spots need to be grounded in “a filmic and art directed reality”. All the environments and sets were shot by director Johnny Green and lit by DOP Lasse Frank. “We then rebuilt all the environments in 3D so the liquid could move correctly through the space. We also added a couple of fully CGI scenes as we developed the narrative of the journey of the liquid, and as we discovered certain sims that looked great from particular angles we’d re-design the scene around them,” adds Williams.
We asked Roisin if he considered doing a Lidar scan of the room. “Lidar was definitely an option when we first discussed the project with Johnny and Brothers & Sisters,” he explained, “but we decided to dismiss it simply because our set had to have a different layout in every shot. We kept on moving furniture, props, lighting in frame for every shot in order to craft nice compositions. Lidar would have captured a frozen set at a certain time and certain layout but it would have been too time consuming to repeatedly scan the set for each shot.”
Williams commented that having realized that Lidar scans weren’t going to work, they needed someone dedicated to “photograph, measure and HDR each setup”. The team also took back to their offices all the key props so their modelling team could scan them after the shoot, and have them as reference.
As for the CG fluid itself, Roisin explained it was not just an issue of producing a good fluid sim. As the script called for very choreographed motion the team wanted to be able to also get into the simulations and tweak them in ways that were not limited to just standard physics. “We had to achieve realistic looking fluid simulation while having a lot of control over it, so we decided to use Side FX Houdini software. It does feature a good fluid solver but Houdini is a software that allows to customize tons of parameters inside and around it. We wanted to provide sensible tools for the simulation team to get good initial results without having to tweak tons of parameters,” said Roisin.
With the long R&D phase the team were able to come up with good ideas of things to shoot for reference on set – such as tossing various different colored ‘bouncy’ balls through each scene (which helped with depth of field, strengths of shadow etc). It also meant, according to Williams, that they got to “explore different looks and behaviors for the liquid. It was highly subjective, as no-one had seen liquefied TVs before. How transparent are they? How viscous? Do they glow? Are they electronic or more like glass? We looked at many different visual approaches to find the right balance of visual qualities”.
The team started with traditional 2D concept drawings, but they soon felt that they needed to begin sims and render tests to get a real sense of how the liquid would move and react. “A lot of the final projection and texturing techniques were still being developed as we went into production, but run-up time allowed us to approach the shoot with confidence,” noted Williams.
The R&D and LookDev explored different ways to make a liquid look real but also containing the sense of the original source TV show or movie. The team played with transparency, viscosity, color palette, wetness, glassiness versus wateriness – all this had to be carefully balanced to obtain a satisfying result that the director liked.
“The previs was also extremely useful,” said Williams. “By this point we had done a lot of simulation tests, so we were able re-purpose these animations to block out the action of the droplets. We worked very closely with Johnny, Aaron at Brothers & Sisters and Lasse the DOP to construct a previs that gave us the bare bones of the shoot in terms of framing, movement and timing.
This was used as a jumping off point for each camera setup and it gave the team something much more tangible than a storyboard to discuss on set. “We were all keen to not be tied down by the previs and react to the shooting environments, so not everything you see was pre-vis’d,” adds Williams.
Previs is also key when a crew needs to shoot around a hero action that does not exist. “A classic post-heavy job would require a far more detailed and precise choreography but we felt we wanted to keep exploring and discovering as we went along,” said Roisin. At the same time the team were also working closely with Paul Mann at Machine Shop to develop hydrophobic liquid ‘second unit’ setups that the team could shoot for real. These proved to be incredibly useful as sim and animation references, and gave the team a reality touchstone which meant their CGI droplets ‘didn’t stray into anything too cartoonish’.
Perhaps one of the most successful aspects of the spot is the lighting. The liquid is beautifully lit both in terms of its own look and how it sat in the context of the lighting design for the plate photograph.
Roisin: “The backlit shots are something that we are really proud of. We had to create another version of our shader especially for strong backlights, at first the liquid was losing too much vividity and the screen/pixel texture was looking too sharp and digital”.
What is unusual is how the team incorporated Sub Surface Scattering (SSS), normally not a key fluid component being much more a skin wavelength dependent rendering construct. “We had to have a slightly different approach with SSS to feel the light shine through this opaque and colourful fluid. We also tweaked the outside edge of the liquid so it catches nice rim lights,” said Roisin.
The team aimed to take their lighting cues from what was shot, but they were always looking to add beauty to each shot and not be just constrained by the ‘correct solution’. For example, the back-lit droplets have a really great sense of depth that is emphasized through the crafting of “a good dollop of sub surface scattering. We had a lot of rendered layers to play with in compositing, so in 2D we were also able to animate color corrections and glows of textures to enhance the lighting interactions.”
Williams had worked previously with the director many times. “Because of the heavy amount of post, we were very much on the journey together throughout the process. Creatively he’s very inclusive, so Francois and I were always throwing creative solutions at him as each scenario arose. Johnny and Aaron the CD at Brothers & Sisters were very involved with us in developing the aesthetic language of the droplets.”
For both the Creative Director’s at The Mill, one senses that they feel there is a real danger with FX heavy jobs like this campaign to moving too far away from reality, so both were careful not to suggest impossible camera moves to the DOP, and everyone looked for real interaction moments in the art direction. “We also tried to limit the number of plates on set so that Johnny and Lasse could be as free as possible with their shot design and movement. The whole job was a very collaborative process between all the key players involved, and we’re all very proud of the results,” they agree.