The Oscars are a chance to reward and celebrate the work of VFX artists, – but on what basis should an Oscar be awarded in the category of Best Visual Effects? Dune has been nominated in the category for the 94th Academy Awards. In talking to Dune’s Supervisor, Paul Lambert, and the two DNEG VFX Supervisors, Tristan Myles and Brian Connor, perhaps the ‘answer is in the plate’, as Paul often comments.
In the past one might have argued that the award has gone to a combination of three things:
- the newest or most original VFX, (e.g. The Curious Case of Benjamin Button)
- the biggest and most complex or challenging VFX, (e.g. Lord of the Rings etc) and
- how much the VFX alone contributed to the story and the film as a whole. Could you have told this story without VFX? (e.g. Gravity)
Perhaps it is also important to note, the VFX Oscar rarely went to a film with limited appeal, great VFX in a corny or poorly received film was, and is, a hard sell to the Academy members. It is also worth noting that nominees reflect the opinion of fellow VFX artists and at the final award-winning level, the wider voting population means more politics and more influence of a less technical nature in evaluating the work. This means seamless effects can be rewarded by a nomination, as we all know how hard it is, but seamless work is often lost on the general Academy voters. For years, the ‘biggest’ VFX film always won, until recently when smaller films such as Ex Machina and First Man won. Both were excellent films, but neither were mega-blockbusters nor the biggest films of their respective years. Those wins may represent a shift that reflects the pervasive role of VFX. VFX is no longer ‘post’. Films such as Jungle Book which won in 2016, was not a film that had visual effects added. VFX was everywhere, in every department, in every way.
Paul Lambert won in 2018 with Blade Runner 2049 and again the next year for First Man. Most experts would agree that Dune is a major contender to win this year, which would be a truly remarkable achievement for Paul Lambert if it is successful. But rather than discuss the merits of the film or the competition to Dune from the other great films in contention this year, we spoke to the team about how they did the VFX in Dune. In so doing it becomes clear that we might need to reconceptualize the notion of evaluating VFX as a separate entity apart from the other departments such as Camera, Wardrobe or Art Department. Dune works on so many levels but this new approach to judging the work lies in Lambert’s comment ‘the answer is in the plate’.
The Answer is in the Plate
Paul Lambert didn’t invent the phrase “the answer is in the plate’, but he completely embraces its message. “That quote actually comes up on every single show I’ve done,” he explains. “If you analyze the plate to which you are compositing, you will see what edges should be doing. You’ll see what the lighting is supposed to be doing. You’ll get an understanding of focus or any chromatic aberration. Basically, get down to the pixel level of the small subtleties, and that’s it.” If Paul Lambert is getting a comp presented to him and something isn’t quite sit in believably, he only has to A/B with the plate to know what the issue is. Encapsulated in this approach implicitly is the need to have things shot in the plate. Having practical things done on set for Dune was not a matter of convenience or possible later use as reference. To perfect the imagery of Dune, it was pivotal to get relevant imagery in the frame to produce the quality of work the Director and the team wanted. Practical items correctly included in the shot became a key to the IP of DNEG’s stunning work. So rather than picking up whatever first unit might be able to provide as reference, the VFX team worked darn hard with every single department to capture relevant intelligent reference in shot, and then to precisely and very diligently support that work until the final release of the film.
Too often we hear about things being shot on set and then later being replaced in ‘post’. The popular press jump on the notion that things were done ‘for real in-camera’. In our VFX community one often sees a backlash pointing to how so much of the ‘real in camera’ work was digitally removed, and computer visual effects were used instead. It is clear to our community that this was the right decision, and the phrase often appears, “but it was great reference”. But the subtext of this is, “we don’t want to offend the practical guys – who worked really hard – but hey, in reality, visual effects are actually done in the computer – you just won’t admit it.” This is not how DNEG and Paul Lambert approached Dune. Rather than consider how much VFX added later to the film – separate from the other departments, this points to is we should actively consider how well the VFX team managed to get the other departments to integrate into one vision.
Director Denis Villeneuve, the creative team, and heads of department did not try to do as much as they could in camera because they think practical is better, – they tried to get the highest visual fidelity and realism by collaborating with every department in complex ways to produce meticulous footage that audience can respond to as real. They painted internal studio ceiling and lighting rigs to better mimic bounce light, they used sand-colored screens angled to catch the sun to get the right illumination in the background, the team built complex rigs to provide real shadows from the actual sun, projected slices of 3D geo on the actors faces and a thousand other thing to get the visual truth in the plate, so the creative artists at DNEG could find the right answers. Find out how to make dragonfly ornithopter fly, composite over a sun-bleached Arrakeen spaceport, or render the sand displaced by an iconic sandworm. Without this attention to visual elements in the plate, Paul Lambert explains “you can spend the majority of the time actually trying to get things to look believable rather than being able to iterate on the creativity of what you’re trying to do.” Brian Connor adds that “Paul is very aware of how things needed to be real,” adding that, “we always have to prove it. We’re not allowed to cheat anything in any kind of environment we had to set it up correctly and show or prove that that’s what you would see if you were there, it was very, very meticulous and it always had to be based on something real.”
Lighting Arrakis Example
For the scenes when the actors are inside the Arrakis Palace about to head out to the ornithopters, the production was filming at the sound stages. Rather than just mount bright lights to illuminate Arrakis behind the actors, the production joined two sound stages together and covered it in white silk. The DOP Greg Fraser then had a large section left open and he bounced the actual sunlight from outside into the set with a huge white reflector, this was the primary light of the scene. But it was clear that the Director would want to see parts of Arrakis outside behind the actors. To get solutions into the plate the art department built parts of the textured walls which would ultimately be CG walls. “We had those positioned outside by the white screen so that like we could visually see, ‘okay, at this exposure, this is how much we actually see of the actual walls outside, and we had something to match to for the CG city,” Paul Lambert explains.
The same approach was used for the overhead lighting in the Nexus, when our heroes are down inside an old concrete disused environmental factory. This time a hole between the Budapest studios was cut above the actors and for about two hours a day, the DOP could film the actors correctly stepping and out of shadows in completely visually correct exposures and contrast ratios. Something very hard to get right with artificial light. Not only that but the light rays needed to be parallel (as if from a far-off sun) so any artificial lights would have had a very different quality of light, as no light can be positioned high enough to replicate such direct sunlight and shade.
Set extension framing
When it was impossible to build a full set, not only did the team add stand-in items, but DNEG build a UE4 augmented reality iPad App using ARKit, that allowed the DOP or the cast to see the empty spaces with live interactive AR overlays. This allowed anyone to walk around the scene and see structure-wise how big everything was, and where a spaceship would be hovering. This helped both the actors and the DOP. with blocking and framing. Based on this special prop sections or references could be positioned in shot.
For the studio work, they also shot with sand-colored screens instead of bluescreen, these screens were also angeled back to catch the light and not be in shade to provide the right illumination and backlight. Inside the studio “we actually had crossbars and pillars for the roof. The ceiling was wrapped really cheaply, just with scaffolding and silk, which had been painted the right color for the Arrakis location. That allowed Greg to light the scene exactly as if he was in the actual environment,” Paul explained. Only by such complete dedication to correctly informing what is in the plate does a team get visual and narrative authenticity and end up with masterful VFX shots.
On the ocean planet of Caladan, a spaceship rises from the water. This was not something that could be shot due to the vast scale and there is also no real reference for massive ships rising out of the sea. The team did hunt for anything to refer to, ” National Geographic is your friend in cases like this,” Brian Conner joked. The team did find a huge ice chunk that broke off a glacier and flipped but it was still not perfect. To reference reality, Paul Lambert came up with a reverse approach. He asked the team to temporarily composite in a correct scale 6ft man in a small fishing boat, right beside the vast submerged House of Atreides flagships. Once that was in the shot, as a size reference, it was possible to judge if the water looked proportionally correct. “And right away as our brain registers,” says Brian. “Obviously you could see, ‘oh, wow, these things are going way too fast, they’re way too big, etc’. And so it really helps us with that, that sense of scale.”
Filming the ornithopters was not viable given real-world physics but the production did build 2 big practical 12-ton ornithopter bodies and hung them from cranes and blasted them with sand. Combined with carefully choreographed helicopter shots, the DNEG team produced believable and authentic ornithopters. To get the correct external footage the team flew for hours around the UAE filming with a bank of cameras. In that footage is artifacts from the actual choppers but these were all kept as they are the sort of details impossible to guess as an artist and yet somehow seem to make complete sense when you see them combined in a shot.
For shots looking into the ornithopter, the production used a gimbal wrap around by a high sand-colored angled wall, nicknamed the ‘dog-collar’. This provided the correctly colored, up bounce light. The team even discovered they could film with glass panels in the ornithopter, as the reflections were out of focus. This saved an entire set of digital glass replacements when filming the actors appearing to be inside flying.
Imagery in the plate is informing so many subtle aspects of a visual and story fidelity that it really is worth doing, and doing well. This is only possible with complete integration of the visual effects agenda and that of each of the key below-the-line departments.
For the worm attack on the huge spice harvester rig on Arrakis. During a flight, Paul Atreides (Timothée Chalamet), his father Duke Leto Atreides (Oscar Isaac), and the team spot a sandworm approaching an active spice harvester with a stranded crew. Leto and his team rescue the workers moments before the sandworm swallows it. This could have been almost all CG with actors composited from green screen. Rather than take that approach, a vast tractor tread prop was built by the production designer Patrice Vermette in Budapest and then shipped to Jordan for filming in the sand dunes. Greg Fraser had the track positioned carefully in relation to the sun so that it consistently was in shadow. This allowed the production to shoot at the spice harvester location for two weeks. “But having that meant we had the right shadow to build out from, which we won’t have had otherwise,” explains Paul Lambert.
As the Sand Worm approaches the sand vibrates and this was again simulated on set with a giant vibrating steel plate. This caused the actors to sink into the sand, just as the script suggested and it looks real because it is, even if the scale of the Worm would require a vast extension of the idea in this and later scenes. “We didn’t have to do anything else to those shots apart from putting the rest of the spice harvester behind them, but that’s all, – it’s all physical,” he adds. Paul Lambert believes in tight collaboration with all the specialist areas. He has stated numerous times that he was on set with some of the most amazingly talented in the world. He gives credit to the Director, Denis Villeneuve, that everyone is there to do their best. “Props, stunts, special effects, visual effects, lighting, whoever they are, – they are all there to do their best. What can happen if you leave things to do in post, is that you are going to miss out on that onset wealth of experience.”
In Jordan, they also had an ornithopter body on a crane which the special effects supervisor was constantly blowing dust towards to actually simulate the wing downdraft. Tristan Myles points out that in addition to having the tread built out there in the desert as a lighting reference in shot, the vast scale of the built prop informed the actors and their performances. “The actors have got great things to visually work with and we’ve got great things to study and build on,” he comments. For this sequence where our heroes are moving between the ornithopters and the harvester, the final images were a combination of multiple shot references. The onset photography was filmed at the red Military Dunes in Jordan, but the ground was built digitally. DNEG took the plates, and their environment team painstakingly sculpted the dunes from that area. “Then we incorporated in a section from Abu Dhabi with UAE desert plates, which were sharper, and sculpted to look much bigger,’ he explains. Then behind where the harvester was a mountain range was added based on reference from the Golan Heights in Israel. This was all carefully shot and managed so that early ideas and references were maintained and communicated to all of the artists involved and not lost in iterations or conversations.
The DNEG created the digital assets to exactly match with the ornithopter body on a crane or the reference that was shot with real choppers, “because like we had perfect reference,” explains Paul Lambert. ” I am a big believer in filling the frame with all the effects which you’re going to want to use. And yes, it’s a harder composite, but coming from a compositing background, I know that this can be done. Yes, there can be a ton of roto but it goes to the core of the art of compositing: taking an image and then being able to mess with it and seeing all the answers in the plate – so that when you add things to it, – it all works.”