In Melancholia, director Lars von Trier tells the story of two sisters, played by Kirsten Dunst and Charolette Gainsbourg, contemplating their own tense relationship as the Earth faces its greatest threat from a passing planet. We talk to visual effects supervisor Peter Hjorth, as well as Platige Image and Pixomondo, about how they brought a subtle and artistic feel to their effects work.
Melancholia is Hjorth's sixth film with Lars von Trier, with earlier collaborations on pics such as Antichrist and Dogville. Also visual effects producer Karen Maarbjerg is a long time collaborator. The result is that Hjorth became involved early on in the writing stage for Melancholia to help the director shape the story. "We played around with different colors and surfaces while Lars was still playing around with in the script," says Hjorth. "He wanted people reading the script to understand what we were going to do. And I also tried to make this polemic statement that everything you see in Melancholia is real."
Working with an illustrator, Hjorth produced a sketchbook template, while also gaining much research from a planetary science expert for the crucial shots of Melancholia as seen from space and from Earth. "I did some 3D concept work and some compositing just as tests," says Hjorth. "We very soon found out if you're showing something on the sky in an earth environment with an earth colored sky, it has to be very realistic. You have to look at how the moon looks, because it's the only celestial object that we have any idea of how it's shaded and how the atmosphere affects its colors. So I looked at the moon a lot."
Once principal photography began, Hjorth would shoot as many extra elements as possible, often using lunch breaks to acquire additional imagery. Hjorth was also the second unit director on the film, and worked closely with the DoP Manuel Alberto Claro while using the ARRI Alexa during production. "I love that camera," says Hjorth. "I would love it to be a 4K camera, but even at 2K, it's just a joy for the DoP and gaffer to have a really wide range of exposure to work with. That also gives me the capability to say, 'Let's try and emulate this with a real light. Let's try and do stuff.' If this was on a slower speed camera - trying to light a complete lawn, say, with colored lights, it would be out of the question."
Above: Watch a breakdown of the visual effects of 'Melancholia' with commentary from supervisor Peter Hjorth.
The film begins with a dream vision sequence punctuated by an operatic montage of slow-motion scenes hinting at the film to come, before Earth is shown being enveloped by Melancholia. Many of the overture shots before the collision were filmed on the Alexa and on the Phantom HD Gold camera at high speed, with visual effects handled by Platige Image in Poland under in-house supervisor Jakub Knapik and in-house visual effects producer Katarzyna Jarzyna. Although Platige's work amounted to only 12 visual effects shots for the sequence, they were given much scope from Hjorth and von Trier to continue the design work that had been started.
"Our work was usually starting with some basic sketches done on Avid by Peter and approved by Lars von Trier," comments Knapik. "We used them as a reference and moved on from that. We also received a set of files from Alexa and Phantom with all sort of plates that could be useful."
Platige's overture shots were mostly 'one-offs' and involved a combination of roto, matte painting and multi-layered compositing using Photoshop and Nuke. In one scene, Kirsten Dunst is seen in her wedding dress moving slowly through flying yarns. "In this shot we had to replace most of the background," says Knapik. "We had some shot plates with trees and yarns waving on the wind to comp it in. We also extended the sky there and to give the picture more depth, added some more matte painted trees in the back."
"That's funny," adds Knapik, "because even at such an extreme frame rate you simply feel that matte is too static so we had to add those small distortions on every added element like trees and so on. Some extra yarns were shot on black for easier extraction but we also used Houdini to add some more 3D ones flying close to the camera and higher above. It all gave that tense feeling to the picture."
One shot featured a 'spray' of bugs emanating from holes in the turf beside Dunst. "We had Kirsten shot on black background lit by a very strong backlight," explains Knapik. "So the first task we received was to change all the light on her, because the idea of lighting changed after the material was sent to us. For the background we mixed some shot plates and a bit of matte painting. The bugs where done in 3ds Max, simulated using thinking particles and brought to Nuke after rendering in mental ray. We had to model five or six bugs based on shot reference and animate the flapping wings that would match them. For close ups, we had a lot of plates with bugs shot in extreme slow motion on black. We had to pick the ones we liked the most, since they were shot in big groups, key them and compose a nice looking mixture of them."
Platige also contributed a key shot of Dunst examining her fingers as electricity comes out of the actress' hands and telegraph poles behind her in a St. Elmo's fire-like effect. "We started out with some 3D Houdini and Maya tests to create plasma curves but it did not give us the look that we expected in the timeframe we were given for this shot," says Knapik. "So we moved to some old school Shake set of plugins and made a number of big plates with animated electric bolts. This was not the most efficient workflow since the final composition was done in Nuke and needed a bit of back and forth work but finally it gave the desired result without 3D. Again, like in other shots, we had to additionally create the background from a set of plates and matte paints, fix the light on Kirsten to match background and add this interesting blueish flickering on her face and hair, because the blue light from the shoot was static."
Another signature shot in the overture was of a horse kneeling down below the Northern Lights. Peter Hjorth filmed the Lights himself on the Phantom after a couple of freezing nights in Iceland. "This shot is actually something that came from the scientists," says Hjorth. "They said one of the first indications that there might be a problem with planet Earth would be that you would see these kind of aurora effects - maybe even in the daytime. I tried to do some simulations and it just didn't look quite right. So I went to Iceland to film some timelapse shots, and I had a good time there even though it was a bit cold."
Platige then experimented with an interesting composition for the final horse shot after roto'ing the Northern Lights plates. Like the other overture scenes, Knapik says the shots were designed to be subtle and interesting, "something that you could hopefully see in a dream." Along with some additional effects work, including the placement of clouds in an aerial horse riding scene, set extensions, color corrections and minor fix-its, Platige contributed around 30 shots over two to three months.
For Knapik the work was an opportunity to break the usual rules of visual effects and search for something different. "The trick was that we were not hunting for typical visually rich, spectacular beauty shots but more for some subtle interesting pictures that would tell their own story. It was a lot of experimentation and work back and forth. Thankfully we did have Peter on board who was supervising all the shots in the show and was a link with the director and his strong and precise vision."
The planets collide
At the end of the overture, Melancholia is shown colliding with Earth, again in slow motion. Effects for the collision - the impact is over a minute long - and other scenes of the planet in the film, were realized by Pixomondo in Frankfurt (and at two of its other studios) under in-house visual effects supervisor Sven Martin and in-house producer Sabrina Gerhardt.
After receiving detailed storyboards and research from Hjorth, Pixomondo also consulted Dr. René Reifarth of the Goethe-University in Frankfurt to consider questions of what would happen if a planet with a hard outer shell collided with a planet consisting mainly of gas. "The peculiar thing about a gas planet, like say Jupiter," says Martin, "is that it looks like it consists of solid marble, yet it only has a very small solid core. The rest is layers of gas. In a collision with Earth, the gas planet would hardly change its shape due to its size. Shock waves and gas vortexes would wander over the surface. After extensive research we understood that a collision of Earth with a huge gas planet would look like a raw egg, falling into a giant drop of honey."
Pixomondo art director Max Riess then worked with Hjorth to devise a previs for the collision in 3ds Max. The collision was hand animated, with the impact forcing the gas layers to ripple and sprawl over the surface of Melancholia. Cloud simulations and fluid renderings were then performed in FumeFX by Pieter Mentz out of Pixomondo's Berlin studio. "These fume simulations were used afterwards to trigger Krakatoa rendered particle effects," says Martin, "creating hairy gas fingers erupting into the planet’s atmosphere."
Nuke was then used to composite the final scene. "The planets were not fully generated with a 3D program, but with Nuke's 3D space by texturing and lighting multiple layers," notes Martin. "For oceans and lava effects we worked a lot with animated textures. The gas and cloud layers of Melancholia are separately pre-rendered simulation textures. Only the necessary subsurface scatter pass had to be generated in 3ds Max. The most difficult task in Nuke was to merge the various elements together, creating the impression of interacting, slightly translucent atmosphere layers without losing the huge planet scale."
Melancholia from Earth
Pixomondo also contributed nighttime shots of Melancholia seen from the protagonists' point of view. On set, Hjorth relied on a glowing balloon as a stand-in for the planet to both light the scene and have something for the actors to look at. "Another sequence shows the appearance of Melancholia in the style of a sunrise," says Martin. "We wanted the shadows of the trees to move along so the team shot fireworks in a suitable angle into the night sky. We combined multiple plates and adjusted the timing so the shadows would move along with the animation of the planet. It was a little more difficult to mount the blue planet into a bright day sky, because luminosity as the combining element is almost completely missing. Therefore we surrounded the planet with additional digital clouds which then made a soft transition of the blue light into the sky."
In the film's final destructive moments, Melancholia and Earth collide and the impact is shown as an almost nuclear blast engulfing Dunst and Gainsbourg. Pixomondo completed the shot based on initial reference from Peter Hjorth. "I did a whole reel of nuclear test footage for the artist," says Hjorth. "When I think about that shot again, I think it was basically my upbringing and nuclear fears going into it. I didn't realize that until later that this is what I was doing. This is something you think about when you're growing up in Europe in the 1980s."
"I knew I wanted the shockwaves and the trees bending and the fumes coming off the trees," adds Hjorth. "They were the most important parts of the shot. I filmed some fire elements and we built a model of the area where the actors were sitting and blew it up a couple of times, and shot it on the Phantom camera. Lars saw it and said, 'Well, this is really beautiful but probably not for my movie - I like these eight frames from here to here!'"
The shot of the characters waiting for the impact was something added late in production, with a last-minute re-shoot to give Charlotte Gainsbourg's character more panicky movement. "I had this feeling that it would be a like a big helicopter about to descend on the people there," comments Hjorth, "so we put some turbulence in the air and some extra lens flares and shaking and debris flying around."
For Pixomondo, the challenge was to combine several real elements into the nuclear-like blast. "We could not combine the elements in z-depth as we could have done it with CG elements," notes Martin. "Because the shockwave should move towards the camera, every single plate had to be timed and arranged in a way that the impression of a seamless forward motion could be achieved. These real elements were combined with digital broken trees, debris and air shockwaves. It was especially difficult to maintain a certain depth within the scene, because it was not possible to integrate an atmosphere into the scenery. Realistically it would then already have been swept away by the planet. In order to let the hair of the actors fly stronger with the shockwave, we placed Travis Nobles, compositor of the shot, due to his long hair in front of the green screen and used ventilators to shoot additional hair elements. This was faster and generated more variation than a simulation of digital hair."
Beautiful and subtle
In addition to Platige and Pixomondo, the visual effects of Melancholia were completed by Filmgate in Sweden, Kingz Entertainment in Germany and Klippengangen Aps in Denmark. Very much part of the creative process, Peter Hjorth continued to push for realistic effects that also helped push the story. "It's actually a constant sparring situation with me and Lars," says Hjorth. "I'm trying to do beautiful and interesting effects shots, and he's trying to do subtle shots all the time. I'm looking at a scene, and I'm saying, 'We need to show the planet here and here otherwise the audience are going to feel it's just a talking scene.' Usually we'll design a few more shots than what Lars is going to use in the film. And hopefully we use some of the back-up shots in the film."
If you liked this article you might like to also check out this vfxshow podcast for our discussion on the films 'Tree of Life' and 'Melancholia' with TyRuben Ellingson, Matt Wallin and Mike Seymour.
We've been a free service since 1999 and now rely on the generous contributions of readers like you. If you'd like to help support our work, please join the hundreds of others and become an fxinsider member.