For Black Swan, director Darren Aronofsky turned to Look Effects to help bring his dark New York ballerina story to life. Look’s visual effects supervisor Dan Schrecker oversaw 220 shots for the film, from full CG swan wings to prosthetic augmentation, face replacements and other enhancements.
Having worked with Aronofsky on all of the director’s previous films, including Requiem for a Dream and The Fountain, Schrecker continued his co-operation with the director this time as a vendor at Look, and the visual effects continued to remain just part of the storytelling process. “The film, which is about this complicated relationship between Nina (Natalie Portman) and her new ballet rival Lily (Mila Kunis), is shot in a vérité style and Darren wanted to have a lot of hand-held stuff throughout,” explained Schrecker. “So the visual effects had to basically be seamless and invisible. Even at the end where the dancer grows wings, it was really important to work that seamlessly into the look of the film. There’s nothing fantastical about it. It’s all very raw and visceral.”
Making wings grow
That scene, in which Nina performs one of the most difficult dances and ends up sprouting black swan wings, was realised as a combination of live action, motion capture and computer graphics. “We spent a lot of time in pre-production on the design of the wings and what the final character would look like,” said Schrecker. “Darren wanted the anatomy of the human to transition into the anatomy of a swan. So he actually ended up buying us a swan skeleton which we digitised and created a 3D mesh of to morph into a human skeleton. It was great to study and do research into how the structure of a wing and a human arm can morph into each other.”
Artists toyed with the idea of giving the character many swan attributes, not only for the wings. “Originally Darren wanted to have a long neck shape but it was always just a little bit goofy-looking,” said Schrecker. “It’s a beautiful moment in the film and in Nina’s head and the wings were such a beautiful vision, and Natalie Portman is really so beautiful, that it didn’t really need much more than that. So we ended up with a wing growth and feathers creeping up her neck only just a little bit.”
New York-based Curious Pictures erected a motion capture set-up on the college location, using 24 Vicon cameras to film a professional dancer carrying out the moves. “It was a little bit of a jerry rig to get a sound stage set up with a motion control rig onto a location,” said Schrecker. “That meant we did have to augment our motion capture data with some hand-tracking, but it was important to start with that basis as far as the actual ballet dance.”
For the wing’s feathers, Look crafted simple models in Maya made up of curved planes for the barbs and cylindrical geometry from extruded curves for the rachises, or shafts. “Each feather has a deformer rig to add bend in two directions and also to allow growth from the rachis outward,” said Look 3D artist Shawn Lipowski. “The body feathers were simplified and usually didn’t contain rigs or separate rachis geometry.”
Scans of actual Tundra swan, or white swan, feathers provided by the U.S. Fish & Wildlife Service Feather Atlas project were used to create feather silhouettes and form a cutout map. The scans were darkened in Photoshop to create a black swan feather. “Normal maps were generated through ShaderMap to add barb roughness,” added Lipowski. “There were ten primaries, eight secondaries, and five generic body feathers in total. Each set was mirrored producing 46 feather textures. The look of the feathers comes from an anisotropic shader, which gives the smooth geometry the sort of directional sheen you’d expect from a real feather composed of thousands of individual barbs. The shader was slightly translucent to allow light through when the wings crossed in front of stage lights. Special care was given to reflection falloff as there is essentially no diffuse lighting on the feathers due to the dark plate and the dark texture maps being multiplied against the diffuse lighting values.”
Look built a rig for the dancer and then matched the wing rig with the same number of joints and similar constraints to follow arm movements and twists. “The joints of the wing rig were skinned to a NURBS foil shape which was more bird-like in proportion than arm-like,” said Lipowski. “The larger feathers, the primaries and secondaries, were hand positioned on this NURBS surface. The smaller feathers which fill in the wings were placed by mel scripts, which I wrote to instance based on texture maps.”
“Additional mel scripts constrained the feathers to the NURBS geometry using Maya’s follicle nodes,” continued Lipowski. “The body feathers were also entirely positioned and scaled based on texture maps and the mel instancing scripts. The total feather count was around 11,000 and I wrote around 1500 lines of mel code for rigging and scene management. Once the wing rig had been completed and attached to the dancer rig, the reference to the dancer in the master file could be swapped with the updated wing rig.”
To grow the rings, Lipowski animated black and white maps in After Effects and generated low-res images sequences. They could then be imported as an expression in Maya to drive things like feather scale and rotation for feather ruffling. “Each feather had extra data associated with it,” said Lipowski, “like UV position, which had been stored previously when rigged by the instancing mel scripts. It was a crude but effective feather system.”
Animating the feathers relied mostly on keyframing, as Lipowski describes: “The primary and secondary feathers were grown by hand by keyframing scale attributes and keyframing the wire deformers of the individual feather rigs which allow growth outwards – the barbs pop out of the rachises. Additionally keyframes were set by hand every couple of frames to keep the wing from twisting, often around the elbows, and to minimize interpenetration. As the shot progressed, the matchmove often had to be tightened up with additional keyframes.”
The scenes were lit with Maya area and spot lights to accommodate the dramatic nature of the shot. “We had enough images to generate HDRIs,” noted Lipowski, “but the lighting changes and large camera movement made it impossible for one or two HDRIs to cover the wild shifts in luminance. Lights were built based on the camera track and plate with numerous stage lights above and to the sides of Natalie, six large chandeliers were approximated, several bright footlights near the orchestra were added and three very bright spot lights casting rim light were critical. Even with all these lights, the black levels in the plate change dramatically due to the smoke in the theater, a volumetric effect lights alone could not approximate.”
“The shot was rendered in mental ray using the Rasterizer because of the huge amount of motion blur,” said Lipowski. “Most of the wing is captured in one big beauty pass with an additional shadow pass and numerous mattes for the compositor.” The final shot also included a head replacement of Natalie Portman and crowd insertion for the auditorium.
Faces, prosthetics, tutus and tattoos
In addition to the key black swan wing shot, Look also contributed some key effects throughout the rest of the film. The opening sequence features Nina dreaming of dancing in Swan Lake, playing out in a black void with no scenery. However, as shot the floor appeared to be significantly scuffed and scratched. “It was made up of really long takes throughout,” said Dan Schrecker, “so we couldn’t really do a full CG replacement of the floor for time and budgetary reasons.
Instead, we came up with an effect that we put over the whole scene that gave a really dreamy look and managed to gloss over the dirt. It also added a layer of caustics, almost, that fed over the Swan Lake idea.” Look Effects enhanced other environments to, such as the multi-mirrored ballet rehearsal spaces that reflected the crew. “It was not very sexy VFX work, but at the same time it let Darren get any camera angle he wanted.”
The director also remained unconstrained by the dance sequences in Black Swan, relying on Look to do some face replacements when complicated moves required professional dancers. “We did 3D scans of Natalie Portman’s and Mila Kunis’ face but we ended up not needing them,” recalled Schrecker. “We had a formula down where we would shoot the dance double how ever Darren wanted to shoot it. Then we would bring Natalie in and have her go through the exact same motion, same lighting, same camera move and shoot it at 48 frames per second to reduce the motion blur. Then it was a matter of doing matchmoving and replacing the head.”
Look also augmented a number of prosthetic effects, creating stylised contact lens looks and animating skin rashes that start appearing on Nina to hint at her eventual transformation. “Renaissance Effects did a great job on all the prosthetic builds and appliances,” noted Schrecker. “We just had to add in animation and tweak the look of them for the horror of the film. Also, by basing a number of these effects off of prosthetics, it let Darren get a lot of things in-camera that he wouldn’t be able to later on in the game.”
In one shot, Nina notices a back rash and picks at her skin as it starts bubbling. “We found that in an early camera test that the prosthetics can serve as their own camera trackers,” said Schrecker, “which is great because you have a piece of skin covered in a bubbly rash that needed to be augmented in 3D. That was a pretty big sequence in getting the skin to bubble and have the feathers poking through – which was a big 3D build.”
A scene in which Nina has blood on her torso also required digital augmentation when production wasn’t able to dress the actor with the stage variety. “At the time Darren wasn’t sure how much blood he wanted to show and how much he could get away with,” said Schrecker. “They also had these very fancy tutus created by well-known designer Rodarte and they only had one at their disposal. So we dressed in a little bit of blood and placed some red tracking markers around the tutu and the corset. Then in post we used some stills to work out the amount of blood, and then animated that to match what Darren was after.”
In a sex scene, a tattoo on Lily’s back appears to a drug-induced Nina to come alive as it morphs from a flower into a set of wings. “We shot the scene with a clean back covered in some tracking markers,” explained Schrecker. “The art department created the initial tattoo of the lily and we hired an illustrator to mimic that style and create the full wings and five or six in-between frames. Then we morphed between those frames to create the animation of the wings growing. The hard part was matching to Mila Kunis’ back and shoulder blades and all that twisting and bending and undulating she was doing.”
As Nina’s life begins to fall apart, she comes across a room of paintings by her mother that are revealed to be childish and disturbing renditions of her. “The original idea,” noted Schrecker, “was that Nina would walk into the room and these paintings would start talking to her. We created a whole series of mouth shapes and eyes that we could slot into these paintings so when we animated them they would start talking.” During the editorial process, Look was called upon to further enhance the paintings, with faces melting and wings sprouting out of dancers, created by re-animating the original artwork.