MPC Replicating Rachael in Blade Runner 2049 (updated)

Replicating Rachael for Blade Runner 2049 was the technical and creative challenge given to MPC in the new Blade Runner 2049.

The team lead by VFX Supervisor Richard Clegg worked closely with Director Denis Villeneuve and Production VFX Supervisor John Nelson.

One of the major challenges was how to approach the CG sculpt of Rachael’s head. Blade Runner was filmed more than 30 years ago and so Sean Young, the actress who played Rachael in the original movie, was obviously younger at the time with different features.

 

---

---

According to EW : John Nelson worked on Rachael’s two-minute appearance in 2049 for a year. “Digital humans are sort of like the holy grail — they’re really hard,” says Nelson. “I knew it would be one of the hardest things I’ve done in my career. We had many challenges in this movie but this one was definitely the hardest one we did.”

Since her son was on the film as a production assistant, “Sean Young got to be on set while actress Loren Peta acted out the performance that would be the foundation for the effects work. And later, both Peta and Young spent a day in Budapest in facial motion-capture rigs, with both women performing Rachael’s scene in the film”, according to reporting on io9.

MPC’s team started off with a detailed present-day scan of Sean Young’s head, captured on a Light Stage. This scan was used as a reference for MPC’s artists to hand model an anatomically accurate 3D skull. The skull is something that actually changes very little over time, so it is a good foundation for the MPC team to build their 20-year-old digital Rachael from.The Light Stage scans of Sean Young and Loren Peta, including facial microgeometry were done at the USC ICT lab Light Stage. MPC received an ICT Light Stage head scan and photographic data, they used this to model the skull.

Body double Loren Peta played the young Rachael (in costume, makeup, and with dotted face) and performed on set with Ford and Jared Leto. She was directed by Denis Villeneuve, with Young also on-set for reference. Roger Deakins provided the stunning cinematography for the scene.  On a ‘secret Sunday shoot’ in Budapest, Sean and Loren were brought in to be captured using the DI4D PRO Facial Capture System. Rachael’s presence in the film was a closely guarded secret. During the shoot in Budapest MPC captured videogrammetry of the actor’s performance using Dimensional Imaging’s DI4D capture rig. MPC captured Sean Young and the body double reenacted the performance for every shot. At the same time, MPC also took their FACS capture kit to capture an array of facial poses and expressions. Without the use of any markers, or makeup, DI4D tracked Loren’s facial performance using a 7,400 vert mesh, the densest mesh DI4D have tracked to date. DI4D subsequently increased the resolution of the tracked data to approximately 30,000 verts, resulting in the delivery of highly realistic facial animation for the scene.

 

The digital sculpt gave the team a clear idea of the proportions of her head including the bridge of nose, cheekbones and jaw line. With the CG skull accurately recreated, it was lined it up against scenes from the original 1982 movie.

The available footage of Rachael wasn’t always ideal, due to the dark and contrasty lighting with a shallow depth of field that regularly put her in soft focus so a great deal of guess work was required. MPC’s 3D modeling artists then spent many hours sculpting the rest of the head over these images until they had created an identical match.

With the head sculpt completed the character’s hair and colour textures were worked on. The hair was styled using MPC’s proprietary groom software Furtility, built by MPC’s R&D team. The Furtility tool was used to match Rachael’s hair, eyebrows and eyelashes from her opening scene in Blade Runner.

To test they had accomplished a perfect likeness, MPC decided to recreate 3 shots from the original movie using their digital character. The CG head was then composited and reanimated over the top of the real footage.

“We invited the filmmakers to see if they were able identify the digital double from the real actress” says VFX Supervisor Richard Clegg. “We knew we were on the right track as this proved difficult to do!”  On set when a body double acted out the performance with Harrison Ford, MPC had multiple witness cameras set up to shoot each take and used to get precise input for the animation team of the body double as well as to track the complex animated stage lighting.

---

Niander Wallace’s room set had 256 ARRI 300-watt Fresnels in two concentric circles. The doors were removed so that each lamphead was literally touching the next. When a slow chase was programmed, it appeared as a soft source of about eight lamps circling the subject. Deakins stunning lighting and cinematography did not lend itself to a simple HDR IBL solution for the lighting MPC needed to replicate, but the effect in the final film is remarkable.

During the shoot MPC’s onset team also captured videogrammetry of the performance using Dimensional Imaging’s DI4D capture rig. They captured Sean Young and the body double, who both, directed by Denis, reenacted the performance for every shot. At the same time, they also took MPC’s FACS capture kit to capture an array of facial poses and expressions.

As reported to Indiewire,  “About four weeks before the end of production, they decided to put in footage of Rachael from the original ‘Blade Runner,’ which set the bar really high,” said Nelson. “What we really did was drill in all of Sean’s mannerisms from the first movie and were all over them. For instance, when Sean has a wry smile and wiggles her head a bit.” Villeneuve wanted a three-beat character arc for the new replicant when she encounters Deckard. First, she displays confidence and then longing before feeling rejected when realizing that she doesn’t measure up.

When it came to animating the shots, the videogrammetry data served as a valuable reference, however the character’s entire performance was hand animated by MPC’s artists. This animation technique gave the animators more flexibility and the director more control. “We captured both actresses doing the scene in a controlled facial capture environment, but, in the end, because we wanted to tell this story with subtle changes, we ended up hand-animating the whole thing,” stated MPC visual effects supervisor Richard Clegg.

Nelson also commented to Indiewire that “In each shot, we replaced the original Rachael and it was hard for Denis and the producers to identify the CG model. We all agreed that the model fit, but we still needed to get the emotions right.”So they kept going back to Young’s original performance and mimicked it until the animation team made it happen and puppeteered it. “The giant ray of light on the ceiling that was constantly moving was difficult to capture, said Clegg. “But I think it helped as well because as the light moved and traveled across her face, you really get a sense of the emotion that is happening.”

Hundreds of facial shapes were built all modeled against footage of Sean Young to maintain the likeness. Every detail was important to the animators, from the shapes that her mouth makes when she talks to the number of folds in her eyelids.

MPC has also revealed it has a secret R&D lab, named ‘Shadow Lab’ at their studio, developing a number of different new technologies including digital humans.