Well that’s one way to solve it: The Blacklist

 

Due to the coronavirus production shutdown in NYC, the NBC’s show The Blacklist stopped shooting in the middle of the season seven finale episode. But instead of just holding onto the footage with an extended hiatus, the show creators decided to turn to a previs house, Proof, to get high-quality animation to finish the episode in record time.

Proof created a custom graphic-novel style in reference to The Blacklist comic books.  Artists in the London and Atlanta studios worked at a rapid pace to complete the show for the May 15th airdate. This was Proof’s first time supplying final animation for a television series, but they have been providing high-end previs to tv, film, and digital media for almost two decades.

The project was supervised by Adam Coglan and Matt Perrin, two experienced senior Proof Previs supervisors. The project came about from a phone call with Proof founder Ron Frankel. Proof was founded in 2002 by Ron Frankel, and it is dedicated to providing high-quality visualization services for the feature film and immersive entertainment industries. As part of that brief, it had developed its own in-house toon shader solution. When the lockdown happened The Blacklist producers asked Frankel if Proof could take this toon-tool, which had only been used for Previs, and use it for final production VFX shots?

The Blacklist had already spawned an actual spin-off graphic novel. As the producers had only half of the final episode, and the producers had seen examples of the Proof toon shader, they proposed completing the rest of the final episode as animation partly based on the look of graphic novels and partly based on the blocking and framing of normal live-action episodes. Frankel jumped at the chance and then phoned Coglan and Perrin, to discuss them sequence supervising the rest of the episode.

The Blacklist episode required non-stop coordination with the remote artists and it caused Proof to reinvent their workflow. Unlike previs, which is seen only by a handful of people in production, The Blacklist work showcased Proof’s work directly to the public on a massive scale.

The team was provided with temp voice audio to animate to, which had been recorded by the editor and one of the show’s producers. This temp audio allowed the team to start animating and then as the episode progressed, the team started getting more and more final audio. The last production audio was only provided in the week before the show was to air. From Proof’s point of view, The Blacklist was a dream client. Given the uncertainty and variability, a less confident and trusting client could have derailed the whole project with revisions due to the lack of a clear vision, but both Coglan and Perrin agreed that the client was outstanding. As the show has already aired over 150 episodes, the editorial and production team sourced reference clips and examples of the show’s cinematography to guide the animators. For example, the show rarely has a still camera, it is almost always filmed as a dolly or tracking shot. Proof was keen to respect all of these stylistic conventions of the show, while also being able to mix in shots that owed a heritage to the dramatic shots of a graphic novel. Thus the final look of the show is very much that of The Blacklist, but as an animated graphic novel.

The final episode is split 50:50 with half the show or some 35 scenes animated. All the show’s characters were modeled in Zbrush and then imported into Autodesk Maya. The proprietory Toon shader was applied. There was also Photoshop to define and refine the textures and add little bits of detail. Some of the main characters actually had hand-drawn detail on their faces, with things such as pencil strokes added. The rendering was just done in Maya.

Lipsync was an issue from the start, the team had wanted to avoid lip-sync, as they knew the team would not get final audio until the end of production, and the models were never designed as fully rigged facial puppets. That being said, some of the animators couldn’t resist adding some lip-sync. “And some of that crept out and the showrunner got a taste for it and said, ‘look, is there anything, anything we can do to get in?’ ” comments Perrin. So Coglan’s team, in particular, set to work trying to kind of find out what Proof could plausibly get done, inside the constraints of their models and the schedule. “They were quite simple previs models. They were very limited, almost a marionette style” Perrin explains. But the animators were so dedicated that some impressive animation was done,  and Coglan recalls “I had to tell everybody just to switch off the lips and say that we can’t show any of this because the production is not really expecting lipsync. We didn’t think we had time, and we were worried that as soon as everyone saw it, that’s what they’ll want”. But as it turned out, in many of the scenes, the animators had bothered to do lipsync anyway, and so in the end, both Coglan and Perrin decide to push for including it.  “Knowing that for many shots it became just a matter of turning the lips back on,” says Coglan. “And we thought it did look great, so yeah, in the end, we pushed for the lipsync and it became a reality,”

Luckily one of the Proof team had been doing some research on an automated approach to the lipsync and basic facial capture using a depth-sensing camera. “We use that to just to supplement the volume of lipsync we had to do and that worked out quite well”, recalls Perrin. As it was such early days on the automated depth camera tests, it was just used as a good base for timing. The automated solution allowed the animators to have a guide for their detailed keyframe animation.  “It was definitely a little bit of a shortcut, – a form of semi-automated lipsync for the rest of the shots,” explained Coglan.

The whole exercise was quite a departure for the editorial department. On the show normally, the production has been known to shoot with six cameras, providing a lot of footage for the editors to craft scenes within. With the animated graphic novel footage, there was no extra footage or live-action style shooting ratio with loads of angles to edit from, the opposite was the case. Proof provided just the main action and then waited for any requests for additional coverage, cutaways, or material from the editors. This translated to 1300 versions of 742 actual shots, which were produced by a team of 29 animators & staff in the UK with an additional 14 crew also working in Atlanta. Luckily, the Proof team had already moved into lockout and everyone had moved to their home offices about a week before The Blacklist project happened. The project proved to be both a technical and creative success.