American Pickle Making

In Seth Rogen’s, An American Pickle, immigrant worker, Hershel Greenbaum, emigrates to America in 1919, in search of a better life.  However, the unfortunate Herschel falls into a vat of pickles in the factory he’s working at killing rats and wakes up 100 years later in 2020.  Herschel then discovers he has one living relative, his great-great-grandson, Ben, also played by Seth Rogen.  The film is directed by Brandon Trost in his feature debut, having worked with Rogen on previous projects as DOP. Nviz provided the visual effects led by the film’s VFX Supervisor, Adam Rowland. Nviz’s own VFX Supervisor was Jason Evans.

All images: Courtesy of HBO Max

An American Pickle presented a set of challenges to the VFX team as the two main characters were played by the same actor. Nviz was an obvious choice to help solve these problems, following their experience of split-screens and face-replacements on the film Legend; where Tom Hardy played the Kray twins.  But An American Pickle added an additional complication, in that one of Rogen’s characters, the accident-prone Herschel, was sporting a rather spectacular beard, while his grandson Ben was clean-shaven.  This meant that the production had to shoot half of the film first and then come back 6 weeks later to shoot the other half.

In order to facilitate this, the split screens had to be very carefully planned to allow the team to shoot the second half under exactly the same conditions, or as near as possible.  Filming took place in Pittsburgh over a 3-month period late in 2018.  During that time the weather moved from warm and sunny, to cold and snowing. Despite this, the production team was meticulous about securing the same location, at the same time of day and with the same conditions. Says Adam Rowland, “I don’t know of any films where this kind of thing has been done on location. Most of the time, if you are doing split screens, you do it on set, in a controlled environment where you can leave the camera in the same position so you can maintain conditions. So that was fairly unique to this production and a big challenge in itself.”

Not all the split screens were shot on location, the majority were captured onset; and though this was easier to deal with, the team still had to allow for shooting the second half of the split screens 6 weeks later. To aid them with this, they used ultraviolet pens to mark the floor of the set; allowing them to see quickly the position the camera needed to be in for each set-up, what lens was on, what shot was being filmed, and so on.  “The ultraviolet pen meant we were allowed to annotate quite freely all over the floorboards of the set”, says Rowland, “and then when we came back we just had to shine a light on it and we could work out where the camera was for that shot.  Otherwise, we would have been constantly moving markers around – or creating many more VFX shots where we’d have to paint them out.”

One of the most difficult split screens to realize was a long shot of Hershel and Ben chatting and walking together towards the camera, which is tracking backwards on a Steadicam.  A lot of consideration went into how to achieve this successfully, as the constantly moving camera and the backgrounds changing in different ways, meant it would be impossible to complete as a traditional split.  The team shot a reference version with Rogen and his double and another plate without the characters in it.  They then worked out how fast the characters were walking in order to keep up with the camera.  Using this data, they went on to shoot the two separate characters against greenscreen and walking on treadmills that were adjusted for them to be walking at the right pace.

Other examples of innovative problem solving was the shots of the two characters walking side by side on a busy New York street.  In order to seem authentic, the bustle had to interact with Rogen’s characters and people needed to walk across their paths.  This had to be carefully choreographed in a version where all the crowd and bustle was shot without the main characters.  Extras were choreographed to walk around and through the Rogen characters’ paths, but only at times when they couldn’t interact with them.  It was an example of a “simple gag” that required a great deal of meticulous, logistical planning, including weather and lighting variations, to pull off correctly, while also taking into account the 6 week time difference between the shooting schedule for the two different characters.

When it came to the full face replacements digital tracked onto another actor, these were used sparingly and only when there was either extensive interaction between Ben and Herschel or there was a complex camera move. In these situations, The production would shoot the main plate as normal with markers on the face of the stand-in. Once the main plate was shot, the editorial team would pick their preferred takes and then prep the footage for a separate face replacement shoot. “This prep work mostly meant stabilizing the shot around the head, both in translation and rotation, with a quick and dirty 2d approach,” explains Evans. “Once we had the locked-off action we could then get Seth to recreate that motion sitting on a stool in front of a greenscreen under a closely matched lighting rig”. The team took multiple takes of each setup and shot at 90fps to ensure that they had something they could use in post-production. They would retime any movements which didn’t quite align perfectly. “In post, the simplest solution to integrate the new face was normally to do a full head replacement using the collar as the join. This worked well on ‘over the shoulder’ shots but on a number of the shots we had to hold on to at least part of the head from the stand-in to make it feel natural” he adds. In this case, if the replacement element lined up well then the team could use the markers placed on the stand-in’s face to track and warp in 2D the replacement element. They would then composite just the main facial features and keeping the neck and hairline from the plate. Where this didn’t work a more complex solution was required. “On a couple of shots, the elements didn’t align very well due to editorial changes or to the element being slightly off in angle or height. Here we projected the texture from the greenscreen element onto a 3D scan of Seth and tracked that model into the plate to align with the action of the stand-in” outlines Evans. Any missing texture was then cleaned up using paint or reference photography. According to Evans this technique gave Nviz the ability to subtly adjust the angle of Seth’s head in the greenscreen element without losing the “lighting, texture or any of the micro-movements in the musculature of Seth’s face and kept us away from that pesky uncanny valley!” 

Note: the two stand-in actors in each of the  two plate shots, with tracking markers on their faces.

Although Nviz worked on many complicated split screens, the bulk of the VFX the team created for An American Pickle centered around transforming modern-day Pittsburgh into early 20th-century Schlupsk (the fictional Eastern European town Herschel came from) or into modern-day New York.  This involved a multitude of matte paintings and environment enhancements.

Source Photography: Pittsburgh for Eastern Europe ‘Schlupsk’.
Final : Herschel and Sarah Greenbaum emigrate to America in 1919 after their village is rampaged by Russian Cossacks

In addition to the split screens and the environment work, Nviz also created the rats that live in the pickle factory. The team was given reference from the director, including cartoons that the director wanted to reference. The aesthetic for the film, particularly the 1919 period is a kind of heightened realism, and the rats are no exception, having a slightly cartoonish quality. The final result is almost photo-realistic caricature versions of rats, exaggeratedly mangy and horrible, and occasionally “scarier” but certainly, “funnier” than your real New York rats! Rowland comments “It was great that they were stylized, and we were really able to lean into it and have some fun.”

The rats were developed in Nviz by CG Supervisor Sam Churchill and Zach Du Toit. From a distance, they look just like real rats, but up close they often look “ridiculous” and have more unique characteristics, such as particularly mangy hair, red or cataract style eyes, and idiosyncratic movements. The director liked the idea that the rats were a gang and like all gangs had a leader, who is even more disgusting than the others!

For the few shots that showed over 60 rats, the team used a crowd simulation tool that could be adjusted with additional keyframe animation. For the majority of shots that included no more than 10 rats, it was hard to justify doing anything other than keyframe animation as the crowd simulation tools are essentially designed for use on bipeds. Says Churchill, “If you get too close with crowd simulations the holes start to appear, so the close-up shots always had to be keyframe.” Churchill created a rudimentary crowd simulation layout pass in Miarmy to establish volume, general position, and motion. Once these points were approved by the director, Churchill wrote a script that converted the Miarmy sims into individual rigs, and Head Animator, Andy Frazer, took over from there.


After Frazer took over the animation, it was simply a case of putting in the time in with traditional key frame animation. This was the really exciting phase for the team, says Evans, “The sequence actually came alive very quickly, we went from the blocking stage to a fun, exciting stage in a short amount of time. Once the animation started to progress Brandon (Trost) realized the rats needed a malevolent pack leader and began to add idiosyncratic touches.” Using the James Herbert book “Rats” as the main point of reference, the team began to add distinctive quirks to the rat pack leader, giving him a glassy, milky eye, scars, bald patches, and a missing ear. The team used Xgen in Maya for the fur and built a system that allowed variation for the different rats.

“It was a great experience working with Brandon.” says Evans, “His notes were brilliant all the way through. He had a very strong idea of what he wanted the feel of the rats to be and was really consistent in the way he communicated. He gave us a lot of creative freedom in animation and design, gave the direction of where he wanted it to be, but the freedom for us to decide how to get it there.”

One of the most challenging and creative shots of the film is a charming “time-lapse” of the outside of the factory building within which Hershel slowly pickles over a hundred years. The shot is used for the opening credits, bridging the two time periods in the story.  Multiple variations of the same NYC skyline matte painting were created to depict it changing throughout the 100 years.

Source Photography

Final shots showing the changes over time

Initially, the idea was simply that it would be a time-lapse from 1919 to 2019. The audience sees the factory boarded up and slowly dilapidating over time, while Manhattan gradually rises up in the background, trees growing and skies changing.  It was decided to keep the time-lapse as daylight in order to maintain gentler transitions and focus on the changing weather and seasons.  The early period section of the film has been largely shot on antique lenses, which has a specific aged vignetting quality – with the rest of the contemporary section shot on modern primes.  It was decided therefore that not only should the shot fast forward through the seasons and the century, but also through the age of cinema.  Over the course of the shot, the aspect ratio changes from 4:3 to 1:1.85, and the grade shifts from almost monochromatic in the 20s, through bright technicolor in the 50s and 60s and culminates with a very sharp digital look by the time it arrives in the 21st century. “Once we’d worked out the ‘rules’ of the shot it just worked; the parameters we set, resulted in an aesthetically more pleasing result.”, recalls Rowland.

 

 

Thanks to Aisling Newton for contributing writing for this story.