We talk to Brainstorm Digital visual effects producer Richard Friedlander about his studio’s work for HBO series Boardwalk Empire. This week Richard received, along with Robert Stromberg, Dave Taritero and Paul Graff, a VES Award for Outstanding Supporting Visual Effects in a Broadcast Program for the show.
Watch the video for Brainstorm Digital’s Boardwalk Empire VFX breakdowns
fxg: The boardwalk sequences are certainly key shots in the series. What kind of planning and preparation was involved for these scenes?
Friedlander: We actually started many, many months in advance. The producers knew that nothing from this world 100 years ago really existed today. Atlantic City has become a very modern resort area that has nothing left from the era that the Boardwalk Empire series takes place in. So we worked very early on to help the production with their physical set construction in terms of what would be needed and how much of a physical set they should have. Then we did a previs with physical models and archival photographs and also temp matte paintings to give production an idea of what could be done.
The questions we helped them with were: Do we really need to go to the ocean to shoot it? Do we need to go to Atlantic City? Do we go some place on the east coast of the United States? Eventually we showed them that they didn’t need to go to the coast – it would probably have been cost prohibitive to film outside the New York metropolitan area in terms of moving hundreds of crew and mounting that kind of operation. We helped them devise a way to shoot on a partial set with bluescreen and then we would add in the beach and ocean plates.
fxg: What kind of set did production end up constructing?
Friedlander: It ended up being one of the largest outdoor sets ever built on the east coast of the U.S., with a boardwalk about 225 feet long shot at Greenpoint, Brooklyn. Still, we enabled them to build something much smaller than they thought they would have to. They were going to put up giant billboards over the boardwalk but we actually said it would be better for us to not have anything up there, and that saved us a lot of roto work. The set had to be constructed with steel substructure – it was outdoors and had to sustain hurricanes and blizzards, on and off. After a couple of months of scouting, they found a piece of land that was pretty close to the studio where the interiors were shot.
fxg: And then how did your planning help with the actual shooting?
Friedlander: We took the land area in Google Maps and we actually prevised for them not just the visual effects, but also the set, how much space it would take up, and where to put a cafeteria, say and all the places for fabrication and other things. Our big concern, though, was the bluescreen. On previous productions I think they had used a steel structure with fabric over the steel on an exterior location. Our advice to them was – if this whole set was going to be built to last, the fabric would probably not withstand the sun and wind and weather, so we thought it was better to have something rigid. Our idea was also a big cost saving. If they had to create a big erect-a-set construction to withstand the weather conditions and to last for a long time, it would usually be very elaborate, but we came up with the idea of using shipping containers and stacking them like Lego sets.
So they spray painted the containers blue and stacked them up, and it ended up that they were very inexpensive and we were able to get a much larger screen than they would have got if they had to construct it. Because they are corrugated, we had the production cover them with stainless steel which was a very quick and easy way to fabricate. They just used these guns that riveted the stainless steel to the containers, then spray painted them blue. They’d periodically paint them again to keep the colour bright. We also built a curtain system of black screens, if there was a shot where we didn’t want bluescreen. So we could close the black curtain and do it that way. The containers also provided a great storage area for a lot of equipment. It worked out really well.
fxg: What do you think were the advantages in shooting outdoors with the bluescreen, and what were some of the challenges of doing that?
Friedlander: We were very much torn in that regard because we knew the interior set would give us much more control over the lighting and bluescreen and we would be able to do much easier keying extraction for the composites, because you can control the sun and the seasonal changes and shadows. But we felt like it would have a much more real feel if we filmed it in natural light, and get all the defects that come into play with clouds and weather. If it was raining on a day and they were scheduled to shoot, they just shot in the rain.
Unfortunately when you’re outside and you can’t control the sun, the shadows and the bright sun would quite often make the bluescreen less than ideal. But that’s what happens when you go outside.
fxg: How was Brainstorm’s previs used on the set?
Friedlander: We had a full 3D grayshaded model of the design of the set extensions. We’d be there on set with the laptop and show them what they would be seeing, so they could line up the shots and compose it appropriately for a hotel building or a tower. We actually previs’d some shots and camera moves to see what the camera would be doing – we actually animated the camera to show where it needed to be. That would help them know how big to build any platforms and get the appropriate crane for the move.
fxg: What was involved in then taking the live action and extending the sets?
Friedlander: Most of the shots were a combination of everything from a visual effects point of view. We had matte paintings, projections onto geometry and then full-blown 3D where we texture mapped the geometry. We worked really closely on the whole colour palette with Robert Stromberg, who is not only a visual effects person but also a production designer. He’s worked on Avatar and Alice in Wonderland.
Sometimes Robert would do a matte painting and that would actually be used if the 2D worked. Sometimes we would project the matte painting onto 3D geometry and with a camera move get the parallax in there by cutting up the matte painting. The bigger, hero shots have a combination of everything going on. Deep backgrounds are matte paintings, mid to near-ground are CG or 3D geometry. Our standard tools were Nuke for compositing, 3D modeling in Maya and matte painting in Photoshop. Another person heavily involved was Dave Taritero. David had worked for many years at Imageworks as a visual effects producer and for HBO he oversaw The Pacific and then Boardwalk Empire.
fxg: How did you go about matching the colour palette established in the live action photography?
Friedlander: The initial palette was established by the live action set. One of the big challenges for the references for the era was that all we had were black and white photographs. There were no colour images, or we couldn’t find any from that age. Everything was hand tinted colour. The production designer had researched it very carefully, including speaking to people who had been around during that era. Basically, we had to make sure there was enough degrading of the structures and the images to make it feel like it had enough wear and tear from the seaside atmosphere. It was a big challenge because you want the viewer to feel it and sense it. It’s a very delicate balance because if you have too much, everything would feel too painterly. You can’t mute the colours too much.
We also ended up doing full sky replacements on everything. We found that the ocean and beach plates never quite captured the actual lighting that would tie in correctly with the principal photography, so it made a lot of sense to completely art direct the skies and the lighting of the skies and clouds. We brought in a matte painter and she art directed the clouds and skies to tie into the cinematographer’s lighting, which was made up of a lot of hard edges and back-lights.
fxg: What kind of other environment work was involved in the series?
Friedlander: For some of other locales, we referenced some great Brooklyn architecture which fits the period and also the area of Atlantic City. They shot there too, but of course you only have small sections of it. We quite often would replace the environment and keep the usable section. That was done for the seaside bungalows in the pilot and in other sequences. So it wasn’t just about the Boardwalk. We’re into other exterior locations and the Times Square sequence, which was fully CG.
fxg: One of the pieces of non-environmental work includes the war injury shots. Can you talk about those?
Friedlander: This was interesting because it was a bit of a pet project of the series creator, Terry Winter. One of the very important things for him to get across this period and for the character Richard, that he was very much affected by the war and they wanted to show how physically mutilated people were. And at the time they didn’t have the medical advances they have now – there was no plastic surgery then. If someone survived and they didn’t get a horrible infection, they were basically pieced together by doctors as best they could. There was an actual woman who used to make these masks for soldiers who were horribly disfigured from the war. So it was an important plot point for him to have a disfigurement that needed to be augmented in CG.
“But once you lift the mask off, it’s all CG.”In season one, he’s seen with the mask, which I personally found very disturbing. We did a lot of research about real soldiers. We took bits and pieces of injuries that we saw and put it together into a hybrid of what we thought worked for this character and the mask. For showing the injury beyond the mask, there were some physical latex appliances that we tied into the CG. But once you lift the mask off, it’s all CG. We worked for a very long time on the concept art until we got the right texture. We had versions where the wound was much earlier on, and then other versions where it had healed, but then we got to the level where it had healed a little but was still obviously noticeable.
fxg: What techniques did you use to matchmove the injury to Richard’s head?
Friedlander: We created a full model of the actor’s head and our animator and rigger worked very hard to animate that. The challenge there was that the production felt very strongly that they didn’t want his face to fully function in a normal manner. The sub-structure, the bones and the muscles had to be injured too. We had to make it look like it functioned realistically and believably, but we didn’t want it to look like bad animation. We wanted to make it feel it was completely believable but also portrayed a real sense of the injury. We used some proprietary tools for the matchmoving. We did the concept work in Photoshop, painting over the concept stills from references we took of the actor’s face. The actual 3D model work was done in Maya, but we used a different paint tool to model the injury and scarring. This was then comped in Nuke.
Images and video courtesy of Home Box Office, Inc.