Brainstorming more VFX on Boardwalk Empire

For its second season, HBO’s Boardwalk Empire once again embraced invisible visual effects work to re-create the show’s Atlantic City setting during Prohibition. We talk to Brainstorm Digital co-founders Richard Friedlander and Glenn Allen about Brainstorm’s VES Awards-nominated work on this series.

Above: watch a breakdown of the Boardwalk Empire season two visual effects by Brainstorm Digital.

fxg: The second series is in some ways more grand in scale – was that the same for the visual effects work you were required to create?

Richard Friedlander: One of the concepts of this season was that whenever they were on the boardwalk they wanted it to show a little bit of growth and change, because as a vacation spot Atlantic City was always expanding and growing. So we would change the advertisements on the billboards, for example. The CG face work for Richard also carried over, but we decided to approach it in a more sophisticated way, and then there are more shots of ships and expansive city work.

fxg: Did production change the approach to shooting on the bluescreen backlot?

Glenn Allen: Well, they did re-paint the bluescreen to keep it bright for us and we improved our approach to rotoscoping and pushing through the shots.

Friedlander: We learned a lot from the first season and we just improved our methodology and approach to some of the shots. We also had more complicated and hero shots this time around, but a slightly lower volume, which is actually how we prefer it.

Original plate.
Final shot.

fxg: Can you give an overview of the work for the boardwalk in particular?

Friedlander: We had two sequences on the boardwalk, which was utitlized less in this season. That is all fabricated. We did boardwalk extensions, store extensions, CG stores, and then would put in 2D plates of the beach and ocean, along with 2.5D matte paintings on cards for piers to get parallax. In the inland direction we used a combination of 3D projections on geometry. Then there were actual 3D buildings where we got close or had big camera moves. There was a large sequence for a riot in episode 10 where they’re protesting on the boardwalk. There were a lot of Technocrane moves with cameras overhead, craning up, which really helped the dynamics of it.

fxg: Can you talk about the CG face shots for the character Richard this time around?

Friedlander: One of the main ones was a big chase through the woods where Richard runs through there and is going to take his own life, and then gets interrupted by a dog who steals his prosthetic mask. That was about 30 CG face shots. He was doing a lot more actions for these than in the first season.

Allen: He speaks and breathes heavily and puts the gun in his mouth, so it was much more complex animation than before.

Friedlander: So we re-sculpted and re-rigged the model. The lighting situation in the woods changed so much as he was running, too, and it was also shot over several days. Our CG crew did such a great job and so did the compositors.

fxg: What sort of approach did you take to filming that?

Allen: We had tracking markers on the face to help with matchmoving.

Friedlander: There was a facial mask with holes drilled in it, so the mask molded to his face each time to make sure the markers were accurate each time. They guided us in terms of facial movement and the tracking. In season one we came up with a look for the wounds by taking a photo of the actor’s face and looking at reference of soldiers from the day. The model was then sculpted in Mudbox, and then brought into Maya and Mari for the 3D work and textures and lighting. It was rendered out in separate layers and was composited in Nuke.

CG layers for the facial wound work.

fxg: How did you deal with blending the CG to the real skin?

Friedlander: It was very shot specific and depended on the actor’s mouth position. One of the methodologies he had for talking in that way was to put cotton in his mouth, which actually reversed the position he wanted his mouth opening to be. So quite often we had to take over the whole mouth and teeth in CG. And this year we’re thinking of given him some kind of dental prosthetic in his mouth, unless it interferes with his performance. But it was really to the credit of our matchmovers and compositors to make it all work.

fxg: For the ship shots, what approach did you take to shooting and inserting the digital models?

Friedlander: For the ships we had the piers in New York and then a port in Ireland, which was shot here away from water. For the Ireland plate, we had some moving ocean plates that had been shot for us. It was shot along the water and we filled it in with some existing water plates. We did create some CG water interaction for background boats.

Friedlander: We did a very rough animatic to show them what portion of the ship they would see. We gave them exact distances to landmarks in the area as well. It was actually tricky getting the right scale ship in there to make sure it was proportioned properly.

Original plate with wireframe ship.
Final shot.

fxg: What was the approach to modeling the ship?

Allen: Again it was a real credit to the artists who did the research into how the ship should look for that period.

Friedlander: The ship had to feel really worn and torn. The construction techniques in those days were not as clean as now, so we worked really hard to create bump maps for the steel plates and seams to show that things didn’t quite match and that it was imperfect. There was also a lot of dust at the locations so we had to re-create that, which is what we did for a lot of other shots in the show and make it look as ugly as possible. That’s what we say to everybody – make it look ugly!

fxg: What were the extra details you were adding in for these shots?

Friedlander: We added in extra workers and sailors and smoke and background action just to give the plate movement.

Allen: Anything that gives you movement means that your eye doesn’t focus on the matte painting itself and you can just believe that it’s there.

Friedlander: We also staged on a separate day a bluescreen shoot of the various actors on the ship. We also had experience from working on the movie Julie & Julia, where we had to re-create the Ground Zero World Trade Tower clean-up pit. When they filmed it the construction was quite advanced so we had to take it back. It was all replaced with matte paintings and construction gear, but at one point it just didn’t feel quite right, so we actually put in just one person moving out in the middle and it really sold the whole shot. We created a hard hat on them and they’re really tiny on the site but it worked – so we used that kind of approach for Boardwalk.

fxg: What were some of the other shots you worked on?

Friedlander: We had to create some flashbacks for Princeton University with the Army corps marching out on the field. It was important to the plot there was a train going by closely. Another flashback involved some WWI shots.

Allen: HBO supplied us with a lot of practical elements they had filmed for The Pacific – smoke and debris and explosions which had been shot in Australia.

Friedlander: We went through their library and selected the appropriate elements. We knew they were going to shoot slow-motion which matched the elements. The scene was then shot in a parking lot in Brooklyn and the art department did a great job making it look like a trench. What made it tricky was that it was a long shot – a couple of thousand frames – so the tracking and roto and mattes and clean-up was complicated.

Original plate.
Final shot of car entering armory.

fxg: There are some great nighttime shots in the series too – can you talk about working on these compared to the daytime ones, especially some of the views of the Ritz Carlton?

Friedlander: I always find the nighttime shots really effective. One of the things production always pushed for us to get right was the lighting on the signage. The light bulb was fairly new at that time and it was being used to light up all the billboards and attractions – it was a new novelty. It was pretty to have the colorful bright blinking lights against the night sky, and we tried to get as much haze in there to get that ocean mist feel.

fxg: There’s a great scene on the breakdown reel of a car driving into a building entrance which has been replaced with an armory building. How was that achieved?

Friedlander: The building was supposed to be Atlantic City Armory. They couldn’t find a location that they wanted, but they found a great-looking entrance. It was actually filmed at an old fortress that had been built in the mid-1800s. They were going to use the bottom and then we would tie into that with our work, but then they found a building for reference and we used stills to stitch together a CG replacement – they liked that so much that we replaced the whole thing, even the ground plane. So the only thing left in the shot is the actual Rolls-Royce vehicle. The toughest part was roto’ing the car and re-creating the shadows on the ground.