Filming stadium sequences that require crowds of spectators can, of course, be expensive and logistically challenging exercises. In recent years, visual effects solutions have been refined for seamless integration of digital crowds for major features, TV shows and commercials. We take a look at two spots with VFX by a52 and Framestore that recently caught our eye for their innovative crowd replication work.
a52’s CG spectators
To help launch new network FOX Sports 1, director Joseph Kahn created ‘Happy Days’ – a journey across football, NASCAR, baseball and other sports. Stadium and arena crowds were generated by a52 using a custom Massive pipeline.
During filming, a52 carried out some extensive on-set data acquisition – from locations in five cities shot over 10 days. “We took multiple HDR’s at each location, laser distance and survey information to help with tracking and geometry placement in 3D, and plotted camera positions on field to match angles for stadium replacements,” says CG supervisor Kirk Shintani. “We took reference photos of everything as well. All of this was sent back to the office every day so the team was able to keep moving forward while the shoot was ongoing.”
The crowd generation began with considering reference from existing footage. “We watched several clips of Raven’s games, USC football, NASCAR races and all the events covered in this spot,” recalls TD Chris Janney. “Kirk and I spent hours going over reference photos of fans, making a list of all sorts of apparel and props fans brought to the games. You’ll see everything from checkered flags in the NASCAR shots, to foam fingers in NFL and baseball. There are over 1,500 textures for hats, hair, parkas, ponchos, sunglasses, shorts, pants, tank tops, and maybe a beer helmet or two. We looked for what made these events fun for fans and tried to cover all the bases…pun intended.”
a52 set up a Massive – Maya – V-Ray pipeline that took the tracked plates, placed planes where seating existed and then curves with points for every seat. “This was exported to Massive as terrain (the planes) and placement (the curves) with the number of agents per curve based on the number of points on that curve,” says Janney. “The agent assignment and weighting was also done in this process, so we were writing out a Massive .mas file from Maya with agent groups, placement and percentages.”
“Now in Massive,” continues Janney, “we would check for bad locators, (fallen off the terrain, overlapping, reversed orientation, etc) as well as adding some noise and randomness to their location. Finally, we would create our sim layers, and command-line sim all the crowds, whose final outcome was a VRay .vrscene file.”
For lighting, artists created a tool enabling a lighter to browse to the latest version of the Massive .vrscene file. “Based on frame offset, etc. we would be rendering out the Massive crowds from Maya, then submitting them to Deadline for rendering on our render farm,” says Janney. “We created some great tools which really cut down on not only the time to move shots down the line, but really reducing the time needed to train people on using these tools.”
Shintani suggests that what made the crowds convincing was “the attention to detail in creating textures and props for them as well as tying the reaction of the AI crowds to the action of the plates. Each event has its own set of props. Each event has its own crowd. There was no ‘one size fits all’ for us on this spot.”
Framestore delivers one seamless shot crowds
In ‘Every Snap’, an Anomaly spot for ‘Dick’s Sporting Goods’, director Derek Cianfrance called on Framestore to fill in stadium crowds over a single camera move. The studio chose a live-action/compositing alternative.
The workflow began with a previs of the entire move. “Previs was an interesting process as the director wanted to use Spydercam to accomplish the long manoeuvring shot,” says senior producer Graham Dunglinson from Framestore in New York. “The camera was ultimately to be programmed by our previs, so precision was absolutely necessary.”
“As we had neither an exact location nor any of the player’s details, it made for an unprecedented challenge of 28 player choreography,” adds Dunglinson. “On-set measurements were taken with a laser distance meter of both stadiums, benches, tracking marker locations on the field and other landmarks so the 3D team could build as accurate a model as possible to aid in tracking.”
Live action was then filmed based on the previs, mostly with empty stands although extras were on hand during the shoot. Framestore’s initial crowd replication approach were CG stand-ins, but the studio soon realized after creating the previs that the shots would be extremely close-up on the spectators. “So instead,” states Dunglinson, “we shot plates of the 100 or so extras in the stands, as well as individuals against greenscreen to be used as sprites to propagate the crowd. Both were directed to go through a series of emotions at certain beats so that the final energy of the stands could be controlled in post.”
Artists tracked the camera move in 3DEqualizer. The stadium was also modeled from reference and measurements taken on set. “The camera and stadium were then brought into Nuke where a combination of the crowd plates and sprites were projected to create the final image,” explains Dunglinson.
The crowd plates had been taken at different camera angles in order to match the approximate angle that the camera would have captured the spectators. “We then had the individual card performances to help us fill in gaps on the edges,” says Dunglinson. “To assist in laying out the cards, we built a crowd generation gizmo in Nuke that allowed us to easily propagate areas of people among the stands, randomize, then individually tweak each fan.”
Compositors then matched lighting, the grade and focus to the plate – which followed the foreground players – and added them back the final shots care of some intense roto. “Having the range of crowd reactions to time offset into their performance was key for enabling us to match the story arc the crowd was to follow,” notes Dunglinson.