Recently, Epic Games and filmmakers’ collective Bullitt assembled a team to test out the latest in-camera VFX toolset, part of the extensive suite of virtual production tools in the upcoming Unreal Engine 4.27 release. To put each of the new tools through their paces, they created a short test piece to mimic a production workflow.

Shot entirely on NantStudios’ LED stage in Los Angeles, the production test capitalized on new workflows that enable dynamic lighting and background changes on-set with minimal downtime, together with multi-camera and traveling vehicle shoots. The project used twin NVIDIA Quadro A6000 GPUs. The test was directed by Ryan McNeely and the DOP was Matt Jenson, ASC. The team was able to produce final pixels in just four days – a demonstration of the efficiency and creative freedom the toolset offers.

Included in the making of video released by Epic are comments from the Russo Brothers (Captain America: The Winter Soldier, Captain America: Civil War, Avengers: Infinity War, and Avengers: Endgame). For renowned director Anthony Russo, what is important is the ability of in-camera VFX to bring everything together and enable the team to see the final result right on set. “One of the things that excites us most, of course, about this is the fact that we can do in‑camera choreography, where all the elements of the frame are actually in concert with one another and organically working on one another to create a more visceral experience,” he explains.

The new 4.27 addresses several areas, especially production issues users had with nDisplay which has been given a major rewrite. The new production-ready  tools in 4.27 include:

  • Better nDisplay integration for simpler use, an easier editor and higher in-camera resolution,
  • OCIO support in nDisplay to enable accurate color calibration that connects content creation in UE4 to what the real-world camera sees on the LED volume
  • Correct motion blur for traveling shots that accounts for the physical camera with a moving background
  • Optimizations for shooting with multiple cameras simultaneously
  • C++ BP Root Actor now consolidates all nDisplay-related features and topics under one roof from configuration file and in-camera VFX to previewing a display cluster in UE4.
  • Synchronization – Better NVIDIA Swap Sync Support: increased performance and stability

 

NantStudios is a state-of-the-art full-service production setup comprised of traditional, broadcast, and virtual production stages. The company is based in Los Angeles with two campuses in Culver City and El Segundo. The company’s virtual production stages are serviced by an experienced team with decades of virtual production, visual effects, and engineering experience. The El Segundo is a seven-acre + campus with a 24,000 sq. ft. sound stage housing an advanced independent in-camera visual effects (ICVFX) volume. Epic Games has chosen to co-locate its newest Los Angeles Lab at the Innovation Campus.

“We are thrilled to be working with Epic Games, using Unreal Engine as the virtual production platform to bring the next generation of film and television production to Los Angeles,” said Michele B. Chan Soon-Shiong, President of NantStudios. NantStudios’ ICVFX sound stage was designed and built in collaboration with Lux Machina who has been a leading force in designing and implementing ICVFX stages.

NantStudios’ services team is led by Gary Marshall, Director of Virtual Production, and Keaton Heinrichs, Director of Studio Operations & Business Dev. They have decades of experience, performing critical roles on such productions as The Lord of the Rings, Avatar, Gravity, The Lion King and The Mandalorian

“We are excited to expand our Los Angeles presence at the NantStudios campus, a hub for the virtual production community and an incredible new stage-for-hire which has been in high demand. This is a space where the Unreal Engine team will not only accelerate testing of new virtual production workflows and cutting-edge developments but also get to see them go into production on real-world projects,” said Marc Petit, Epic Games General Manager of the Unreal Engine.

The team also took advantage of new Level Snapshots that enable directors to easily match previous takes when returning to an earlier set or lighting configuration; sophisticated remote control UI building tools that offer creatives direct control of the Unreal Engine-powered elements of the shoot from the convenience of a tablet or laptop; and OpenColorIO color management that ensures artists creating Unreal Engine assets can see what the final captured output will look like. Visit the Unreal Engine blog for more information about this project.

Unreal Engine 4.27 is currently available in Preview. The new workflows and tools it offers are influenced by input from filmmakers who are currently pushing the limits of virtual production. The Bullitt team discovered that the new toolset represents a leap forward in quality, efficiency, and ease of use.

NantStudios’ ICVFX sound stage features LED panels and processors integrated into a novel modular-designed volume video system. The volume can be configured for real-time camera-tracked Unreal Engine environments, pre-rendered plate playback, or a hybrid approach. Live composite feeds and volume spy-cams can be streamed to any remote location on NantStudios’ accelerated fiber network infrastructure. The NantStudios ICVFX sound stage and innovation campus began test productions in late 2020 and has been booking production since Feb 2021. For example, Resorts World Las Vegas, Psyop, and Hooray Agency booked the UE4-powered innovation campus in El Segundo to utilize the ICVFX volume for their latest promotional campaign, featuring the talents of Celine Dion, Carrie Underwood, Katy Perry, Luke Bryan, Zedd, and Tiesto. The project was shot almost entirely in-camera utilizing real-time LED backgrounds, without the need for post green screen composites.

 

ALSO in 4.27

While not directly VP, also new in 4.27 is the ability to tune facial capture with Live Link Face to the individual performer. This improves the quality of the animation data streamed to UE4 with the new calibration feature in Live Link Face. This enables a neutral/rest pose expression to be set, which is used by the app to offset the data from ARKit and provide a more tailored result that is a closer match to the actual performance.

The new toolset in Unreal Engine 4.27 also includes GPU Lightmass, which takes advantage of light baking on multiple GPUs to radically accelerate set changes, making them as much as an order of magnitude quicker to do. It can reduce an overnight change of sets to something under an hour.  “As a director, time is your biggest enemy, and the thing about this, the speed at which we’re able to move the amount of setups, that’s honestly the dream,” McNeely comments. “The ability to find new shots on the stage is just incredible. In the same amount of time that it would take me to change a light in the physical world, I’m able to have new lighting on the wall. As a director, that’s a game changer.”

Unreal Engine 4.27 is currently available in Preview, and the ICVFX project will be soon released for users to experiment with themselves.