This fxguide story is sponsored by HP.
Next-gen studio, ZeroSpace in Brooklyn, NY has leveraged NVIDIA multi-GPU workstations for high-fidelity real-time workflows.
Part service provider and technology incubator, ZeroSpace specializes in a first-of-its-kind studio specializing in digital, virtual, and live content. With more than 50,000 square feet of rentable studio space, the innovative production and creative studio is home to one of the largest LED volumes on the East Coast, as well as a core team of 20 world-class artists, engineers, and technicians. Along with providing in-camera visual effects (ICVFX) services to clients. ZeroSpace offers performance capture, full-body photogrammetry, and volumetric capture. It is the only studio with both a fixed installed subway and car process in New York. The company also has an in-house virtual art department (VAD), and a real-time art department (RAD) team offering 3D and VFX services; all the while developing its own creative IP and proprietary technology.
Much of the studio’s robust technical capabilities are achieved through strategic partnerships, and leveraging cutting-edge equipment and pipelines. The ZeroSpace engineering team is constantly exploring ways to improve internal workflows and move the overall virtual production industry forward. Most recently, this technology-first approach drove the studio to adopt Z by HP’s new Z8 Fury desktop workstation, which supports up to 56 cores in a single CPU and up to four high-end GPUs.
“We received our first Z8 Fury in late June and it’s already been a major lifesaver for many reasons. With it, we were able to simultaneously capture multiple performers using markerless motion capture for a crossover music video brand event,” shared Evan Clark, ZeroSpace Head of Research.
Powering past software limitations
When a customer approached the company to do a motion capture shoot with a five-member band in an unrestrictive environment, the team at ZeroSpace was up for the challenge. However, the studio’s AI-assisted markerless motion tracking software was only capable of capturing two performers simultaneously.
“We were trying to come up with a workaround. We worked closely with our partner, who was able to send us an early beta version of new software that could split the performer models on multiple GPUs, and then recombine them downstream. It was Linux-based, and we were already running Ubuntu on our Z8 Fury, so we decided to install it and give it a try. Like magic, we were able to split the workload across three NVIDIA RTX™ A6000 GPUs, allowing us to track five people simultaneously at 60 frames a second; it made a significant difference,” Clark noted. “We were able to shoot all the content we needed in a single day; this was a game changer as anyone in the business knows that when it comes to film shoots, time is money.”
Though the Z8 Fury supports up to four NVIDIA RTX GPUs, the team opted to leverage one of the card slots for a 25-gig networking adapter to run the array of 12 machine-vision FLIR cameras with sufficient bandwidth to receive all the frame data from the performers. Additionally, optical motion capture was used in tandem for prop tracking the performers’ musical instruments. All data was fed through Epic Games’ Unreal Engine, where it was synchronized and recorded. Clark found building out the new workflow to be relatively straightforward and had the setup tested and running in about a day. Typically, the studio’s real-time pipeline for motion capture requires two or three machines; using a single box simplified the approach.
“If it wasn’t for the Z8 Fury, we would’ve been out of luck because most of the machines we have in-house are all single NVIDIA GPUs. Fortunately, it gave us the absolute best baseline to work with; otherwise, we would have had to record the band in separate takes, which would have compromised their performance and taken away from the final result.”
Evan Clark, ZeroSpace Head of Research.
Advancing real-time fidelity for ICVFX
Clark and his team have also been testing the Z8 Fury for ICVFX applications, including experimenting with the SMPTE ST 2110 protocol and Unreal Engine graphics splitting. He mused, “Pushing the level of quality on an ICVFX volume while maintaining high frame rates is the Holy Grail. How can you get enough definition to really make it look realistic without having to leverage camera tricks or any type of post-VFX work? There are limitations when you’re trying to scale a real-time render engine to millions, -if not billions or trillions of pixels, but you need to get that level of fidelity to convince the eye that what it’s seeing is real. With the density of the Z8 Fury and some of the new Unreal Engine features, we can run bigger scenes and have a really solid mechanism to split and re-merge the scene on a single machine in a very deterministic way.”
Created in partnership with 4Wall Entertainment, ZeroSpace’s sizable in-house LED volume is nearly 40 panels wide by eight panels tall, measuring 65 feet on the curve, with a total resolution of slightly more than 8K. Having multi-GPU machines for wall rendering makes it easier for the team to split up workloads, thereby reducing the amount of geometry run by a single game engine instance. This load redistribution enhances performance, enabling faster frame rates and better overall visual quality since the machine is managing a small subset of the task with the entirety of the compute resource.
“In-game engine ray-marching and ray-tracing techniques are really starting to shine because of multi-GPU workload distribution. With that additional resource overhead, you can allocate for dynamic lighting in a scene and be able to change scene parameters in real-time. As these types of scenes improve and the technology evolves, dynamic changes will be able to be made more effectively on stage during a shoot,” said Clark. “Image-based lighting has also resurfaced as a technique. and requires significant processing and computing for pixel mapping to make a film shoot possible on a volume. When you have sufficient compute, it unlocks creative handcuffs for your artists.”
Building on a strong foundation
Established in 2018 as a downtown Brooklyn immersive art space, ZeroSpace evolved into a production and incubation operation in 2020. At the outset of its virtual production journey, the team built custom machines, in some cases using consumer-class GPUs in the workstations. Opting to standardize on the Z8 Fury workstations freed the engineering team from building and maintaining workstations, allowing them to focus instead on workflow innovation.
“One major benefit of adopting Z by HP workstations is the level of support we receive. If we run into an issue, the HP technician is out right away. On film shoots, every minute is burning hundreds of thousands of dollars, so fast machine repair and replacement is a major advantage; you can’t do that with custom machines,” concluded Clark. “Our HP machines are the most reliable in the building, and the level of support provided by HP is fantastic.”
The Z8 Fury configuration included:
- 3x NVIDIA RTX A6000
- Intel Xeon W7-3455 24C / 2.5GHz
- 128GB (8x16GB) DDR5-4800 ECC RAM
- 2x1TB NVMe
- Dual 10/25G NVIDIA Mellanox ConnectX-6
You can learn more about the extreme power and performance of the HP Z8 Fury, here.
ZeroSpace is a customer of HP and was provided with a machine for testing purposes.
© Copyright 2023 HP Development Company. L.P.
