In the latest podcast in our Visual Disruptors series, Mike Seymour chats with Mohen Leo from ILMxLAB, Lucasfilm’s immersive entertainment division, about the virtual production techniques that led to VR experiences like Star Wars: Secrets of the Empire. He also talks with Rick Grandy at NVIDIA about the groundbreaking RTX technology behind the Reflections video, which showcased real-time ray tracing using Star Wars Storm Troupers.
Mohen Leo, Creative Director and Visual Effects Supervisor at ILMxLAB, explains that ILM’s interest in virtual production is what led his group to develop experiences like Star Wars: Secrets of the Empire. “ILM as a whole realized that if you can use these techniques to let directors interact with the world of Star Wars, then audiences would want to interact with the world of Star Wars,” he says. “We’re trying to basically have the end result be something that’s actually available to consumers.”
As a result of this exploration into virtual production and real-time rendering, ILMxLAB teamed up with The VOID to produce the immersive VR experience Star Wars: Secrets of the Empire, where up to four players can enter VR together as avatars to meet a challenge, interacting with physical objects and even feeling wind on their skin.
“Unreal Engine and NVIDIA have been helping us because they get the tools that we use,” he says. “You don’t have to apply a whole different mindset to trying to get a realistic-looking image in real time anymore,” explains Leo.
In the second part of the podcast, Rick Grandy, Senior Solutions Architect at NVIDIA, explains the RTX technology that makes real-time ray tracing possible.
NVIDIA stunned the VFX community at GDC 2018 when, in conjunction with Epic Games and ILMxLAB, they used RTX to showcase real-time ray tracing in Unreal Engine on the highly reflective Star Wars characters in the video Reflections. After that, “pretty much every studio was calling us asking how they get their hands on it!” laughs Grandy.
He explains that RTX architecture introduces two new types of cores for distributing rendering tasks, Tensor and RT, with RT devoted to accelerating ray tracing. Grandy also explains how NVIDIA’s denoising techniques reduce render time for reflections and shadows while keeping visual quality sharp.
Ray tracing with RTX isn’t yet part of Unreal Engine, but NVIDIA continues to work with Epic to push the envelope and advance this technology, as evidenced by the Speed of Light demo at SIGGRAPH 2018 (see above).
And visit our Virtual Production hub of our media partners Epic Games, to get more videos, podcasts, and insights into this emerging field.