Renowned computer vision scientists and animation specialists to continue pushing real-time facial capture and accelerate digital human production with Unreal Engine.
Epic’s founder and CEO Tim Sweeney has flagged how much the company is committed to complete photo-realism and believable animation, and that Digital Human’s is ‘the hardest part, as we all have very elaborate brain systems recognizing faces, intent, and emotion”. Being able to convey film quality digital humans in games is a huge challenge. While film budgets allow for long render times, games need to render in real-time, at frame rates well above theatrical releases.
As part of that R&D agenda, Epic Games today announced that Cubic Motion, a leading provider of automated performance-driven facial animation technology and services is now part of Epic Games. “Digital humans are not only the next frontier of content creation but also the most complex endeavor in computer graphics. With Cubic Motion bringing their computer vision, animation technology and expertise to our digital humans efforts, Epic along with our team at 3Lateral are one step closer to democratizing these capabilities for creators everywhere,” explains Sweeney.
This joining of forces extends Epic’s commitment to advancing the state of the art in the creation of believable digital humans, building on the company’s acquisition of 3Lateral just over a year ago. 3Lateral are world leaders in delivering highly advanced character creation solutions, with enormous depth in facial rigging, and increasingly in facial capture.
“Facial animation that conveys the slightest nuance of human expression is essential to crossing the uncanny valley. We believe that holistically combining Epic’s Unreal Engine with 3Lateral’s facial rig creation and Cubic Motion’s solving technology is the only way to answer this challenge, and ultimately, to reach the pinnacle of digital human artistry with Unreal Engine,” said Epic Games CTO Kim Libreri
Along with facilitating high-quality performance-driven facial animation for AAA video games, Cubic Motion’s expertise has been integral to several noteworthy Unreal Engine real-time demonstrations, including the first “Hellblade: Senua’s Sacrifice” live character performance at GDC 2016, followed by “From Previs to Final in Five Minutes,” which earned Best Real-Time Graphics and Interactivity at SIGGRAPH 2016. Cubic Motion also collaborated with Epic, 3Lateral, and other partners on the “Meet Mike” (2017) and “Siren” (2018) projects. These were photorealistic digital humans animated by real-time performance capture. “We’ve done some of the best work in the industry with the Epic family. The acquisition was the natural next step in our relationship and will allow us even greater achievements,” explains Cubic’s CSO Ben Minall.
“We are delighted to be joining Epic Games and look forward with excitement to this next chapter in our story,” said Cubic Motion CEO Dr. Gareth Edwards. “Together, we are uniquely positioned to push the boundaries of digital human technology, bringing even more realism and immersion to all forms of visual entertainment.”
Epic Games now employs over 2000 people. All Cubic staff will be joining Epic Games, with no layoffs or staff reductions. The Cubic team will continue as is, including the Cubic Motion brand, much as 3Lateral has done since they joined Epic in Jan 2019. “We will remain a proud resident of Manchester as we expand,” comments Minall. “We love the city and have the advantage of being next to one of the world’s great Universities with its unique place in the history of computing.”
Interactivity and real-time is part of the core DNA of the Unreal Engine, and Cubic’s focus “remains to deliver the highest-fidelity animation for all projects whether realtime or offline” comments Minall. Much like the expansion of Epic’s UE4 to the professional market, he adds that “we are already seeing an increased demand for Digital Humans outside of games”.
Cubic Motion’s facial animation technology has also been used on many notable AAA titles including Sony’s God of War and Insomniac Games’ Marvel’s Spider-Man. More recently they contributed to The Dark Pictures: Man of Medan.
Some have asked if this acquisition means that Cubic’s advanced facial tech and services will only be available for UE4? Minall has told fxGuide that Cubic “will continue to provide our products and services to customers regardless of the platform”.
Persona
Cubic Motion’s Persona system, launched mid-2019, provides an end-to-end hardware and software solution for capturing and translating an actor’s performance onto their digital counterpart in real time, and enables immediate character facial animation in Unreal.
As part of Epic Games, Cubic Motion will continue selling hardware and software as standalone solutions. Cubic Motion will continue to support all existing customers, and Epic is working to incorporate their services and expertise into ongoing digital humans efforts.
Persona is powered by the next generation of Cubic Motion’s high fidelity, facial animation technologies. This is built on the experience learned at a host of live events including GDC, FMX and in 2016 winning the SIGGRAPH award for Real-Time Live. The company provided the facial tracking for fxguide’s own Mike Seymour as MEET MIKE in 2017. These early award-winning projects advanced the companies proprietary computer vision techniques which are now available to anyone in Persona. The tech in Persona uses the next generation of facial tracking and solving at Cubic Motion, as product manager Tony Lacey explained: “we’ve designed the new HMC, the on-body system, and all the software.”.
Persona is a bespoke high-end system, that aims to produce the best data possible, but how does that sit with Epic’s general approach of democratizing technology? “We share Epic’s desire to democratize technologies for content production even as the industry moves to photorealism over the years ahead” states Minall.
Persona does not require tracking markers on the face of the actor, although they can be used as part of the quality assurance stage of building a bespoke Persona for a particular actor, (as seen in the top image of fxguide’s Mike Seymour). Cubic Motion uses advanced computer vision and AI to read the actor’s face and provide high-quality facial data.
Cubic Motion will continue to support its customers and industry partners while also accelerating efforts around Persona and Unreal Engine real-time facial animation. Epic’s digital human endeavors have a long planning horizon. Sweeney recently indicated that he believes it is the hardest problem in computer graphics and yet Epic games seek to have completely photoreal real-time people in Unreal, perhaps inside ten years. Until then there is an enormous market for higher and higher quality characters, and capturing high fidelity performances of actors is a key part of that process.
Please note (Disclaimer):
fxguide’s Dr. Mike Seymour has research engagements with Cubic Motion and Epic Games via his Digital Human research Motus Lab.