GTC demo showcases an AI-powered avatar responding to natural speech and making intelligent recommendations linked to Epic Games MetaHumans.
Unveiled this week at GTC, NVIDIA showed ‘Violet’ is a cloud-based avatar driven via NVIDIA Omniverse Avatar Cloud Engine (ACE), a suite of cloud-native AI microservices that make it easier to build and deploy intelligent virtual assistants and digital humans at scale.
To deploy interactive avatars such as Violet, developers need to produce the 3D character can use computer vision, natural language understanding, AI and advanced avatar visualization tools. This has been extremely difficult up until now. While Epic Games provided a powerful tool for MetaHumans to create the avatar itself, most applications of the MetaHumans typically had them being driven by a person, akin to a puppet. The ACE platform provides artists and TD with a way to build digital humans driven by a core AI engine. This allows the avatars to effectively be independent and ‘come alive’.
The Violet demo showcased Omniverse ACE avatar development, delivering all the AI building blocks necessary to create, customize and deploy interactive avatars. The first part of the demo did this with a cartoon-style avatar which was shown being deployed in a simulated restaurant ordering environment. The use case for such agents is customizable AI assistants that are bespoke to virtually any industry and can help organizations enhance customer experiences.
This same avatar was then shown to be able to be expanded to answer more general questions (see below). The final part of the demonstration was replacing the cartoon Violet with an Unreal Engine-powered MetaHuman.
How Omniverse ACE Brings Violet to Life
The demo shows Violet as a fully rigged avatar with basic animation. To create Violet, NVIDIA’s team used the company’s Unified Compute Framework (UCF), a fully accelerated framework that enables developers to combine accelerated microservices into real-time AI applications. UCF helped the team build a graph of microservices for Violet that were deployed in the cloud.
Omniverse ACE powers the backend of interactive avatars, essentially acting as Violet’s brain. Additionally, two reference applications are built on ACE: NVIDIA Tokkio and NVIDIA Maxine. NVIDIA Tokkio leverages ACE, to make it easier to build and customize lifelike virtual assistants and digital humans.
NVIDIA Maxine is a suite of GPU-accelerated AI SDKs and cloud-native microservices for deploying AI features that enhance audio, video, and augmented reality effects in real-time. Maxine’s AI models create high-quality effects such as changing where someone appears to be looking on a Zoom call, by inferring a new neural render of them looking directly at the web camera. This works even with a standard computer and webcam. Maxine integrates the NVIDIA Riva SDK’s real-time automatic speech recognition and text-to-speech capabilities with real-time “live portrait” photo animation and eye contact features.
The demo with Violet highlights how developers of digital humans and virtual assistants can use Omniverse ACE to accelerate their avatar development workflows.
Omniverse ACE also delivers microservices that enable developers to access the NVIDIA AI technology, with no coding required. Some of the latest microservices include:
- Animation AI: Omniverse Audio2Face simplifies the animation of a 3D character to match any voice-over track, helping users animate characters for games, films or real-time digital assistants.
- Conversational AI: Includes the NVIDIA Riva SDK for speech AI and NVIDIA NeMo Megatron framework for natural language processing, allowing developers to quickly build and deploy cutting-edge applications that deliver high-accuracy, expressive voices and respond in real-time.
These tools and AI platforms allow for much wide use of digital humans than just in Media and Entertainment. As an indication of how widespread the use of this technology is likely to be leading professional-services consulting company Deloitte will be working with NVIDIA to help their customers deploy these digital human applications. At GTC, Deloitte announced that new hybrid-cloud offerings for NVIDIA AI and NVIDIA Omniverse services and platforms, including Omniverse ACE, will be added to the existing Deloitte Center for AI Computing.
