AT GTC, NVIDIA Edify, a multimodal architecture for visual generative AI, has been shown with expanded capabilities. As part of CEO Jensen Huang’s extensive 2-hour keynote, a host of new tools were briefly touched on, and many will become increasingly important to VFX teams.
3D asset generation is among the latest capabilities Edify offers developers and visual content providers. With new generation tools, users can exert more creative control over their AI image generation. For example, Shutterstock is rolling out early access to an API built on the latest Edify architecture that lets creators rapidly use text prompts or images to generate 3D objects for virtual scenes.
Visual content creator Getty Images will now add custom fine-tuning capabilities to its commercially safe Generative AI service, helping enterprise customers generate visuals that adhere to brand guidelines and styles.
Developers can soon test drive pre-trained Edify models by Getty Images and Shutterstock as APIs through NVIDIA NIM, a collection of microservices for inference announced today at the keynote. Developers can also train and deploy their own generative AI models using the Edify architecture through NVIDIA Picasso, an AI foundry built on NVIDIA DGX Cloud.
Livestreaming platform Be.Live is using the NVIDIA Picasso foundry service to provide real-time generative AI that enables the automated creation of visuals and an interactive experience for audiences. Bria, a holistic platform tailored for businesses developing responsible visual generative AI, has adopted Picasso to run inference. And creative studio Cuebric is enhancing filmmaking and content creation by developing Picasso-powered generative AI applications to build immersive virtual environments.
Shutterstock’s 3D AI Services, available in early access, will enable creators to generate virtual objects for set dressing and ideation. This capability can drastically reduce the time needed to prototype a scene, giving artists more time to focus on hero characters and objects.
The commercially safe model was trained on Shutterstock’s licensed data. Shutterstock has actually compensated hundreds of thousands of artists, with anticipated payments to millions more, for their content IP’s role in training generative technology.
Using the tools, creative professionals will be able to rapidly create assets from text prompts or reference images and choose from a selection of popular 3D formats to export their files. The Edify 3D based service will also come with built-in safeguards to filter generated content.
Product visualization rendering
Getty Images
Getty Images continues to expand the capabilities offered through its commercially safe
generative AI service, which provides users with indemnification for the content they generate.
The Getty Images AI is trained exclusively on Getty Images’ own creative content. As such, the new images created are commercially safe—trained exclusively on permission-approved data to prevent problematic imagery generation.
New Edit tools
Getty Images released Edify-powered APIs for inpainting, to add, remove or replace objects in an image, and outpainting, to expand the creative canvas. These new features are now available on both Gettyimages.com and iStock.com and starting in May, Getty Images will also offer services to custom fine-tune the Edify foundation model to a company’s own brand and visual style.
The new services will feature a no-code, self-service method for companies to upload a proprietary dataset, review automatically generated tags, submit fine-tuning tasks and review the results before deploying to production.
As part of custom fine-tuning tools, Getty Images will release a collection of APIs that provide finer control over image output, one of the biggest challenges in generative AI. Developers will soon be able to access Sketch, Depth, and Segmentation features — which allow users to provide a sketch to guide the AI’s image generation, copy the composition of a reference image via a depth map, and segment parts of an image to add, remove or retouch a character or object. Users can download and license any visual they generate, at the user’s desired resolution.
360 AI HDIs
Dassault Systèmes’ 3DEXCITE applications for 3D content creation and CGI studio Katana are incorporating Shutterstock generative 360 HDRi APIs into their workflows based on NVIDIA Omniverse, a computing platform for developing OpenUSD-based 3D workflows and applications.
Shutterstock is also building Edify-based tools to light 3D scenes using 360 HDRi environments generated from text or image prompts.
Conversation Prompts to 3D environment
Accenture Song is using the Omniverse platform to generate high-fidelity Defender vehicles from computer-aided design data for marketing purposes. Coupled with generative AI microservices Accenture Song is enabling the creation of cinematic, interactive 3D environments via conversational prompts. The result is a fully immersive 3D scene that couples realistic generated environments with a digital twin of the Defender vehicle. The NVIDIA Text to image prompt tools are currently available in 75 languages.
Clients
Getty Images’ API services are already being used by leading creatives and advertisers including:
● Dentsu Inc.: The Japan-based ad agency is using Getty Images’ generative AI API service to power MAFA: Manga Anime For All, an app that can create manga and anime-style content for marketing use cases.
● McCann: The creative agency harnessed generative AI to help develop an innovative game for its client Reckitt’s over-the-counter cold medicine Mucinex, in which customers can interact with the brand’s mascot.
● Refik Anadol Studio: Known for using generative AI in its artwork, the studio is showcasing a rainforest-inspired art installation at GTC, created using Getty Images’ AI model fine-tuned with Refik’s rainforest catalog.
● WPP: The marketing and communications services company is partnering with The Coca-Cola Company to explore how fine-tuning Getty Images’ model can help to build custom visuals that meet brand styles and guidelines.
