Mill Mascot is an animation system developed by The Mill that combines real-time animation and game engine technology with motion sensors that enables a digital puppeteer to control the animated character through hand gestures.
The project was initially developed in the company’s New York offices but it is now a global offering with Mascot setups in every one of The Mill’s studios. The formation of Mill Mascot was led by The Mill’s Creative Director, Jeffrey Dates. He developed the initial idea for Mascot based on his background as a VFX artist. He conceived it as a way to create fast animated episodic content. The system was then evolved further and developed by Mill Technical artist, Xuan Seifert, who worked closely with Dates. Eric Renaud-Houde has more recently joined as a part of the Mascot’s development team, working mainly on the facial tracking.
The Mill offers this as a service to give directors, creatives and clients the ability to ‘direct’ CG character performance live on set, giving feedback and making creative decisions on the fly. It can also be used at live events, as it was at HPE Discover, allowing brand-created characters to interact with consumers.
The system was developed by The Mill to address the ever-growing client needs for fast and intelligent characters, enables broadcast quality CG characters that are not animated but performed live.
Mill Mascot combines real-time game engine technology using Unreal UE4, with motion hand sensors, so characters can be puppeteered with hand and facial gestures.
The facial controls are driven by an iPhone App that was custom built by the join Head of Creative Technology, Eric Renaud-Houde.
Hand tracking is done without the need for a data glove, at the moment, the team uses the Leap Motion system, but the system is hardware agnostic and not tied to any one sensor.
The system by-passes the lengthy timelines involved with traditional animation and pre-rendered content, reducing animation and rendering time down to 42 milliseconds per frame. The development of Mill Mascot is part of The Mill’s commitment to re-define the future of the industry by actively pursuing, researching and developing product solutions that help tell better stories.
The ability to interact with CG characters and create animations on the spot, is a great solution for immediate and relevant social media content featuring brand mascots, as well as live performance and episodic entertainment.
Pop Llama
The Mascot system is related to the technology used for the Pop Llama project from 2017. But the Pop Llama live experience was initially conceived to show early prototypes of Mascot combined with The Mill’s Cyclops technology. Whilst Pop Llama employs some of the initial early-stage setups of Mascot, such as live puppet input, this earlier test project was focused on the real-time reflection and lighting integration that Cyclops offers. The Cyclops system was a key part of the Epic Games/Mill The Human Race project. The Cyclops toolkit enables tracking, lighting and rendering of CG objects in real-time. This is a key aspect of seamlessly integrated the CGI into live action environments.
Most real time systems do not read the changing real world lighting and incorporate that into the interactive render. But Pop Llama was built with an environmental 360 input data system. With this technology, the animated CG objects are lit by their dynamic environments. Cyclops is not currently a part of the current Mill Mascot pipeline, it’s a separate entity.
Control
The Mill is currently running the system on Windows. Mill Mascot is a full-service package that the company offers for linear VFX character content as well as live performance. The software isn’t for sale, but the team can license experiences using Mill Mascot for installations.
Below are some campaigns that have used the technology :
HPE ‘Tame the IT Monster’ – LIVE in Las Vegas
Seanna’s Ocean Buddies – A Sesame Studios Short
Monster.com ‘Vonster’ – Social Media Campaign
The Mill offers character design and development as a key part of the Mascot pipeline, but the team can also take existing CG characters and re-purpose them to be used in the real time Mascot system.