fxguide recently visited Cubic Motion In Manchester to look at their advanced facial tracking services, which is one of the most advanced in the world. The company operates from a small industrial complex in Manchester, north of London where teams of production artists and R&D engineers work on some of the most complex facial solvers in the industry.
The company regularly produces hours of facial solutions for its game clients and increasingly is being hired as outside specialist to help vfx project teams. The company is not a general effects house, they do not offer end to end solutions, although they are happy to consult. They just know faces and how to solve them, – very quickly and very well.
Cubic’s solutions are powered by advanced computer vision breakthroughs applicable across a wide range of applications. Cubic Motion’s vision technology enables some of the world’s fastest solutions for high quality facial animation. Recent projects include:
- Call of Duty: Modern Warfare Remastered
- Call of Duty: Infinite Warfare
- Call of Duty: Advanced Warfare
- Kingsglaive: Final Fantasy XV
- Batman: Arkham VR & Batman: Arkham Knight
- Squadron 42 (Star Citizen)
- Until Dawn & Until Dawn: Rush of Blood
- The Order: 1886
- Hellblade
- Ryse: Son of Rome
- Shadow of Mordor
- Battlefield 4
Cubic Motion gained further international recognition after being a key part of the real time facial animation system that won The Real-Time Graphics & Interactivity Live Award at Siggraph 2016. Cubic Motion used stereo tracking for Siggraph Live, although as we reported they had previously used a mono solution for the initial GDC HellBlade Demo.
Cubic Motion in the Hellbade demo provided all the high speed markerless stereo facial tracking that was solved into the facial rig provided by 3Lateral and then rendered in real time in the UE4 engine. The company only has milliseconds to solve the face as the system normally runs at 60fps in stereo. The Cubic Motion system used Glenn Derry’s Technoprop head rig and its computer vision cameras as the input helmet device.
Cubic Motion does handle solving entire facial animation projects outsourced to it by clients, normally game developers. It also works to support teams keen to keep the creative process within the walls of their studio or vfx house. The company is designed around working as part of a pipeline solution, supporting as a specialist company other larger teams which means the company has a very adaptive pipeline. Cubic Motion boast years of highly specialised computer vision experience, and it is that understanding that lets them support vfx and games teams still struggling with fast evolving facial technology.
The company is headed by Dr Gareth Edwards, who joined in 2009 from Image Metrics. He is an expert in computer vision. Edwards, along with Prof. Chris Taylor and Prof. Tim Cootes were awarded the IEEE 2015 Test of Time Award for their 1998 paper “Interpreting Facial Images Using Active Appearance Models”, that they wrote together at the University of Manchester. The award recognizes outstanding, influential papers from the past 15-20 years. It has been cited over 500 times by other academic papers, and it is a seminal paper in the field.
The company is still based in Manchester, close to the University and the local research community but they just expanded in December, opening an office at Pinewood Studios. The company’s new Pinewood Studios office is headed up by Cubic Motion’s recently hired computer vision expert Dr Jane Haslam. “We’re thrilled to join the community at Pinewood Studios. The technology and creativity at Pinewood combined with the historical significance and heritage makes it a dynamic location to have a further base. Pinewood is the ideal place for Dr. Haslam to lead our efforts to bring our outstanding real-time solutions to wider markets” commented Edwards.
One of the ways that Cubic Motion works is by gaining training data and using advanced cutting edge techniques to then run in real time during event such as Real Time Live. The company uses some very advanced computer learning approaches but as hard core experts they are also quick to point out that the ‘sudden’ explosion of Deep Learning / Neural Network solutions is partly true but also partly hype. “There were many good ideas around for decades” explains Edwards “… but certainly the key research is not just in neural nets” He points out that a key component of the deep-learning explosion that is currently being experienced is the availability of massive datasets and increased computational power. “This kind of data simply wasn’t around in the 1990’s or earlier”. He recently wrote “if you’re starting out in machine learning or computer vision, don’t assume you have to focus on deep-learning (or at least not the neural net variants thereof – plenty other methods are equally “deep” in my opinion). There’s a goldmine of earlier research waiting to be exploited and advanced, without a neural net in sight”.
Cubic Vision currently solves face tracking and capture problems with “precision currently untouchable by deep-learning methods” he adds. Edwards does agree that there has been a major growth in demand for engineers with AI backgrounds. However, partly due to their location and the depth of experience the company has in specialist machine vision, “we normally can hire the right people we want, when we are hiring”, which the company is currently doing at the moment. The demand for specialist face tracking, leading to digital humans, avatars and digital agents is currently exploding as there occurs a nexus of gaming technology meeting visual effects expertise.
Not only has the company been hiring computer vision experts but familiar faces from the visual effects world are also joining Cubic’s efforts. Bill Collis – President and Board Member of The Foundry has joined Cubic Motion’s Advisory Board. Collis is a Sci-Tech Award Winner, who steered the Foundry for many years with the vast expansion into Nuke, Mari, Katana and Modo. He has his own PhD in signal processing and numerous patents. “I was present at the amazing presentation at Siggraph in September this year when Cubic Motion won the Real-Time Graphics and Interactivity Award. Without doubt Cubic Motion have unparalleled computer vision technology and I’m really looking forward to helping the company realise their ambitious future plans.”
The Advisory Board at Cubic Motion also includes Vladimir Mastilovic, CEO & Founder of 3Lateral (who were also part of the HellBlade Team – see photo above) and Manchester University’s Professor Chris Taylor O.B.E who co-authored the winning Test of Time Paper, in 1998 with Edwards. This brain trust aims to continue expanding slowly but primarily with a focus on excelling at the near nuclear arms race of high end facial tracking and solving.
