Arnold: used to help in the fight against child exploitation

The extent to which photorealistic characters can be used outside of filmed entertainment and games was recently made apparent with ’Sweetie’. Envisaged by Mark Woerde, founder of Dutch advertising agency Lemz, the Sweetie campaign for Terre des Hommes Netherlands aimed at bringing attention to internet child exploitation through the creation of a photorealistic CG child. This child was made to be a 3D model – ultimately rendered in Solid Angle’s Arnold – of a 10-year-old Filipino girl (called Sweetie) who then appeared and interacted with people in public chat rooms via webcams. Lemz says it was able to gather incriminating evidence of 1,000 people from 71 countries who interacted with Sweetie.

Creating a believable ‘Sweetie’ that would serve the purpose of convincing people in chat rooms that she was real was no easy task. A small team had to work in secrecy and the producer didn’t reveal the client’s real name. “I was quite excited to be part of this,” says Mao Lin Liao, a freelance digital artist who was responsible for the modeling, shading and rendering of Sweetie after being hired for the project by motion capture studio Motek Entertainment. “But the real heroes behind this project are the people who were chatting for two months with these perverts – imagine what goes through their minds – but fortunately they (the Sweetie team) got mental support from a psychiatrist during those two months and after.”

The brief that Liao received from the client for Sweetie was for a young female, aged about 10 and from the Philippines. In addition, he was provided with some blurry images that represented a girl filmed from behind the webcam. “Sweetie was modeled using Zbrush,” says Liao. “Both for modeling, texturing and hair styling. I have my own workflow developed to utilize Zbrush and Maya nHair.”

For the actual video chats, a series of pre-rendered clips were created similar to how games might use a library of animations for the playable characters. “They included responses like nodding, shaking head, several idles like typing and listening,” explains Jasper Brekelmans, who was tasked with overseeing the mocap tracking/face rig/webcam emulation software. “Animations were created in such a way that they would be seamless when played back to back by looping and blending them.”

Brekelmans says that a custom C++ app was developed that had icons and text representing each of the clips the operator could customize. “By clicking icons the software would queue up the next clip and make sure the currently playing clip was properly played to the end before playing the next for a seamless result. Under the hood it also made sure only clips could be selected that would have matching beginning/end poses, that when no clips were selected random idle clips were chosen to keep Sweetie alive, and that the same clip would not be used twice in a row. This was then all streamed out as a simulated webcam feed into the chat applications and sites.”

A performance capture approach was determined for the final animation and carried out at Motek with a Vicon optical motion capture system, using Blade software to capture body motion, and a helmet mounted 120fps infrared camera for facial capture. The team used facial tracking software from Dynamixyz. “This meant we could record all the nuances from a real world actress, but it also allowed us to do different takes and variations,” notes Brekelmans. “Especially for the webcam clips this provided us with different loops of the same action that would help hide the fact that Sweetie is not a real girl.”

Body animation was retargeted and tweaked with Autodesk MotionBuilder. For facial animation, artists tracked the 2D helmet camera footage through Dynamixyz. Then they took the raw tracking data and fed this into a custom facial retargeting solver developed by Brekelmans. “This did not try to solve the data to a list of known shapes like most other solvers do,” he says, “but tries to directly warp motion from ‘actor space’ to ‘character space’ in order to retain much more fine detail and nuances of the performance.”

– Above: an example of the markerless motion capture approach by Motek Entertainment – the same approach was used for Sweetie.

“The face rig in Maya recreates additional detail like wrinkles, skin bulges and things like lip contact based on the incoming motion data,” adds Brekelmans. “But as with any production that involves mocap we also did manual animation work to artistically tune things and make lip contacts during lip sync slightly snappier.”

Liao notes that rendering in Arnold was extremely fast, although he had to ‘crank up the samples for the haircut’. “I used Min Pixel Width” (MPW) to get a smoother look. But the clients declined it and thought the hair looked too thick. So we had to increase the samples. But I kept the rendertime under 30 minutes per frame on a 12 core machine.”

The final imagery was rendered at 720p – a deliberate resolution choice at webcam quality so as to help convince viewers Sweetie was real, despite the fact that no real 10 year old girl was used as reference. “For any project involving performance capture it can be a challenge to drive a virtual character with this data in a convincing manner,” says Brekelmans. “Especially when the character cannot rely on scanning a real person’s face with all nuances during certain poses, so for us this was also very hard to do.”

Credits:

Modelling/Shading/Rendering: Mao Lin Liao
Animation/Body Rig: Michiel van Iperen
Mocap Tracking/Face Rig/Webcam emulation software: Jasper Brekelmans
Actress Body/Face/Stem: Klavertje Patijn
Production & Motion capture facilities: Motek Entertainment – Jolande Junte
Facial mocap hardware/software: Dynamixyz
Body mocap hardware/software: Vicon
Agency: Lemz
Rendering software: Solid Angle Arnold renderer