Digital skin alteration – studying racial bias

It’s always interesting to hear about how VFX and CG techniques can be used in ways other than entertainment. We recently heard that digital beauty VFX specialist and fxphd prof. Nathaniel Westveer had got involved in a ‘emotional response through racial bias’ research run out of the University of Cambridge. Here, Westveer was asked to darken the skin color of several people who had agreed to be part of the study. We asked him about the process.

The project came to Westveer from a friend, Gabriela Pavarini, a PhD student in the Department of Psychology at the University of Cambridge. Her research is in the area of facial mimicry of in-group (that is, a social group to which a person psychologically identifies as being a member) versus out-group (the opposite) members.

Pavarini requested Westveer, who has had considerable experience doing beauty work as a freelance VFX artist, to show changes in skin tone from light to dark on the subjects as part of THIS research into racial bias. “The research I was briefed on,” says Westveer, “was to show a comparison between a Caucasian person or a light-skinned person and someone who might have a darker skin background – and to analyze how people react to emotional cues.”

So how did Westveer do it? Well, firstly he received the videos and saw an immediate challenge in changing the skin tones based on resolution. “In this case it was just 8-bit compressed files to work with,” notes Westveer. “There’s not a whole lot to work with pulling down color values-wise, so you’ve got to more or less fill up with the colors on top of it as well as working with the footage.”

“First,” says Westveer, “I look at the plate and I see how far you can push the skin tones, how far you can hue shift it – and build on top of that.” Since the footage was a little too noisy to key, Westveer relied on roto to isolate the skin parts of the face. Then he art directed how much of a change in skin color would occur based on aspects such as hair type and what he could find as reference.

“Some ended up with more of a Hispanic/Latino look,” states Westveer. “Others I went as dark as I could, just based on how accurate it did look with the hair type. The study focused on Mediterranean people who had already darker hair and hairstyles, but still had a fair complexion.”

Westveer was surprised he could push one subject, a male with very short hair, to have quite a dark skin color. “There was also another subject – he has a toothy grin and he’s hair was kind of all over. When I got into the process of pushing the skin tones down, I realized I more mistook him as Indian than anything else – I didn’t really expect to get that result. And then I art directed him from there.”

The study is ongoing, but already the researchers were surprised with the highly photorealistic results after seeing Westveer’s work. “The first response was, did I do anything with the eyes? Which I didn’t. When the skin tones got darker it made the eyes appear differently in some plates. The researchers didn’t really know what to expect from this process, so they just gave me an objective to give them darker skin tones. But they were very excited about the results – they come off pretty photorealistic.”

You can find out more about Westveer’s visual effects work at his website: http://nathanfx.com.