Artificial intelligence helps with make-up

Icon: enlarge

If you are fair-skinned and have blue eyes, you only have to enter a few keywords for make-up tips on the Pinterest photo platform. Searching for words like lipstick, eyeliner and mascara alone brings up thousands of images on the styling suggestions page. But users with dark skin color hardly stand a chance with these general search terms. You need to narrow your search down with specific terms that describe your skin tone.

With the help of artificial intelligence, Pinterest wants to change that and has revised its skin tone filter, which is now being introduced in countries such as Canada, Ireland and Great Britain after the USA. With one click on the color tiles, the user selects which tone the skin of the models in the pictures should have. It is still unclear when the filter will be introduced in Germany.

In order to display suitable results to around 400 million users, Pinterest analyzes the images on the portal and sorts the people shown by skin color. So far, the online platform has primarily relied on face recognition when selecting images. But due to the high error rate with dark skin tones, a color filter now also checks the photos for skin pixels.

In a blog post on the "Medium" platform, Pinterest AI boss Nadia Fawaz writes: "It is a complex challenge for image recognition programs to identify the range of skin tones in a photo." This is due, among other things, to shadows, different lighting conditions and other obstacles. "Since we had to deal with more complex images, we broke away more and more from face recognition."

According to Pinterest, this has several advantages: Among other things, faces that are covered with a mask should be recognized, which is becoming increasingly common during the corona pandemic. In addition, the software now also suffices to take close-up photos of lips and eyes to filter out images for make-up tips.

Training the AI ​​decides on distortion

Most make-up apps, however, still rely on face recognition. The problem: Studies show that AI software recognizes white people in photos and videos much better. Black and Asian-looking people have up to a hundred times more errors. In recent months, this has led, among other things, to government websites either not recognizing the passport photos of dark-skinned citizens or interpreting a closed mouth as open.

Because of this discrimination, some manufacturers of AI make-up tools now advertise that their software also recognizes black people well, such as developer Atima Lui on Twitter:

The developer and her team at Algoface have made sure that enough images of people with dark skin are fed in to prepare the software for face recognition. Like the make-up app ModiFace from L'Oréal, Algoface relies on real-time make-up tips online.

On the website, users can use webcam to try out how blush works on their cheeks and how strong the eyeliner can be. The virtual make-up is applied to the correct areas of the face by software and users see the result on the screen in real time so that they do not have to buy the products first.

more on the subject

Face recognition: How racism gets into softwareBy Jörg Breithut

From day one, one made sure that the software was not biased, said Algoface technical director Taleb Alashkar in an interview with "The Sociable". "The training data will be biased when skin tones are missing," says Alashkar. Therefore, all skin colors were taken into account when the AI ​​was trained. A point on which many researchers agree: The training is decisive in determining whether the software is biased or not.