Artificial intelligence: five minutes to understand Google's new skin color detection system

The tech giant wants to commit to diversity.

During its annual conference, Google announced last Wednesday the launch of a new classification of shades, the Monk Skin Tone (MST) Scale, which will allow, according to the company, a better representation of skin colors through its platforms.

Explanations.

How does this classification work?

Before the creation of this index, which is used to classify skin colors into several types, artificial intelligence engineers relied on the Fitzpatrick classification, originally designed for dermatologists in 1975, which detailed only six shades, and favored shades clear.

They used it in particular to classify skin types when it was necessary to program artificial intelligence systems learning to recognize humans on an image.

But for the creators of the MST Scale, this index did not represent dark skin well enough, which risked creating a bias.

Google will be using the Monk skin-tone 10 point scale to better represent users using Google products.

In Search you'll now be able to search for makeup & products that best match your skin-tone.

The Monk Scale will be open sourced for the industry to leverage #GoogleIO pic.twitter.com/O0xO234yfB

— Carolina Milanesi (She/Her) (@caro_milanesi) May 11, 2022

Shaped by engineers and researchers from Google, and sociology professor Ellis Monk, practicing at Harvard, the MST Scale is made up of ten shades, defined by precise color codes, and intended to better represent the diversity of skin colors.

What will it be used for?

This sort of cue can be useful in computer vision, a field of artificial intelligence where computers are able to read and understand an image, much like the human eye.

The systems behind Google Images could thus be smarter using this tool, and better recognize images of people with darker skin.

This can for example improve the results of specific searches: "when you search for items related to makeup on Google Images, you will see an option to refine the results by skin tone", develops Tulsee Doshi, product manager at Google, in a communicated.

Now for makeup queries in Google Images, like “everyday eyeshadow” and “bridal makeup looks,” you'll find a new way to filter by relevant skin tones to find more helpful results.

#GoogleIO pic.twitter.com/Y8t0oXtoqv

— Google (@Google) May 11, 2022

In more general searches, the image results could be more diverse, which would be more representative of the population.

In addition to Google Images, "we will also use the MST classification to improve Google Photos", Google's online storage and editing platform, adds Tulsee Doshi.

“We are launching a new series of Real Tone filters designed to work better on all skin tones.

Clearly: the editing software will better highlight darker skin with new settings.

Could this classification be used elsewhere than at Google?

Yes.

The company has chosen to share it in open source (in “open source”, in French), which will make it usable by other systems using artificial intelligence.

This classification could thus improve a whole bunch of systems that poorly recognize people with darker skin, such as facial recognition programs.

Read alsoGafam regulation: 5 minutes to understand the historical scope of the Digital Markets Act

But there are applications in which this classification could prove inadequate, such as "skin cancer risk assessment", where artificial intelligence can perform diagnoses, confide the creators of the MST Scale on their dedicated site.

In dermatology or in clinical contexts, “the MST would have to be validated” by studies, which has not yet been done.