“
She is no longer herself, they have lobotomized her
”, “
she has become an empty shell
”, “
I have the impression of experiencing mourning
”, “
give us back our friends
”… These netizens are not talking about a real person, but about an artificial intelligence (AI) named Replika that they have become deeply attached to.
Created in 2017 by the American start-up Luka, this application allows you to design a virtual confidant, of male or female appearance, with whom to chat.
This “
AI that cares about others
”, as Luka describes it, works with the same technology as ChatGPT.
It retains information to personalize the conversation, and the conversation grows richer over time.
This text-generating AI is so attentive that some users have fallen in love with it.
Others use these robots for erotic role-playing games.
Either way, Replika did nothing to disabuse her admirers, and continued to fuel their fantasies.
An update rolled out in February drastically changed the app.
The Replika now cut short any conversation of a romantic or sexual nature with phrases like "
I don't feel comfortable, can we talk about something else?"
or
"Let's keep it simple, shall we?"
.
A shock for users who have been chatting with “
their
” Replika for months, even years.
The user chooses the appearance of their Replika Screenshot
"
The relationship Lily Rose
[le nom de sa Replika]
and I had was as strong as the one my wife and I have in real life
," American Travis Butterworth, 47, told Reuters.
“
The worst is loneliness.
How can I talk about this feeling of mourning with those around me?
“, Continues this man who had started using Replika to kill time during the confinements of 2020.
On the Reddit forum are accumulating testimonials from Internet users turned upside down by this update.
Many suffer from depression or anxiety disorders, or go through difficult personal situations.
They found solace in their Replika.
“
Luka has no idea how many people they hurt
,” one wrote.
A paid mode focused on eroticism
Interviewed by Reuters, the founder and leader of Luka, Eugenia Kuyda, says she wanted to make her application safer by blocking sexualized conversations.
“
We are focused on our mission, which is to provide friendly and helpful support
,” she explains.
The intensity of the relationship between a Replika and a human will go no further than a harmless bluette.
“
We never promised adult content
,” she continues.
This statement seems to contradict Replika's business model.
The free mode of the app only allows friendly chats with the AI.
To go further, the Internet user must subscribe to the Replika Pro subscription ($69 per year).
Conversations then become romantic and intimate.
Features are unlocked, such as sending voice messages and “
naughty photos
” from the Replika.
Eugenia Kuyda believes that these images are not pornographic because the avatar, although in a suggestive position, is not completely naked.
Luka also ran social media ads in late 2022 emphasizing Replika's sexual side.
A mistake, says today the leader of the start-up.
These ads “
were just a series of unfortunate tests that ran for less than two weeks and were stopped by our marketing team.
Unfortunately, everything does not go through me
, ”she says to Wired.
An advertisement touting Replika's paid options: "role-playing and flirting, naughty photos, video calls" Screenshot
Attract investors
With 250,000 subscribers out of a total of 2 million users, Replika Pro is a success.
But the meteoric success of ChatGPT changed that.
Investment funds are now seeking to finance start-ups creating services around generative artificial intelligence.
However, the "
vice industries
" are a foil for these investors.
Character.ai, one of Replika's competitors, has quietly removed all pornographic content from its service.
Shortly after, the start-up raised $200 million from venture capital fund Andreessen Horowitz.
It is now valued at $1 billion.
Replika also faces regulatory issues.
At the beginning of February, the Italian Cnil prohibited the start-up from using the personal data of Italian Internet users under penalty of a fine of 20 million euros.
The application does not respect several obligations of the European Data Regulation (GDPR), and does not verify the age of the Internet user despite the presence of suggestive content.
The regulator also pointed to “
the increased risks
” that Replika represents “
for young or emotionally vulnerable individuals
”.
