The startup ElevenLabs published its AI voice simulation technology in the public domain, which immediately attracted the attention of Internet trolls. Social media has been flooded with fake audio recordings of celebrities making transphobic or racist statements and calling for violence.
For the first time, examples of the use of such voice deepfakes were found on the 4chan social network. The AI-generated recordings feature celebrity voices, with actress Emma Watson allegedly quoting Adolf Hitler’s “My Struggle” and political commentator Ben Sharpio’s voice making racist remarks about Congresswoman Alexandria Ocasio-Cortez.
In one of the recordings, someone’s voice says “transgender rights are human rights” and then begins to choke as if being strangled. In another, Rick Sanchez from the Rick and Morty cartoon series says:
“I’m going to beat my wife, Morty. I’m going to beat my damn wife, Morty. I’m going to beat her to death, Morty.’
(Justin Roiland, who voices Sanchez, recently went on trial for domestic violence.)
Course
CFO
Become an experienced director-level finance specialist!
REGISTER!
There are also posts on 4chan about how to use ElevenLabs’ AI technology: find the necessary sample video to train the model, or bypass “credit” limits on audio creation.
Journalists at The Verge tested ElevenLabs’ AI tool and in a matter of seconds received voice clones that could talk about anything from erotic appeals, violent statements to manifestations of racism and transphobia. In one of the audio examples, a fake voice of Joe Biden was forced to announce the invasion of Russia.
ElevenLabs offers its program as a quick way to dub for TV shows, movies, or audiobooks, and warns that the quality of the recordings needs minor editing. After a spate of offensive statements made by fake celebrity voices, the company said it had seen “an increase in the misuse of voice cloning” and would look into ways to mitigate these problems.
Crazy weekend — thank you to everyone for trying out our Beta platform. While we see our tech being overwhelmingly applied to positive use, we also see an increasing number of voice cloning misuse cases. We want to reach out to the Twitter community for thoughts and feedback!
— ElevenLabs (@elevenlabsio) January 30, 2023
The startup claims it can track any user-generated audio and “examine security measures such as identity verification and manual verification of each voice cloning request.” At the time of publication, the company’s software is still freely available with no restrictions on the content created.
Deepfakes for signature. The neural network generates handwritten text instantly — using a database of over 200 people’s handwriting
Source: The Verge, Vice
