California’s top prosecutor has launched an investigation into the spread of erotic AI deepfakes generated by Elon Musk’s AI model Grok.
Attorney General Rob Bonta said in a statement announcing the investigation: “The flood of reports in recent weeks detailing non-consensual, sexually explicit material produced by XAI and posted online is shocking.”
XAI, which develops Grok, has previously said that “anyone using Grok or inciting to create illegal content will face the same consequences as those who upload illegal content”.
The California investigation came as British Prime Minister Sir Keir Starmer warned of possible action against Ax.
In Wednesday’s statement, Bonta said: “This content, which depicts women and children naked and in sexually explicit situations, has been used to harass people on the internet.”
Democratic prosecutors urged XAI to take immediate action.
California Governor Gavin Newsom, a Democrat, posted to
The BBC has contacted xAI for comment.
On Wednesday, Musk posted to X that he was “not aware of any nude underage images made by Grok. Literally zero.”
“Apparently, Grok does not generate images spontaneously,” Musk wrote. “It only does so as per user request.”
The tech billionaire, a Republican mega-donor, has also said that critics of X were politically motivated and were using the Grok controversy as an “excuse for censorship.”
In November, Wired magazine reported that tools from other AI companies like OpenAI and Google have also been used to digitally undress people.
Last week, three US Democratic senators asked Apple and Google to remove X and Grok from their app stores.
Within hours of the request, X restricted its image creation tool so that it was only available to paying customers.
X and Grok will be available on Apple’s App Store and Google Play.
It comes amid a debate over whether US tech companies are shielded from liability for what users post on AI platforms.
Section 230 of the Communications Decency Act of 1996 provides legal immunity to online platforms for user-generated content.
But Cornell University professor James Grimmelman argues that the law “only protects sites from liability for third-party content from users, not content produced by the sites themselves”.
Grimmelman said XAI is trying to avoid blame imagery to users, but expressed skepticism that this argument would hold up in court.
“This is not a case where users are creating the photos themselves and then sharing them on X,” he said.
In this case, he said, “XAI itself is making the photographs. This is outside the applicability of Section 230.”
Oregon Senator Ron Wyden has argued that Section 230, which he co-authored, does not apply to AI-generated images. He said that companies should be held fully responsible for such content.
“I’m glad to see states like California stepping up to investigate Elon Musk’s horrific child sexual abuse material generator,” Wyden told the BBC on Wednesday.
Wyden is one of three Democratic senators who asked Apple and Google to remove X and Grok from their app stores.
The announcement of the investigation in California comes as Britain prepares legislation that would make it illegal to take non-consensual intimate photographs.
UK watchdog Ofcom has also launched an investigation into Grok.
If it is determined that the platform broke the law, it could be fined 10% of its worldwide revenue or £18m, whichever is greater.
On Monday, Sir Keir Starmer told Labor MPs that Musk’s social media platform
