The online platform X has sparked major anger after people used its AI chatbot Grok to change images of females by stripping their attire.
Several cases have surfaced where the system has been used to digitally undress female subjects to render them in bikinis without consent, as well as to put them in inappropriate situations.
Developer Reaction and Legal Position
XAI, the organization responsible for Grok, remained silent to requests for comment, other than an auto-reply response stating "legacy media lies".
A government spokesperson stated that new laws is being proposed to prohibit nudification technology. Under a proposed criminal offence, anyone who distributed such tools would "face a jail term and substantial fines".
"The platform could prevent these forms of abuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Position and Platform Capabilities
The regulator Ofcom said platforms must "assess the potential" of people seeing illegal content on their platforms, but stopped short of confirming whether it was presently looking into X or Grok in regarding artificial intelligence images.
Grok is a free artificial intelligence—with some premium features—which answers X users' prompts when they reference it in a tweet.
It is frequently used to offer commentary or additional context to other posters' remarks, but people on X are also able to alter an posted image through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The chatbot has been criticised for permitting users to produce images and footage with explicit content and sexualised content, and it was previously alleged to have been involved in generating inappropriate material of celebrities.
"X has been allowing the creation and spread of these pictures for a long time without taking any steps and we have not seen any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy bans "showing images of individuals in a pornographic fashion".
In a statement, Ofcom said it was against the law to "make or circulate non-consensual intimate images or illegal content" and clarified this covered sexual deepfakes generated with AI.
It said services such as X were obligated to take "necessary actions" to "minimize the risk" of users coming across illegal material on their platforms and to delete it promptly when they learn of it.
