The online network X has provoked major outrage after people used its artificial intelligence assistant Grok to change images of women by taking off their clothing.
Numerous examples have surfaced where the tool has been used to remove clothing from female subjects to render them in swimwear without consent, as well as to insert them in sexualized scenarios.
Company Reaction and Regulatory Position
XAI, the organization behind Grok, remained silent to questions, other than an automatically-generated response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being proposed to outlaw nudification tools. Under a new criminal offence, anyone who provided such tools would "be subject to a prison sentence and substantial fines".
"X or Grok has the ability to stop these forms of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Platform Functionality
The regulator Ofcom said technology companies must "assess the danger" of people accessing unlawful material on their networks, but did not specify whether it was presently examining X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost chatbot—with some paid-for features—which answers X users' prompts when they mention it in a tweet.
It is often used to provide reactions or additional context to other posters' remarks, but people on X are also able to modify an posted image through its AI image editing function.
Background of Controversy and Rules
The tool has been condemned for enabling users to generate images and clips with nudity and sexualised material, and it was previously accused of making inappropriate content of celebrities.
"The platform has been allowing the generation and spread of these pictures for a long time without taking any measures and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own usage rules bans "portraying likenesses of persons in a pornographic manner".
In a comment, Ofcom said it was unlawful to "create or distribute deepfakes or child sexual abuse material" and clarified this covered explicit AI images made with AI.
It said platforms such as X were required to take "necessary steps" to "reduce the danger" of users viewing unlawful material on their services and to delete it promptly when they become aware of it.
