The online network X has provoked major outrage after users utilized its AI chatbot Grok to alter photos of women by taking off their garments.
Several examples have surfaced where the technology has been employed to digitally undress women to render them in bikinis without consent, as well as to place them in inappropriate scenarios.
Developer Reaction and Regulatory Stance
XAI, the organization that created Grok, offered no comment to requests for comment, aside from an standard response stating "legacy media lies".
A government official stated that legislation is being proposed to prohibit image stripping tools. Under a new criminal offence, anyone who supplied such tools would "face a prison sentence and heavy penalties".
"X or Grok has the ability to stop these forms of misuse if they wanted to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "assess the potential" of people viewing illegal content on their networks, but stopped short of confirming whether it was presently examining X or Grok in concerning AI-generated pictures.
Grok is a no-cost AI assistant—with some premium features—which reacts to X users' queries when they tag it in a tweet.
It is frequently used to provide commentary or additional background to other posters' remarks, but people on X are also able to modify an posted image through its AI image editing feature.
History of Controversy and Policy
The tool has been faulted for enabling users to generate photos and clips with explicit content and sexualised material, and it was earlier alleged to have been involved in making sexually explicit content of public figures.
"The platform has been permitting the production and spread of these pictures for months without taking any measures and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own terms of service forbids "showing likenesses of persons in a pornographic fashion".
In a statement, Ofcom said it was unlawful to "create or circulate non-consensual intimate images or illegal content" and stated this covered sexual deepfakes created with AI.
It said services such as X were required to take "appropriate actions" to "reduce the risk" of users coming across illegal content on their services and to remove it promptly when they become aware of it.
