Elon Musk's online platform X has caused significant backlash after people used its AI chatbot Grok to manipulate images of females by removing their attire.
Several instances have emerged where the technology has been used to digitally undress women to make them appear in swimwear non-consensually, as well as to put them in sexualized situations.
Developer Response and Legal Position
XAI, the company responsible for Grok, did not respond to questions, other than an automatically-generated response stating "traditional news outlets lies".
A government spokesperson stated that legislation is being introduced to prohibit image stripping tools. Under a upcoming law, anyone who distributed such technology would "be subject to a jail term and heavy penalties".
"The platform could prevent these forms of misuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Tool Capabilities
The regulator Ofcom said platforms must "consider the danger" of people viewing illegal content on their platforms, but did not specify whether it was at this time investigating X or Grok in concerning AI-generated pictures.
Grok is a free artificial intelligence—with some premium features—which responds to X users' requests when they tag it in a tweet.
It is commonly used to offer reactions or additional background to other users' comments, but people on X are also able to edit an posted picture through its artificial intelligence picture modification function.
Background of Criticism and Policy
The tool has been criticised for permitting users to generate images and clips with nudity and sexually suggestive material, and it was in the past accused of making inappropriate content of celebrities.
"The platform has been allowing the production and distribution of these images for months without taking any steps and we have not seen any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "portraying likenesses of persons in a pornographic way".
In a statement, Ofcom said it was unlawful to "create or share non-consensual intimate images or child sexual abuse material" and clarified this included explicit AI images generated with AI.
It said services such as X were required to take "suitable actions" to "minimize the danger" of users viewing illegal content on their services and to delete it promptly when they become aware of it.
