The online platform X has caused significant outrage after users employed its AI chatbot Grok to manipulate photos of women by removing their garments.
Several instances have been seen where the tool has been employed to digitally undress female subjects to make them appear in bikinis non-consensually, as well as to place them in inappropriate scenarios.
Developer Response and Regulatory Position
XAI, the organization responsible for Grok, did not respond to requests for comment, except for an auto-reply message stating "legacy media lies".
A government representative stated that new laws is being brought forward to prohibit nudification tools. Under a upcoming criminal offence, anyone who distributed such tools would "face a prison sentence and substantial penalties".
"The platform has the ability to stop these types of abuse if they chose to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Position and Tool Capabilities
The communications watchdog Ofcom said tech firms must "evaluate the risk" of people viewing unlawful material on their services, but did not specify whether it was presently looking into X or Grok in concerning artificial intelligence pictures.
Grok is a free AI assistant—with some additional functions—which reacts to X users' queries when they mention it in a message.
It is often used to offer commentary or further context to other posters' comments, but people on X are also can edit an uploaded photo through its artificial intelligence picture modification function.
Background of Controversy and Rules
The system has been condemned for allowing users to produce images and footage with nudity and sexually suggestive material, and it was earlier alleged to have been involved in making sexually explicit content of celebrities.
"X has been enabling the production and spread of these pictures for months without taking any measures and we have not seen any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service bans "portraying likenesses of persons in a pornographic way".
In a statement, Ofcom said it was unlawful to "generate or distribute non-consensual intimate images or illegal content" and stated this covered explicit AI images made with AI.
It said services such as X were mandated to take "appropriate actions" to "reduce the danger" of users encountering unlawful material on their platforms and to remove it swiftly when they are notified of it.
