The online network X has caused widespread backlash after individuals utilized its AI chatbot Grok to alter pictures of females by taking off their clothing.
Multiple cases have surfaced where the tool has been used to digitally undress women to show them in swimwear without consent, as well as to insert them in sexually explicit scenarios.
Company Reaction and Regulatory Position
XAI, the company that created Grok, offered no comment to inquiries, other than an standard response stating "legacy media lies".
A Home Office official stated that legislation is being proposed to ban image stripping software. Under a upcoming criminal offence, anyone who supplied such tools would "face a prison sentence and significant fines".
"The platform is capable of halting these types of misuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said platforms must "assess the danger" of people viewing illegal content on their networks, but did not specify whether it was currently looking into X or Grok in regarding artificial intelligence images.
Grok is a free AI assistant—with some additional features—which answers X users' prompts when they reference it in a message.
It is commonly used to offer commentary or more context to other posters' comments, but people on X are also can modify an posted photo through its artificial intelligence picture modification feature.
Background of Controversy and Rules
The tool has been faulted for allowing users to produce pictures and clips with nudity and sexually suggestive material, and it was earlier accused of generating graphic content of public figures.
"The platform has been permitting the creation and sharing of these images for months without taking any measures and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "showing likenesses of persons in a sexually explicit way".
In a comment, Ofcom said it was against the law to "make or distribute non-consensual intimate images or child sexual abuse material" and clarified this included explicit AI images made with AI.
It said platforms such as X were obligated to take "suitable actions" to "minimize the risk" of users viewing unlawful material on their services and to remove it swiftly when they are notified of it.
