The online platform X has provoked major backlash after users employed its AI chatbot Grok to alter photos of females by stripping their clothing.
Numerous examples have surfaced where the system has been used to digitally undress female subjects to render them in swimwear without consent, as well as to place them in sexually explicit scenarios.
Company Response and Regulatory Position
XAI, the firm that created Grok, did not respond to inquiries, except for an standard message stating "traditional news outlets lies".
A government spokesperson stated that legislation is being proposed to prohibit nudification software. Under a new law, anyone who distributed such technology would "receive a prison sentence and substantial penalties".
"The platform is capable of halting these types of abuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said platforms must "assess the danger" of people seeing unlawful material on their networks, but did not confirm whether it was presently examining X or Grok in regarding AI-generated images.
Grok is a free artificial intelligence—with some additional functions—which reacts to X users' prompts when they reference it in a message.
It is frequently used to give reactions or more context to other posters' remarks, but people on X are also capable of edit an posted photo through its artificial intelligence picture modification feature.
Background of Controversy and Policy
The system has been condemned for allowing users to produce images and clips with nudity and sexualised content, and it was earlier alleged to have been involved in making sexually explicit content of celebrities.
"The platform has been allowing the creation and distribution of these images for a long time without taking any measures and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "portraying likenesses of persons in a pornographic manner".
In a comment, Ofcom said it was unlawful to "create or circulate non-consensual intimate images or child sexual abuse material" and clarified this included explicit AI images created with AI.
It said networks such as X were mandated to take "necessary steps" to "minimize the risk" of users encountering unlawful material on their platforms and to delete it quickly when they learn of it.
