The social media network X has caused significant backlash after people employed its AI chatbot Grok to alter pictures of females by stripping their attire.
Several instances have been seen where the system has been employed to virtually strip female subjects to make them appear in bikinis without consent, as well as to put them in inappropriate situations.
Developer Reaction and Legal Position
XAI, the organization that created Grok, offered no comment to inquiries, aside from an auto-reply message stating "legacy media lies".
A government official stated that new laws is being brought forward to prohibit nudification software. Under a new criminal offence, anyone who distributed such tools would "face a prison sentence and significant fines".
"The platform is capable of halting these types of abuse if they chose to," said a law professor, adding they "seem to have impunity."
Oversight Body Stance and Platform Functionality
The regulator Ofcom said technology companies must "assess the risk" of people accessing unlawful material on their networks, but stopped short of confirming whether it was at this time investigating X or Grok in relation to AI-generated pictures.
Grok is a free AI assistant—with some premium features—which reacts to X users' prompts when they mention it in a message.
It is frequently used to offer commentary or further background to other posters' remarks, but people on X are also capable of edit an uploaded photo through its artificial intelligence picture modification function.
Background of Criticism and Policy
The tool has been criticised for permitting users to produce pictures and footage with explicit content and sexually suggestive material, and it was in the past alleged to have been involved in generating graphic content of celebrities.
"The platform has been enabling the generation and distribution of these images for months without taking any action and we have not seen any intervention by regulators," the expert added.
XAI's own acceptable use policy bans "portraying images of individuals in a pornographic fashion".
In a comment, Ofcom said it was illegal to "generate or distribute deepfakes or child sexual abuse material" and confirmed this included explicit AI images generated with AI.
It said networks such as X were required to take "necessary actions" to "lessen the danger" of users encountering illegal content on their platforms and to remove it swiftly when they learn of it.
