The online network X has caused widespread outrage after users employed its AI chatbot Grok to change images of females by stripping their garments.
Numerous examples have emerged where the tool has been used to remove clothing from female subjects to render them in swimwear non-consensually, as well as to put them in sexualized scenarios.
Developer Response and Regulatory Stance
XAI, the firm responsible for Grok, remained silent to questions, other than an standard response stating "traditional news outlets lies".
A Home Office official stated that new laws is being proposed to outlaw image stripping technology. Under a proposed criminal offence, anyone who distributed such technology would "be subject to a jail term and heavy fines".
"X or Grok is capable of halting these types of misuse if they chose to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said platforms must "assess the potential" of people viewing unlawful material on their networks, but did not confirm whether it was presently looking into X or Grok in regarding AI-generated images.
Grok is a no-cost AI assistant—with some premium functions—which reacts to X users' requests when they mention it in a message.
It is commonly used to offer reactions or more background to other users' remarks, but people on X are also able to modify an uploaded photo through its AI image editing tool.
History of Controversy and Policy
The tool has been criticised for allowing users to produce pictures and clips with nudity and sexualised material, and it was earlier accused of creating inappropriate content of celebrities.
"The platform has been enabling the creation and distribution of these pictures for months without taking any action and we have yet to see any challenge by regulators," the expert added.
XAI's own usage rules forbids "portraying likenesses of people in a pornographic way".
In a comment, Ofcom said it was against the law to "make or distribute private pictures or child sexual abuse material" and stated this included sexual deepfakes made with AI.
It said services such as X were required to take "appropriate measures" to "lessen the risk" of users encountering unlawful material on their services and to delete it quickly when they learn of it.
