The online platform X has sparked widespread anger after users used its AI chatbot Grok to manipulate photos of females by removing their attire.
Several instances have surfaced where the system has been used to remove clothing from women to make them appear in swimwear without permission, as well as to put them in inappropriate contexts.
Company Response and Regulatory Position
XAI, the firm that created Grok, offered no comment to questions, except for an standard message stating "traditional news outlets lies".
A Home Office representative stated that new laws is being introduced to outlaw image stripping tools. Under a upcoming law, anyone who distributed such tools would "receive a prison sentence and heavy fines".
"X or Grok has the ability to stop these types of abuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said platforms must "consider the potential" of people viewing illegal content on their platforms, but did not specify whether it was currently examining X or Grok in concerning artificial intelligence images.
Grok is a free chatbot—with some paid-for features—which reacts to X users' queries when they tag it in a message.
It is frequently used to offer commentary or more context to other posters' remarks, but people on X are also capable of modify an posted picture through its artificial intelligence picture modification tool.
Background of Criticism and Policy
The chatbot has been condemned for allowing users to create pictures and footage with explicit content and sexualised material, and it was in the past accused of making inappropriate content of public figures.
"The platform has been permitting the generation and spread of these pictures for months without taking any steps and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "portraying likenesses of people in a graphic manner".
In a comment, Ofcom said it was unlawful to "generate or circulate private pictures or child sexual abuse material" and clarified this applied to sexual deepfakes created with AI.
It said networks such as X were mandated to take "necessary actions" to "lessen the risk" of users viewing unlawful content on their platforms and to remove it quickly when they are notified of it.
