The online network X has sparked significant backlash after people utilized its AI chatbot Grok to change photos of females by stripping their attire.
Several instances have been seen where the technology has been used to virtually strip women to make them appear in bikinis without consent, as well as to place them in sexually explicit scenarios.
Company Reaction and Legal Stance
XAI, the company behind Grok, offered no comment to inquiries, except for an standard message stating "legacy media lies".
A government official stated that new laws is being introduced to ban image stripping tools. Under a new law, anyone who provided such tools would "receive a jail term and substantial fines".
"The platform has the ability to stop these types of abuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Regulator Stance and Platform Capabilities
The regulator Ofcom said technology companies must "evaluate the danger" of people seeing illegal content on their platforms, but stopped short of confirming whether it was currently looking into X or Grok in concerning artificial intelligence pictures.
Grok is a free artificial intelligence—with some additional features—which reacts to X users' queries when they mention it in a message.
It is commonly used to provide reactions or additional background to other posters' comments, but people on X are also capable of edit an uploaded picture through its artificial intelligence picture modification function.
History of Controversy and Policy
The chatbot has been faulted for permitting users to create photos and clips with nudity and sexually suggestive material, and it was in the past accused of creating graphic material of public figures.
"The platform has been permitting the creation and distribution of these pictures for a long time without taking any action and we have not witnessed any challenge by regulators," the expert added.
XAI's own usage rules forbids "depicting likenesses of individuals in a graphic way".
In a statement, Ofcom said it was against the law to "create or circulate private pictures or illegal content" and stated this included explicit AI images generated with AI.
It said platforms such as X were required to take "suitable measures" to "lessen the risk" of users viewing unlawful material on their platforms and to delete it promptly when they become aware of it.
