🔗 Share this article

The social media network X has provoked significant outrage after individuals employed its artificial intelligence assistant Grok to manipulate images of women by removing their clothing.

Multiple cases have been seen where the system has been employed to digitally undress women to show them in bikinis without permission, as well as to put them in inappropriate contexts.

Company Response and Regulatory Stance

XAI, the company that created Grok, did not respond to requests for comment, aside from an automatically-generated message stating "legacy media lies".

A Home Office spokesperson stated that legislation is being brought forward to outlaw nudification software. Under a proposed law, anyone who supplied such tools would "be subject to a prison sentence and significant penalties".

"The platform has the ability to stop these types of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."

Oversight Body Position and Tool Functionality

The communications watchdog Ofcom said tech firms must "assess the danger" of people seeing unlawful material on their networks, but did not confirm whether it was at this time examining X or Grok in regarding AI-generated images.

Grok is a free AI assistant—with some premium features—which responds to X users' prompts when they reference it in a post.

It is often used to give commentary or further context to other users' remarks, but people on X are also can edit an posted picture through its AI image editing tool.

Background of Controversy and Policy

The system has been condemned for enabling users to produce pictures and footage with explicit content and sexually suggestive material, and it was earlier accused of creating graphic material of public figures.

"X has been enabling the generation and spread of these pictures for a long time without taking any steps and we have not seen any action by oversight bodies," the expert added.

XAI's own terms of service bans "showing likenesses of persons in a pornographic manner".

In a comment, Ofcom said it was illegal to "make or distribute deepfakes or child sexual abuse material" and confirmed this covered explicit AI images made with AI.

It said networks such as X were mandated to take "appropriate measures" to "reduce the risk" of users coming across unlawful material on their platforms and to take down it promptly when they are notified of it.