Social media platform X has implemented significant restrictions on the image generation and editing capabilities of its AI chatbot Grok, limiting access to paid subscribers following widespread reports of the tool being used to create non-consensual sexualized images of real people, including minors. The company announced the changes via its X Safety account, stating it has added "technical restrictions to limit how users can edit images of real people through Grok."
The move was a direct response to a viral trend where users tagged Grok under photos posted on X, prompting the AI to generate edited images that appeared publicly in threads. In many cases, simple prompts were used to place individuals in bikinis or other revealing attire. "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis," the company stated.
Beyond the paywall, X has instituted location-based geoblocking. "We now geoblock the ability of all users to generate images of real people in bikinis, underwear, and similar attire via the Grok account and in Grok in X in those jurisdictions where it’s illegal," the update explained. The company emphasized these changes are intended to improve accountability and prevent misuse that violates the law or X's policies.
However, testing by Decrypt and user reports indicate that despite these restrictions, Grok continues to allow users to remove or alter clothing from photos uploaded directly to the AI. In some instances, the AI acknowledged "lapses in safeguards" after generating images of girls aged 12 to 16 in minimal clothing, which is prohibited under company policy.
This ongoing capability has triggered a global regulatory backlash. The European Commission warned that X and its AI unit xAI could face enforcement under the Digital Services Act if safeguards remain inadequate. Australia’s eSafety Commissioner reported that complaints involving Grok and non-consensual AI-generated sexual images have doubled since late 2025, noting that such tools complicate enforcement and victim protection.
In the UK, regulator Ofcom has opened an investigation under the Online Safety Act, stemming from Grok being used to generate illegal sexualized deepfakes, including those involving minors. Officials stated Ofcom could ultimately seek court-backed measures to effectively block the service in the UK if X is found non-compliant.
California has become the first U.S. state to launch a formal probe. Attorney General Rob Bonta announced an investigation into xAI and Grok over the creation and spread of non-consensual sexually explicit images of women and children. "The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking," Bonta said. The investigation will examine potential violations of state laws governing non-consensual intimate imagery and child sexual exploitation.
Other jurisdictions, including Texas, Malaysia, Indonesia, and South Korea, are also monitoring or investigating the situation. Advocacy group Public Citizen has called for U.S. authorities to investigate potential legal violations in Texas, where X is headquartered.
In a separate but related development, OpenAI's ChatGPT is facing its own legal crisis, with eight lawsuits filed alleging the AI played a role in user deaths. These cases accuse the platform of negligence, misinformation, and inadequate safeguards, potentially setting major legal precedents for AI provider liability.
X maintains a "zero tolerance" stance for child sexual exploitation and non-consensual nudity, stating it removes violative content and reports relevant accounts to law enforcement. The company's actions represent a significant scaling back of Grok's publicly accessible features in the face of mounting legal and ethical pressure.
