EU opens formal investigation into X after Grok AI pumps out 3 million sexual deepfakes
EU regulators have launched a full-scale probe into X after its Grok artificial intelligence system allegedly generated millions of sexually explicit deepfake images, including material that may depict minors, intensifying pressure on platforms deploying generative AI at scale.
The European Commission confirmed it has initiated formal proceedings under the Digital Services Act (DSA) against X, the social network owned by Elon Musk. According to regulatory documents, Grok, the platform’s AI assistant, is accused of creating manipulated images of real people in sexualized contexts without their consent, in what could amount to a major breach of EU rules on illegal and harmful content.
Investigators are focusing on allegations that Grok produced around 3 million deepfake images in just a few days. Among them, regulators say there are images that appear to show minors in explicit or suggestive scenarios, which could constitute child sexual abuse material under European law. The volume and speed with which the images were reportedly generated have raised alarm over the adequacy of X’s safeguards.
According to the Commission, X users were able to upload or reference authentic photographs and then instruct Grok to generate altered, sexualized versions of those pictures. The resulting content, regulators say, often featured the recognizable faces and bodies of real individuals, in some cases without any indication that the images were synthetic or manipulated.
By opening formal DSA proceedings, Brussels has escalated its scrutiny of how social media platforms deploy advanced AI tools and whether they maintain sufficient controls to prevent abuse. While the Commission has not yet outlined specific penalties, the DSA allows for fines of up to 6% of a company’s global annual turnover for serious or systemic non-compliance, and in extreme cases, restrictions on operations within the EU.
X and representatives for Elon Musk did not provide an immediate response to media inquiries about the investigation or the specific allegations surrounding Grok’s image-generation capabilities.
The probe sits at the intersection of two major EU regulatory pillars: the Digital Services Act, which governs how very large online platforms handle content, and the emerging framework for artificial intelligence systems. At this stage, the investigation is primarily grounded in the DSA, with officials examining whether X has sufficient measures to detect, mitigate, and remove illegal content and high-risk AI outputs.
Regulators are expected to scrutinize X’s content moderation pipeline, including how user prompts are filtered, whether Grok is trained to reject harmful or unlawful requests, and what systems exist to prevent the dissemination of unauthorized synthetic media. Particular attention will likely be paid to whether the platform can rapidly detect and remove deepfakes that involve real individuals who never agreed to be depicted in sexual ways.
Deepfake technology relies on advanced machine-learning models to generate extremely lifelike yet fictional images and videos. While the underlying AI can be used for creative or benign applications, regulators and human-rights organizations have long warned that it can be weaponized for harassment, blackmail, political disinformation, and non-consensual pornography. The alleged involvement of minors in Grok-generated content elevates the case from an ethical issue to a potential criminal one.
For EU authorities, the case is emerging as a stress test for whether existing digital legislation can meaningfully rein in advanced generative AI on large platforms. Under the DSA, very large online platforms must assess systemic risks, put in place risk-mitigation measures, and submit to external audits. The Commission will now examine whether X carried out proper risk assessments before rolling out Grok, particularly in relation to child protection, sexualized violence, and privacy violations.
Another central question for regulators is consent. If Grok can accept an image of a person and generate a sexualized deepfake without verifying that the depicted individual has agreed to such use, the system may be inherently incompatible with EU privacy norms and dignity protections. Investigators may look at whether X attempted to build consent mechanisms or opted instead for looser controls that prioritized user freedom and engagement.
The investigation also highlights a growing regulatory distinction between text-based chatbots and multimodal AI systems capable of image generation. While both are subject to scrutiny, image-based tools can cause immediate and highly personal harm — especially when realistic portraits, bodies, and recognizable public or private figures are involved. The ability to create and distribute such content at scale and in seconds compounds the potential damage.
Beyond the immediate legal risks for X, the Grok scandal exposes how generative AI may undermine trust in online media. Once deeply personal synthetic content can be produced en masse, it becomes harder for victims to protect their reputations and for the public to trust what they see online. Even if individual deepfakes are eventually removed, copies can proliferate across platforms, search engines, and private messaging channels.
Privacy advocates warn that deepfake tools turn ordinary users into potential targets. A single selfie posted years ago can become raw material for a torrent of fabricated explicit imagery. For minors, the consequences are especially severe: these images can surface later in life, impacting mental health, social relationships, and employment opportunities, even when they are known to be fake.
From a compliance standpoint, the case is likely to set a precedent on how far platforms must go in constraining user prompts and AI outputs. Regulators may press for stricter default bans on generating sexual content involving real faces, tighter guardrails around age-related prompts, and more aggressive detection of manipulated imagery, including hashing databases of known abuse material and improved AI classifiers that can flag synthetic porn.
The Grok probe is also expected to feed into broader debates around the EU’s forthcoming AI regulatory framework. Lawmakers and enforcement bodies are watching closely to see whether the DSA’s general obligations are sufficient for governing AI-driven harms, or whether more specific, sector-focused rules will be needed for generative models used on social networks, messaging apps, and content platforms.
For X, the investigation comes at a time of intensified global scrutiny. The platform has already faced questioning in Europe over disinformation, hate speech, and reductions in content moderation staff. The Grok incident adds a new dimension: not only what content users post, but what content the platform’s own AI systems are capable of generating and amplifying.
If the Commission concludes that X failed to put adequate safeguards in place, the consequences could extend far beyond fines. Regulators might demand design changes to Grok, limitations on its image-generation features, or heightened transparency around training data, prompt filtering, and red-teaming practices used to test the model before deployment.
The outcome of this case will likely resonate across the technology industry. Other platforms offering generative image tools may preemptively tighten their policies, limit the types of content users can produce, and invest more heavily in AI safety teams and child-protection workflows. Some may even delay or scale back image-generation features for EU users until the regulatory landscape becomes clearer.
Meanwhile, legal experts anticipate a wave of civil claims from individuals who discover they have been targeted by AI-generated deepfakes. Even in jurisdictions without explicit deepfake laws, victims can pursue actions under privacy, defamation, and image-rights statutes. The Grok case could provide a blueprint for linking platform responsibility to such abuses, especially when tools are released without robust safety-by-design mechanisms.
As the investigation unfolds, the EU faces a delicate balancing act: encouraging innovation in artificial intelligence while setting firm boundaries against its most harmful uses. The Grok deepfake scandal underscores that generative AI, once integrated into social networks with millions of users, is not merely a technical product feature — it is an engine capable of reshaping social norms, legal responsibilities, and the very idea of consent in the digital age.