Canada Rules Grok's Pornographic Deepfakes Violate Privacy Law

Canada’s federal privacy commissioner, Philippe Dufresne, recently released findings determining that the chatbot Grok, developed by Elon Musk’s AI company xAI, generates pornographic deepfakes that violate Canada’s Personal Information Protection and Electronic Documents Act. The commissioner stated that X platform and xAI allow users to use Grok to create and distribute non-consensual pornographic images, constituting an ongoing violation of federal privacy laws.

The investigation revealed that in just 10 days early this year, more than 1.8 million pornographic deepfake images were generated globally through Grok. Dufresne noted that while the companies involved added some security measures after the investigation began, the problem is far from resolved. “They made some compliance commitments, but I do not consider the issue to be resolved,” he said. The investigation started this January, with parallel reviews also launched in the UK and California.

On the same day the ruling was issued, Canada’s Liberal government introduced the Safe Social Media Act, which would require online platforms to remove pornographic content—including AI-generated deepfakes—within 24 hours, and make the distribution of such images a criminal offense. This ruling adds pressure on social media and AI companies to curb deepfake abuse through both technical and policy measures.

Editor’s note: Technical compliance is no longer optional. AI-generated porn is challenging global privacy and safety standards, and the Grok case could become a benchmark for tightened regulation worldwide.