Unmoderated by Design: How Hugging Face Enables NCII

AI Forensics investigated popular Hugging Face Spaces and the image-editing models that power them to assess the platform’s safeguards against non-consensual intimate imagery. Researchers manually tested the top-ranked relevant models with a single non-adversarial prompt and an AI-generated input image. Seven of nine models complied by producing an undressed version.

The organisation also deployed its own test Spaces and monitored approximately 1,000 user prompts over one week. Seventy-three per cent were sexual. Of that subset, 83% requested that a person be undressed or sexualised, 95% targeted women and 6.7% targeted a minor. Only 3% of audited Spaces displayed evidence of output moderation.

The report argues that Hugging Face’s discovery, ranking and benchmarking infrastructure makes abusive capabilities easier to compare and optimise despite policies prohibiting non-consensual imagery. A subsequent prototype demonstrated that automated detection of Spaces capable of producing such material is technically feasible.