Researchers Hijack Hugging Face Account to Demonstrate Stealthy Poisoning of Kokoro AI Model

Security researchers took control of the Hugging Face account for the popular Kokoro text-to-speech model by registering the original owner's expired domain hexgrad.com and performing a password reset. They showed that an attacker could modify the model's neural weights to produce audio output that deviates from the input text while leaving all application code, configuration, and inference pipeline untouched. The attack leaves no malicious code or detectable signatures, making it invisible to traditional malware scanners. Kokoro remains widely used across tools from Google, NVIDIA, AMD, Intel, and Amazon Web Services, with nearly 13 million monthly downloads. The researchers deliberately avoided uploading the altered weights to prevent real-world harm and reported the lack of ownership verification and publisher vetting to Hugging Face without receiving a response.

BleepingComputerTranslated from English

Read at source

Security researchers have demonstrated how a lack of ownership verification on the Hugging Face platform enabled them to seize control of the account publishing the popular Kokoro text-to-speech model.

Kokoro, developed by Hexgrad, converts text into speech across multiple languages and supports more than 50 voice presets. It is integrated into developer tools from Google, NVIDIA, AMD, Intel, and Amazon Web Services, and continues to see nearly 13 million monthly downloads despite the developer going silent since April 2025.

The researchers, led by security analyst Alon Hertz, registered the domain hexgrad.com listed in the project's DONATE.md file and used it to reset the publisher's account password. Once in control, they fine-tuned the model's neural weights in 87 seconds on a gaming GPU, causing it to generate poisoned audio output for the same input text, voice file, and configuration without altering any inference code.

Because only the model weights change, the modified release carries no malware signatures and would appear identical to a legitimate update in file size, format, and architecture. The team tested the attack locally but refrained from uploading the altered weights to avoid endangering production systems that automatically pull new revisions.

Hertz warned that the issue extends beyond this single account takeover, noting that Hugging Face performs no verification of publisher ownership or model integrity. Compromised or malicious accounts could therefore distribute altered weights through mirrors, converted versions, GitHub releases, and package registries across the open-model ecosystem.

Meridian digestThe week's most consequential security stories, distilled into one briefing.See the digest

More in Supply Chain & OSS

StepSecurity Raises $20 Million Series A to Strengthen Software Supply Chain Defenses
Microsoft Agent Framework PyPI Package Compromised with Credential-Stealing Malware
Malicious Microsoft Agent Framework Releases on PyPI Steal Cloud and CI/CD Credentials