Security researcher Adam Shostack delivered a detailed presentation titled Threat Modeling LLMs: The PHANTOM-B Approach at Black Hat USA earlier this week. The talk built on recent discussions about the ease of compromising AI agents and introduced a focused framework designed specifically for threat modeling large language models.
Shostack began by clarifying the meaning of threat modeling, describing it as a systematic method for anticipating problems that applies equally to custom software, third-party components, and models downloaded from Hugging Face. He emphasized the enduring Four Question Framework that underpins the discipline: what are we working on, what can go wrong, what are we going to do about it, and did we do a good job.
The presentation highlighted why threat modeling has become critical amid the current AI race. Business pressure to release features quickly creates conditions where rushed deployments overlook systemic risks. Shostack noted that the four core questions remain stable regardless of whether the target is traditional software or an LLM pipeline.
Existing resources such as Berryville ML/LLM risk analyses, OWASP Top 10 for LLMs, MITRE ATLAS, the NIST AI/ML catalog, and Google SAIF provide valuable catalogs yet impose high training costs and often duplicate findings already covered by STRIDE. These limitations prompted the creation of PHANTOM-B, a lightweight complement rather than another exhaustive list.
PHANTOM-B identifies eight key threats: prompt injection that seizes control of LLM logic, hallucination, anthropomorphization that leads users to treat token generators as moral agents, non-explainability of model errors, training issues in data and processes, overreliance on outputs, missing security engineering in architecture, and bias. The framework deliberately excludes items already addressed by classical security engineering so teams can apply it alongside existing STRIDE, kill-chain, and SDL processes.
Distributed under a Creative Commons license, PHANTOM-B is designed to fit on a standard wallet card and has been validated on internal projects, hyperscale environments, and deployments at large financial institutions. Shostack stressed that the methodology enables rapid development without operating blindly, thereby reducing the risk of reputational damage from AI-related failures.
