Attackers move first. On average, exploitation now starts before a patch exists, according to Mandiant M-Trends 2026. In the first half of 2026, 23% of vulnerabilities VulnCheck added to its exploited list were hit on or before disclosure day.
AI pentesting vendors are not interchangeable. Most look convincing in a demo, yet gaps appear in logged-in flows, coverage depth, and production safety. The checklist requires vendors to prove capabilities on an organization's own web apps by scoring context, reach, validated findings, guardrails, and retests.
AI is speeding attackers up
For the first time, Google's Threat Intelligence Group identified a threat actor using a zero-day exploit believed to have been developed with AI: a 2FA bypass rooted in a logic flaw that conventional fuzzing and static analysis can easily miss. Agentic pentesting can help close that gap, and 87% of US security leaders surveyed are planning, piloting, or using it.
Every demo looks good. Your app is the real test.
Three agents may all demo well. The first finds real flaws but stops at the login page. The second navigates the app but cannot quantify coverage. The third exploits real issues yet lacks hard limits for production environments. A structured bake-off on the customer's own target reveals these differences before any contract is signed.
Ask what the agent knows before it attacks
Reconnaissance is real attack work. In CyCognito's model, recon and analysis consume 20 to 30 of the 63 to 95 hours typically required for a manual pentest. Some agents rebuild the map from a blank URL on every run, while others maintain an existing model and continue discovery beyond the initial map. A starting map should provide a head start, not impose a boundary.
Ask for proof, not suspicion
A suspicious pattern is not an exploitable vulnerability. The agent must exploit what it suspects, reproduce findings, and deliver verifiable evidence. During an assessment of a major insurer's production website, an AI pentesting agent discovered that the one-time passcode request could send login codes to any email address. It proved the flaw by logging in with a code sent to its own inbox before guardrails halted the test. The insurer had conducted a pentest on the same site approximately six months earlier, according to a Reflectiz case study.
Three questions that expose a weak AI pentesting vendor
The checklist provides 15 checks covering starting context, coverage proof, attack chaining, access across roles, data handling, guardrails, and retesting. Demos run on the vendor's target; bake-offs must run on the customer's target with identical scope, test accounts, and timeline. The scoring sheet weights safety and validated outcomes above speed.


