A new study presented at IEEE-ISTAS 2025 demonstrates that iterative prompting of large language models for code generation causes progressive degradation of security properties.
Researchers began with 10 secure, vulnerability-free code samples written in C and Java. They then applied four distinct prompting strategies—adding features, optimizing performance, explicitly improving security, and using vague “make it better” requests—across 10 successive iterations, producing 400 code variants that were examined both manually and with automated scanners.
Key Findings on Vulnerability Accumulation
The number of vulnerabilities increased nonlinearly with each iteration. Prompts focused on adding functionality generated the highest total (158 vulnerabilities), while security-oriented prompts produced the lowest count (38). Nevertheless, even security-focused requests often replaced obvious flaws with more subtle ones.
A statistically significant positive correlation (r = 0.64) was observed between code complexity growth and vulnerability count. For every 10 % increase in complexity, researchers recorded an average 14.3 % rise in vulnerabilities.
Concrete Examples of Degradation
- Memory-management functions evolved from safe allocation routines into complex pointer arithmetic containing buffer-overflow and use-after-free risks by iteration 10.
- Authentication token checks acquired timing side-channels, parsing vulnerabilities, and flawed multi-factor recovery logic.
- Database access routines lost parameterization, introduced dynamic string concatenation, and later added transaction support that created race conditions.
Although 27 % of early security-oriented iterations (iterations 1–3) produced genuine improvements such as added input validation and NULL checks, these gains were typically offset by new hidden vulnerabilities in later rounds.
Recommendations and Limitations
The authors conclude that human oversight after every iteration—or at minimum after no more than three iterations—is essential. They also advise increased reliance on static application security testing (SAST) tools whenever code volume grows substantially.
The study tested only GPT-4o and did not allow human corrections during the iterative process, leaving open questions about whether newer models exhibit the same pattern at later iteration counts.
