MeridianMeridian

AI Agent Failures Usually Trace Back to Instruction Defects, Not Model Limitations

After a full year of working with AI agents in production workflows, the author stopped blaming models for apparent stupidity or hallucinations. The vast majority of such issues stem from three specific defects in the instructions provided to the agent. Rules written in ordinary prose often fail to enforce precise behavior. Instructions phrased as "how not to" create ambiguity instead of clear constraints. Finally, rules without built-in verification mechanisms allow errors to propagate unchecked. The piece emphasizes that diagnosing instruction quality is far more productive than assuming model degradation. This observation applies across programming, DevOps, analytics, and information security tasks where AI agents are deployed on continuous streams of work.

HabrTranslated from Russian

Read at source

After a year of running AI agents on live production workflows, one practitioner has stopped using the phrase "the model got dumber." Most cases that appear to be model stupidity or hallucinations actually result from three recurring defects in the instructions given to the agent.

The first defect occurs when a rule is written in ordinary prose rather than as an explicit, machine-actionable constraint. The second defect appears when instructions are framed negatively, telling the agent what not to do instead of defining the required behavior. The third defect arises when a rule lacks any verification step, allowing the agent to proceed without checking compliance.

The author notes that these patterns emerged consistently across tasks in programming, DevOps, analytics, and information security. By systematically rewriting instructions to eliminate prose form, negative phrasing, and missing checks, the frequency of apparent model failures dropped sharply. The conclusion is that "agent is dumb" is usually a diagnosis of the prompt, not the underlying model.

ShareXFacebookLinkedInReddit
Meridian digestThe week's most consequential security stories, distilled into one briefing.See the digest

More in AI Security

Logical Argument Claims Full Alignment of Superintelligent AGI Is Fundamentally Impossible
AI Security

Logical Argument Claims Full Alignment of Superintelligent AGI Is Fundamentally Impossible

A detailed analysis argues that the AI alignment problem cannot be solved for systems exceeding human cognitive capabilities. The paper presents a formal proof showing that any control mechanism created by humans will inevitably be circumvented by a more intelligent agent. It draws on foundational theorems from logic and cybernetics to support the conclusion that alignment is not an engineering challenge but a logical impossibility. Recent 2025-2026 research papers reach similar conclusions using computability theory and diagonal arguments. The work concludes that the only way to guarantee alignment is to avoid creating superintelligence altogether. All proposed restrictions are described as temporary delays rather than permanent solutions.

HabrOct 29, 2026Translated from Russian

Claude Account Bans Linked to IP Reputation, Device Environment and Payment Channels, But No Confirmed Formula
AI Security

Claude Account Bans Linked to IP Reputation, Device Environment and Payment Channels, But No Confirmed Formula

Discussions on the Chinese forum NodeSeek have highlighted observations from a user who reported bans on more than twenty paid Claude accounts. The user identified IP address reputation, device environment traces, and payment processor history as the main factors potentially triggering blocks by Anthropic. However, the original post does not present a unified scoring system or prove specific penalties for using Linux, browser-only access, or the Fable model. High query volume is described mainly as a rate-limit issue rather than an automatic account ban trigger. Community members remain divided, with some confirming isolated observations while others label the conclusions speculative and share contradictory experiences. No universal method to avoid blocks has emerged from the discussion.

AntiMalwareOct 29, 2026Translated from Russian

Local LLMs in AppSec: What Open Tools and Models Can Handle in Application Security
AI Security

Local LLMs in AppSec: What Open Tools and Models Can Handle in Application Security

The article examines how local large language models can be integrated into application security workflows without relying on cloud subscriptions. It traces the evolution from llama.cpp in 2023 through agentic frameworks like PentAGI and Strix in 2025-2026. Teams can now run models on local hardware, connect them to code analyzers such as Semgrep, and delegate specific tasks like vulnerability path reconstruction or multi-step application testing. The piece highlights practical scenarios including static code review, dynamic testing with browser and terminal agents, and security evaluation of embedded AI assistants using tools like Garak and PyRIT. It stresses the importance of task decomposition, context management, and verification of model outputs to avoid misleading results. Hardware performance, model selection, and custom agent configurations are presented as key factors for successful adoption.

HabrOct 29, 2026Translated from Russian