AI Agent Workflow Exceeds 100-Unit ERP Limit Reaching 120 Due to Check-Then-Act Race Condition

A detailed case study demonstrates how an AI workflow authorized to order no more than 100 units of COMP-17 ended up creating 120 units across three suppliers. Each individual order of 40 units passed authorization checks returning ALLOW, yet the cumulative result violated the human-delegated mandate. The root cause was a classic check-then-act race condition where stateful permission tracking read the same initial balance concurrently before any updates occurred. Sequential stateful checks prevented the overrun, but concurrent execution allowed all three requests to read zero consumption and receive ALLOW. An improved design using PostgreSQL SELECT FOR UPDATE performs atomic reservation of the grant balance before calling the ERP system, ensuring only two orders succeed and the third is rejected with DENY_CAPACITY. The experiment used a deterministic planner without any LLM to isolate the authorization mechanism from model reasoning quality. Five key questions are provided for production deployments involving cumulative business limits delegated to agents.

HabrTranslated from Russian

Read at source

A new case study examines how an AI workflow authorized to order no more than 100 units of goods nevertheless created orders totaling 120 units in an ERP system. The scenario involved three separate orders of 40 units each to suppliers A, B, and C. Every individual request passed through the authorization layer and received an ALLOW response, yet the final ERP state showed 120 units.

The core issue lies in the difference between local and global authorization checks. A simple rule checking if requested_quantity <= 100 answers only whether the current order stays under the threshold. It does not verify whether the sum of all orders will remain within the original human-delegated limit of 100 units for COMP-17 across approved suppliers.

Sequential stateful checks correctly accumulated consumption and rejected the third order, leaving 80 units in the ERP. However, when requests executed concurrently, all three handlers read the initial balance of zero before any updates, allowing all three ALLOW decisions. The final ERP state reached 120 units with no lost updates or double counting.

The corrected implementation performs an atomic reservation inside a database transaction using PostgreSQL SELECT ... FOR UPDATE. This reserves the requested quantity from the grant balance before the ERP call occurs, ensuring that once 40 units are reserved they become unavailable to other concurrent requests. In concurrent tests, two orders received ALLOW while the third was rejected with DENY_CAPACITY, resulting in a correct total of 80 units.

The study emphasizes that an ALLOW decision only proves the authorization component approved the request. Without atomic reservation tied to the overall grant, multiple locally valid decisions can collectively exceed the delegated mandate. The experiment deliberately used a deterministic planner without any LLM to focus purely on the authorization boundary.

Five practical questions are recommended before deploying agents with cumulative business limits: what exactly the human authorized, how actions are linked to a single grant with scope and remaining balance, whether ALLOW is bound to atomic reservation, whether parallel branches or retries can overspend the same balance, and how final state is independently verified beyond authorization logs.

Full reproduction materials, 44 passing tests, and evidence files are published under the tag shared-cumulative-authority-v1.0 on GitHub. The findings apply to any agentic system where multiple actions consume a single finite human-delegated mandate such as purchase volumes, payment limits, or message quotas.

Meridian digestThe week's most consequential security stories, distilled into one briefing.See the digest

More in AI Security

Logical Argument Claims Full Alignment of Superintelligent AGI Is Fundamentally Impossible
Claude Account Bans Linked to IP Reputation, Device Environment and Payment Channels, But No Confirmed Formula
Local LLMs in AppSec: What Open Tools and Models Can Handle in Application Security