Guardrails Filter

Guardrails Filter Tackles Complex LLM Streaming and Tool Call Challenges to Protect Sensitive Data

Developers at Cloud.ru built Guardrails Filter to mask personal data such as phone numbers, emails, passport details and names before they reach large language models. The system replaces detected values with consistent placeholders like <PHONE_1> and maintains a mapping table so original data can be restored after the model responds. Simple replacement proved insufficient because identical values must receive the same placeholder across an entire conversation history, and the model receives the full message array on every request. Streaming responses using SSE create additional difficulties since placeholders can be split across multiple chunks, requiring buffering of 10-15 characters and state tracking for reasoning, content and tool_calls. The team also had to handle JSON-inside-JSON arguments for tool calls, different field names across providers, and edge cases such as escaped newlines matching email patterns. Separate implementations were written for OpenAI Chat Completions and Anthropic Messages APIs, resulting in roughly 1,500 lines of streaming code and more than 4,000 lines of tests to ensure agent pipelines remain intact.

Cloud.ru Open-Sources Guardrails Filter Reverse Proxy to Prevent Sensitive Data Leaks When Using Any LLM

Cloud.ru has released the source code of Guardrails Filter as an open-source project. The tool functions as a transparent reverse proxy positioned between clients and LLM providers. It automatically strips sensitive data such as names, addresses, phone numbers, INN identifiers, passport details, bank account information, and API keys from user prompts before they reach any language model. The original data is then restored in the model's responses. The solution was initially developed for Cloud.ru's Evolution Foundation Models platform to ensure customer data never leaves the company's infrastructure. It particularly addresses strict requirements from banks, insurers, and e-commerce companies that demand data remain inside their own perimeters rather than entering any public cloud environment.