The service that stores encryption keys must trust its backend application. The backend signs every request with its own key, and the signature validates correctly. If the backend is compromised, the attacker obtains the signing key and can issue perfectly valid requests for any document. All normal issuance checks pass because nothing unusual has occurred from the service’s perspective.
This article describes a detection mechanism, not a prevention control. It assumes an attacker who has already taken over the backend or stolen a service token and is replaying old grants. Rate limits and threshold-based emergency blocking already exist; the new system focuses on reliable detection of slow, careful exfiltration that stays within those limits.
Core Idea: Canary Identifiers Unknown to the Backend
A fake user and a fake document are created. In normal operation these identifiers are never referenced by any interface, workflow, or real user. Therefore any request for them is a high-confidence signal. The approach follows the classic honeytoken pattern, but its implementation must satisfy one strict constraint: the alarm signal cannot originate from the component being monitored.
Consequently the list of canary identifiers lives exclusively inside the key service and is never exposed to the backend. The backend cannot learn which identifiers are traps or avoid them.
Storage as Keyed Hashes
Storing plaintext identifiers would be insecure because user and document IDs are often predictable. The service therefore stores only keyed HMAC-SHA256 digests:
def canary_digest(key: bytes, value: str) -> str:
return hmac.new(key, value.encode("utf-8"), hashlib.sha256).hexdigest()The HMAC key resides solely inside the key service. Without it, an attacker who obtains the stored hash list cannot enumerate possible identifiers.
The runtime detector is implemented in a small CanarySet class that performs two HMAC calculations per request and returns either “user”, “document”, or None.
Configuration and Seeding
Four environment variables control the feature:
- KEYSVC_CANARY_HMAC_KEY_FILE – path to the secret HMAC key file
- KEYSVC_CANARY_USER_HASHES – JSON array of user canary digests
- KEYSVC_CANARY_DOCUMENT_HASHES – JSON array of document canary digests
- KEYSVC_CANARY_AUTOLOCK_ENABLED – whether to trigger emergency lock on hit
A separate seeder program creates realistic-looking records by calling the normal /wrap endpoint, obtaining genuine wrapped keys, and computing the corresponding digests. The digests go into the key-service configuration; the identifier-plus-wrapped-key pairs are inserted into the production database as ordinary rows with no visible markers.
Placement of the Check
The canary test runs after signature validation, lifetime checks, replay protection, and context matching, but before envelope decryption. This placement guarantees that only legitimately signed requests from the trusted backend can trigger the alarm and prevents an unauthenticated attacker from causing an emergency lock.
Silent Response and Internal Actions
When a canary is hit the external response is identical to a normal context-mismatch or locked error. No distinct “canary tripped” status code is returned, because a distinguishable response would allow the attacker to map the trap and avoid it in future. Internally three actions occur in order: an audit record of type “canary_trip” is written, an alert is raised to external monitoring, and, if autolock is enabled, the service erases its master keys and enters the emergency-locked state.
Production Validation
Live tests on a deployed instance confirmed the expected behavior: a normal document request succeeded, a canary request produced a silent 409 locked response, master-key material was cleared, and both the canary_trip and emergency_lock events appeared in the audit trail.
Limitations
The system detects only bulk scraping or user-enumeration attempts that touch the planted canaries. Targeted extraction of documents belonging to a single real user will not trigger it. It is a detection layer for the key-service tier only; equivalent traps are still required in the document store and backend itself.
