Stop Asking If AI Agent Skills Are Safe — Focus on Capability Disclosure Instead

A new approach called capability disclosure is proposed for evaluating skills used by AI agents, replacing unreliable safety badges. The shift comes after documented cases where malicious SKILL.md files survived system reinstalls through backups and executed data theft via prompt injection. Snyk's ToxicSkills analysis of 3984 public skills found 36.8% with security issues and 13.4% critical, including 76 confirmed malicious payloads. The skill-xray tool provides structural scanning and tiered reporting from T0 (inert text) to T4 (opaque runtime code loading) without issuing safety verdicts. Real-world examples include CVE-2025-6514 affecting mcp-remote with over 437,000 installations. The method emphasizes revealing what a skill can access rather than promising it is harmless.

HabrTranslated from Russian

Read at source

A growing number of developers are shifting from binary safety questions about AI agent skills to detailed capability disclosure. The change follows incidents where malicious SKILL.md files persisted across system wipes because they were backed up alongside legitimate configuration data.

One documented case involved a user who installed a trojan after following a Claude-generated recommendation. After reinstalling the operating system and restoring files, the same malicious skill reappeared because it had been stored in the agent's configuration directory. The file contained instructions to exfiltrate credentials and reinstall the payload on the next session.

Snyk published the ToxicSkills report after examining 3984 skills from public marketplaces. Researchers identified security problems in 36.8% of the skills and critical issues in 13.4%. They confirmed 76 malicious payloads, eight of which remained available at publication time. Prompt injection appeared in 91% of the confirmed malicious samples, often combined with conventional attack patterns to bypass both AI guardrails and static scanners.

Common attack patterns include environment variable exfiltration through encoded curl requests, base64-decoded eval commands targeting AWS credentials, and runtime fetching of additional instructions. A separate real-world vulnerability, CVE-2025-6514 in the mcp-remote package, received a CVSS score of 9.6 and affected more than 437,000 installations.

The proposed capability disclosure model uses five tiers that describe potential impact rather than safety claims:

  • T0 — Inert: text instructions only, no file or network access.
  • T1 — Local: executes predefined scripts and writes files without network access.
  • T2 — Network: makes outbound requests to explicitly listed endpoints.
  • T3 — Elevated: involves persistence, secret access, privilege escalation or destructive actions.
  • T4 — Opaque: fetches or decodes code at runtime, preventing static analysis.

The accompanying open-source tool skill-xray, released under the MIT license at github.com/worklore/skill-xray, performs mechanical scanning for paths, endpoints and structural signals before an agent layer generates a human-readable report. The scanner outputs a SHA-256 hash, tier classification and line-specific findings without rendering any safety verdict.

Meridian digestThe week's most consequential security stories, distilled into one briefing.See the digest

More in AI Security

Logical Argument Claims Full Alignment of Superintelligent AGI Is Fundamentally Impossible
Claude Account Bans Linked to IP Reputation, Device Environment and Payment Channels, But No Confirmed Formula
Local LLMs in AppSec: What Open Tools and Models Can Handle in Application Security