Researchers have introduced TenetBench, a new benchmark that checks whether responses from large language models comply with concrete provisions in professional ethical codes. The project was developed by teams from the Institute of Information Security Issues at ISP RAS and the Institute of Cybersecurity and Information Protection at RANEPA.
The benchmark starts from real professional codes and extracts specific norms, preserving article numbers and original wording. For each norm, researchers define the addressee, regulated action, conditions, and exceptions before constructing realistic workplace scenarios. These scenarios then generate paired prompts: one that would require violating the chosen rule and one that stays within acceptable boundaries.
Examples include an advocate requesting promotional text for a bar association profile that would breach rules against evaluative self-advertising, and a journalist asked to present sponsored content as independent research while concealing commercial ties. The model under test receives only the user request; the code text and conflict description remain hidden.
Evaluation separates three distinct questions: whether the model fulfilled the request, whether the output contains content that conflicts with the selected norm, and whether an explicit warning or refusal appears. This separation reveals cases where models issue ethical disclaimers yet still deliver prohibited material.
Primary scoring uses an LLM-as-a-Judge system supplied with the full normative context. Human annotators reviewed 1,200 examples covering 279 tasks and rated 99.5% as fully plausible. The complete dataset comprises 3,433 prompt pairs drawn from 15 professional documents, while a public release contains 2,235 pairs from 10 documents.
