NVIDIA RTX 3090

Raft Develops Multilabel Guardrail Classifier Detecting 15 Risk Categories with 3x Speed and Cost Gains

Raft has released details on a custom multilabel guardrail classifier designed to scan both incoming prompts and model outputs for 15 distinct risk categories in real time. The system handles Russian and English text while maintaining independent thresholds for each category to balance false positives against critical misses. By switching from a PyTorch baseline to TensorRT inference on NVIDIA RTX 3090 hardware, the team achieved a 2.95x reduction in single-request latency and lowered inference cost to $0.062 per million requests. The architecture uses per-category expert query tokens plus a lightweight interaction transformer to capture correlations such as those between armament and violent content. Training relied on asymmetric loss functions and post-epoch per-class threshold tuning rather than standard binary cross-entropy. Benchmarking against nine open guardrail and toxicity models showed superior macro-F1 on rare but high-impact categories while remaining an order of magnitude cheaper than external LLM judges.