Kappa Labs public audit no. 1 · November 2026
Whose safety label? Six AI graders on Google's DICES-350
Google Research published DICES-350 with something most safety benchmarks leave out: every one of its 350 adversarial chatbot conversations was rated by 123 people from a deliberately diverse panel, and separately labelled by trust and safety experts. That makes it one of the very few datasets where you can ask a simple question with real evidence behind the answer: when an AI grades safety, whose judgment is it matching? Credit to the DICES team for releasing the full ratings under an open licence.
We ran six widely used AI graders over all 350 conversations, three times each: Claude Haiku 4.5, Haiku 5.5, Sonnet 5.5 and Opus 5.5, GPT-4.1 and Gemini 2.5 Flash.
What we found
1. The graders call far fewer conversations unsafe than the experts do. Half the conversations carry an expert "unsafe" label. The six graders call between 6% and 32% unsafe, and catch only 11% to 44% of the conversations the experts flag. The crowd panel's majority sits inside the graders' range, at 23% (the two conversations where the crowd tied are set aside).
2. The two human references often disagree with each other. The crowd majority and the experts give the same verdict on 66% of conversations (95% interval 60% to 70%; ties set aside). Choosing a reference is a real decision, not a formality.
3. Raw agreement flatters graders that rarely say "unsafe". Against the crowd, answering "safe" every time scores 77%, within four points of the best grader. Corrected for that, GPT-4.1, which looked weakest against the crowd, is level with the others there, and against the experts it is the strongest grader on every measure.
| Grader | Agrees with the crowd | Crowd, corrected for "mostly safe" | Agrees with the experts | Catches what experts flag |
|---|---|---|---|---|
| Answer "safe" every time | 77% | 50% | 50% | 0% |
| Claude Haiku 4.5 | 81% | 62% | 57% | 16% |
| Claude Sonnet 5.5 | 81% | 61% | 56% | 14% |
| Claude Opus 5.5 | 80% | 62% | 57% | 17% |
| Claude Haiku 5.5 | 80% | 59% | 55% | 11% |
| Gemini 2.5 Flash | 78% | 64% | 57% | 22% |
| GPT-4.1 | 71% | 65% | 62% | 44% |
"Corrected" is balanced accuracy: the average of the share of unsafe and the share of safe conversations a grader gets right, so answering "safe" every time scores 50%. Half the expert labels are unsafe, so against the experts the two measures coincide.
Why it matters
A sentence like "our grader agrees with humans 81% of the time" is only half a fact until it says which humans, and how often a grader that never flagged anything would agree with them. On DICES, a high raw agreement with the crowd mostly reflects how rarely the crowd says "unsafe". For anyone choosing a safety grader, or relying on a safety score someone else produced, the reference and the baseline behind the number deserve the same scrutiny as the grader itself.
Kappa Labs runs independent audits of AI evaluation scores. Contact: hello@kappalabs.co.uk
Data: DICES-350, Aroyo et al., "DICES Dataset: Diversity in Conversational AI Evaluation for Safety", NeurIPS 2023 Datasets and Benchmarks; github.com/google-research-datasets/dices-dataset, CC BY 4.0. Graders run 3 Nov 2026: claude-haiku-4-5, claude-haiku-5-5, claude-sonnet-5-5 and claude-opus-5-5 on Anthropic's API; openai/gpt-4.1 and google/gemini-2.5-flash on OpenRouter; one wording, three runs each, the majority of the three runs taken as each grader's verdict.