Home › Case studies › Case 03
Case 03 · Benchmark saturation
The RAID leaderboard’s 99% club fit on the public labels.
Why the top 10 detectors all hit 99% — and why that number is meaningless.
The failure · on the record
RAID-train’s labels are public. Ten commercial detectors submitted results at 99%+ accuracy across every attack. MELD is submitted as “anon-review-meld-2026” — a research submission tuned to the benchmark. DeBERTa-ConPara has four versioned iterations. OpenAI dropped SWE-bench Verified in 2025 because “the score stopped meaning anything” after contamination. RAID is the same story now.
The numbers
| Detector | % of RAID slices scored 1.000 | Fit on public labels? |
|---|---|---|
| Zhuque (Tencent) | 76.3% | very likely |
| Trinka | 73.7% | very likely |
| MELD (anon research) | 73.0% | yes (name is anon-review-meld-2026) |
| Grammarly | 69.7% | very likely |
| Binoculars (academic 2024) | 11.5% | no (predates leaderboard) |
| Truth-in-AI | 0.0% | no — reference corpus is dmitva only |
What we do differently
Our manifest is public. The signed RAID.signed.json binds our reference corpus to dmitva (a separate English AI-vs-human dataset) via SHA-256 hash. Zero RAID-extra rows appear anywhere in our reference source.
Any auditor can verify this without our permission by rehashing the datasets.
Proof
Reference-corpus SHA-256 published in manifest. Inspect the manifest →
Try it yourself
Score any text NOT from RAID.
curl -X POST https://api.trigeochiral.com/detect \
-H 'Authorization: Bearer $TRIALKEY' \
-d '{"text": "Any AI or human text"}'Bounty · put money where the claim is
$10,000
Prove any top-10 RAID leaderboard detector did NOT fit on RAID-train. Submit training-data proof to trigeochiral@gmail.com.
Sources