Home › Signed benchmark verification

Signed benchmark verification — RAID, 2026‑09‑04.

The Truth‑in‑AI benchmark result on this page is Ed25519‑signed by TriGeoChiral Engineering and its payload is anchored into the Bitcoin blockchain via OpenTimestamps. Every published metric can be recomputed from the signed artifact. You can verify it in your browser (the button below) or from a shell in one command. The detector itself is not distributed.

Signature

Ed25519 · FIPS 186‑5

64‑byte detached signature over canonical JSON payload.
Benchmark

RAID (Dugan et al., 2024)

Public dataset. arXiv:2405.07940
Time anchor

Bitcoin · OpenTimestamps

SHA‑256 of the signed payload committed via opentimestamps.org. Verify with ots verify RAID.signed.json.ots.

Result — RAID, 30 configurations

0.9302

Mean AUC across code · German · Czech, 10 attacks each. Full breakdown — and a “Verify now” button — directly below.

A+

Every one of the 30 configurations.

Each row is one (language × attack) configuration on the RAID benchmark. The Published column is the AUC in the signed artifact. The Best published (same test) column shows the highest AUC any commercial or academic detector has publicly reported for that exact configuration — sourced and cited; left as not published where no comparable public number exists rather than guessed. The Verify column populates when you press “Verify now” below — it recomputes the AUC on your device from the stored score array for that row and compares it to the published number. A row is green when |recomputed − published| < 0.005.

📊 Download Full Rankings CSV 📦 Download Complete Proof Package (.zip) Or use the CLI ↓

Ready. Press “Verify now” to run all three stages in-browser. This runs entirely on your device — no account, no upload.

Domain	Attack	n_h	n_a	Published AUC	Best published (same test)	CI 95%	TPR@5%FPR	ECE	Verify
Loading signed artifact…

n_h = human test items; n_a = AI-generated test items in that configuration. CI is a bootstrap 95% confidence interval on the AUC. ECE is expected calibration error — how well the score can be read as a probability. Best published is cited from Dugan et al., 2024 (Table 4) where the paper reports a per-attack number for that domain; RAID does not publish a per-attack breakdown for code, German, or Czech beyond the paraphrase headline comparison, so most rows read not published honestly rather than backfilled from an unrelated aggregate. See Case 03 for why leaderboard-wide numbers are a separate, weaker comparison.

Verify from a shell.

Three checks, one command. Signature · dataset manifest · per‑configuration AUC recompute. Prints ✓ RESULT VERIFIED on success.

# from PyPI, no clone required pip install truth-in-ai-verify curl -O https://trigeochiral.com/proof/RAID.signed.json curl -O https://trigeochiral.com/proof/pubkey.pem truth-in-ai-verify --result RAID.signed.json --pubkey pubkey.pem # → ✓ RESULT VERIFIED · Mean AUC 0.9302 across 30 configs

The truth-in-ai-verify package is on PyPI — pypi.org/project/truth-in-ai-verify — and its source is BSD-license verifier code only (no detector). It uses cryptography, numpy, and scikit-learn for the AUC recompute.

Downloads & Full Benchmark Artifacts.

Each artifact ships with its SHA‑256 checksum. Download the full ZIP or CSV for complete offline analysis.

File	Size	SHA‑256	Purpose
📦 DMITVA_AND_RAID_FULL_RANKINGS_PROOF.zip	161 KB	c8a14b92d77a02b1f8e4392a105f2b88137a50e9321f5791a8264e12e87ab690	Complete Proof Package: DMITVA (English) + RAID signed JSONs, CSV rankings, pubkey, & verifier script.
📊 truth_in_ai_full_rankings.csv	4.4 KB	f491a27e908b1c4e7235a90d81b49f9024c185a672901458e0a12e4719b02a3c	Exhaustive CSV Rankings: Complete per-configuration table with DMITVA English & RAID metrics.
DMITVA.signed.json	1.8 KB	a2849b1c70e954817a32b901f41e57c8230914c81059281a074e1957291a8b92	Signed DMITVA English academic & human writing benchmark payload + Ed25519 signature.
RAID.signed.json	760 KB	38e402bb42d8d3b54a4fb20c7ff4a5d57f868a6d7292edfff4f38f9f2aa283fa	Signed RAID benchmark result. Payload + Ed25519 signature.
pubkey.pem	113 B	0396b4e330b63601898b474d9679a7eb5b0a5b6f6bb8015e1c4e12f848b63e24	Ed25519 public key. Verify the signatures above.
RAID.signed.json.ots	735 B	d32d4b204fe28583917734a87a02bb8d611dc68c4a72eb82fe7853e8712b3de4	OpenTimestamps proof. Anchors the SHA‑256 of the signed payload to Bitcoin.

Independent SHA‑256 check: sha256sum RAID.signed.json pubkey.pem RAID.signed.json.ots README.md

Head‑to‑head, adversarial paraphrase.

Paraphrase is the RAID attack where the majority of published detectors collapse. Numbers below are AUC on the paraphrase configuration of the RAID benchmark — the same “Best published” number cited in the German paraphrase row of the table above. Every competitor row cites its own public benchmark.

Detector	German paraphrase AUC	Source
GPTZero	0.62	Dugan et al., 2024, Table 4
Originality.ai	0.71	Dugan et al., 2024, Table 4
Binoculars	0.76	Dugan et al., 2024, Table 4
Truth‑in‑AI	0.9958	This artifact · `domain=german, attack=paraphrase`

What the signature is over.

The Ed25519 signature covers the canonical JSON of the payload field of RAID.signed.json: sorted keys, no whitespace, UTF-8. If a single byte of the payload were altered, the signature verification fails, and so does the SHA-256 check.

Algorithm: Ed25519 (RFC 8032, FIPS 186‑5, §7.6)
Public key: MCowBQYDK2VwAyEAreFyqDy09PFT10TA4YWkkQz52w9znrFoKmiRmS6IuRs=
Payload SHA-256: 999e835b487637633926f9f0124f5a5e491603d293f0695b1cab25ab598bd29d
Signature: shown by the verifier · 128 hex chars (64 bytes)
Anchor: Bitcoin, via OpenTimestamps · ots verify RAID.signed.json.ots

How to verify from scratch, without our tooling.

For the paranoid buyer. The verifier package is a convenience; nothing in it is privileged. Anyone can reproduce every check with off-the-shelf tools.

# three-line, no dependencies beyond the standard cryptography library python3 - <<PY import json, hashlib from cryptography.hazmat.primitives.serialization import load_pem_public_key signed = json.load(open("RAID.signed.json")) canonical = json.dumps(signed["payload"], sort_keys=True, separators=(",", ":")).encode() assert hashlib.sha256(canonical).hexdigest() == signed["payload_sha256"], "payload SHA mismatch" pub = load_pem_public_key(open("pubkey.pem","rb").read()) pub.verify(bytes.fromhex(signed["signature"]), canonical) # raises on failure print("Ed25519 signature: VALID") PY