Verifiable vs black-box AI detection
Black-box AI detectors give you a score you cannot check. Verifiable detection gives you a signed result you can recompute independently. When an AI-detection result affects someone’s grade, job, or publication, being able to audit the number — not just trust it — is the difference that matters.
| Can you… | Verifiable detection (Proofline) | Typical black-box detector |
|---|---|---|
| See a signed result, not just a score? | Yes — signed (Ed25519) + published artifacts | No — score only |
| Recompute the result yourself? | Yes — in your browser, from the signed artifact | No |
| Hand a third party a result they can confirm? | Yes — signed and time-anchored | No |
| Avoid a generative model in the measurement path? | Yes — a measurement, not a model’s opinion | Often not |
| Get a language-agnostic document result? | Yes — Document Review mode | Usually English-tuned |
The right-hand column describes the common pattern across black-box detectors as a category, not any single product. Every claim in the left column is one you can check yourself on the proof page.
These are documented, public events. Each is a case where the inability to audit a detector became the problem.
OpenAI withdrew its own AI-text classifier
OpenAI took down its AI Text Classifier citing a “low rate of accuracy.” By its own published figures the tool correctly flagged only about a quarter of AI-written text and misflagged roughly one in eleven human passages. If the company that built the models couldn’t make a reliable black-box classifier, an unauditable score deserves scrutiny.
Vanderbilt disabled Turnitin’s AI detector
Vanderbilt University turned off Turnitin’s AI-writing detector, citing false-positive risk and, explicitly, “no insight into how it works.” Opacity was the stated reason. A signed, reproducible result answers exactly that objection.
Detectors were found biased against non-native English writers
A Stanford study (Liang et al., Patterns) found GPT detectors flagged more than half of non-native English (TOEFL) essays as AI-written, while rarely flagging US student essays. A result you can inspect and reproduce is how such failures get caught rather than hidden.
- — OpenAI AI Text Classifier withdrawn (accuracy figures), July 2023: Search Engine Land.
- — Vanderbilt disables Turnitin’s AI detector, Aug 16 2023: Vanderbilt University.
- — Liang et al., “GPT detectors are biased against non-native English writers,” Patterns, 2023: arXiv:2304.02819.