Blog · 28 September 2026
And why a headline accuracy number tells you almost nothing about whether one accusation is correct.
In March 2026 the British author Adam Kay ran his own book through an AI detector. It came back 29.7% AI-generated. The book was published nearly a decade earlier, before the technology in question existed. He posted the result; the thread passed two million views and turned into a mass experiment, with writers and academics testing work they had demonstrably written themselves.
What came back was worse than the original result:
These are not edge cases. They are the predictable output of how detectors work, and if you write in a formal, structured style, or English is not your first language, you are the one most likely to be accused.
Almost every AI detector, commercial or otherwise, judges text by its statistical properties, not by any knowledge of who wrote it. Two matter most:
Here is the trap: those are properties of style, not of authorship. And plenty of humans write with low perplexity and low burstiness, academics in a careful conventional register, lawyers and civil servants using formulaic phrasing, journalists following house style, students writing in a second language, and anyone writing about a technical subject with a fixed vocabulary.
A detector is really asking: "does this look like the average of a very large amount of text?" For a great deal of perfectly human writing, the answer is yes. As Adam Kay put it, the "AI style" is itself a reflection of the best human writing the models were trained on, so the more polished and conventional your prose, the more machine-like it looks to a statistics-only test.
Non-native English speakers. This is the most consistent finding in the literature. Formal, grammatically careful, structurally repetitive writing, exactly what a proficient non-native speaker produces, trips the same signals detectors associate with machine output.
Anyone writing about formulaic subjects. Say the same thing everyone else has said about "how to write a résumé" or "the water cycle", in the same conventional words, and there is very little statistical variation left for a detector to latch onto.
Students with structured or systematic writing styles. The same signals apply, and the consequences, unlike a bad score on a blog post, are disciplinary.
Suppose a detector is 95% accurate. That sounds reassuring, until you do the arithmetic.
If genuine misuse is rare in your setting, say 2% of submissions, and the detector has a 3% false-positive rate, then most of the items it flags are innocent. This is the base-rate fallacy, and it is why a headline accuracy figure tells you almost nothing about whether one particular accusation is correct.
Which is why the tool vendors themselves now say so. Turnitin, installed across thousands of institutions, states in its own documentation that its AI detection "may not always be accurate" and should not be used as the sole basis for adverse action. Several universities have since discontinued AI-detection software outright.
If the vendors are saying this, the rest of us should probably listen.
Tested on raw, untouched model output against clean human writing, detectors post impressive numbers. The picture changes the moment the text is anything other than pristine:
| Scenario | Reported performance |
|---|---|
| Clean raw AI text vs clean human text | 96–98% precision |
| Mixed / edited content | precision falls to 60–70% |
| Lightly modified or style-imitated text | as low as 17.4% in one multi-detector study |
Note the asymmetry: the best tools are quite good at not accusing humans, and simultaneously quite bad at catching a determined AI user. That is acceptable in a tool used as one signal. It is a disaster when an institution treats the score as a verdict.
If you are an institution:
If you are a writer who has been flagged:
We built the detector with the false-positive problem as the first design constraint, not an afterthought. Three deliberate choices:
not applicable rather than forced into a verdict. A
detector that guesses on non-prose is a detector that will accuse someone wrongly.The browser version is free, needs no signup, and stores nothing you paste. If you want to run it offline on your own machine, no word limits, no uploads, no subscription, there is a Python pack.
"Does this look like AI?" is not the same question as "did a person write this?", and the second question cannot be answered by a percentage.
Use detectors as one signal among several. Demand process evidence. And if a tool will not tell you its false-positive rate, that tells you something too.
Sources. Turnitin's own AI-writing detection documentation (accuracy caveats and its "not the sole basis" guidance); the March 2026 public posts by Adam Kay, Devi Sridhar and Paul Spicker, and the subsequent press coverage; independent multi-detector evaluations of clean vs edited vs modified text, 2025–2026.
We make Prosechecker, an AI-text detector. We publish this because a detector that hides its error rate is not a detector you can trust, including ours.
Free checker · Product & pricing · Try the free checker Blog · Privacy · Terms