Blog · 28 September 2026
61.22% of real human TOEFL essays were labelled AI-generated. The cause is not a bug in someone's sentiment; it is an engineering choice about whose writing counts as human.
In 2023, researchers at Stanford published a paper in Patterns that tested seven widely used AI detectors on two sets of writing that were definitely human: 91 TOEFL essays written by non-native English speakers, and 88 essays by US eighth-graders.
The detectors were almost perfect on the native-speaker essays, a false-positive rate around 3.2%. On the non-native essays:
| Finding | Rate |
|---|---|
| Non-native essays misclassified as AI-generated (average across 7 detectors) | 61.22% |
| Non-native essays flagged by all seven detectors unanimously | 19.78% |
| Non-native essays flagged by at least one detector | 97.80% |
Read that third row again. Of 91 human-written essays, 89 were accused by at least one detector. The researchers computed a bias multiplier of roughly 19×: a non-native writer's genuine work was about nineteen times more likely to be falsely flagged than a native speaker's.
Most detectors lean on a language-model statistic called perplexity, how surprised a model is by the next word. Predictable text scores low. AI text tends to be predictable, so low perplexity is treated as an AI signal.
But perplexity does not measure who wrote something. It measures how formulaic and simple the prose is. And non-native writing systematically scores lower on the things that drive it:
A second-language writer with a clean, careful, straightforward style lands in the same statistical neighbourhood as a language model, not because they used one, but because formal simplicity looks similar from the outside. The Stanford authors found the unanimously-misclassified essays had significantly lower perplexity (p ≈ 9.7 × 10⁻⁵).
The uncomfortable part: the trait that gets you flagged is often the trait that makes writing clear. If English is your second language, writing plainly is not a flaw. It is usually good practice.
The bias is not an immutable property of "AI detection". It was a property of how those models were built, particularly what was in their human baseline.
Two follow-ups make this concrete. Pangram Labs re-ran the identical 91-essay benchmark and reported a 0% false-positive rate, attributing it to having held those essays out of training. GPTZero published a debiasing response and its updated model reported 7.7% on the same benchmark, still above the native baseline, but an order of magnitude better than 61%.
Same benchmark. Same essays. Different engineering choices. The gap between 61.22% and 0% is not a difference in ethics; it is a difference in what the model was taught to consider human.
When we built our human baseline, we had to decide which writing to treat as "human" for the model to learn from. It is an easy choice to get wrong, because the convenient corpora, literature, Wikipedia, news, are overwhelmingly written by native speakers with formal, complex prose.
We deliberately included ESL and second-language learner writing in the human corpus, alongside 19th-century novels, public-domain 1900–1930 genre fiction, Wikipedia, Stack Exchange, Hacker News and news. Concretely, our validation set is 5,035 held-out human samples from 12 sources, and non-native English writing is part of what "human" means to this model, not an edge case bolted on afterwards.
| Our measured numbers (held-out, strict train/test separation) | strict | balanced |
|---|---|---|
| False positives on real human text, 5,035 samples, 12 sources, 0.6% source-balanced | 0.9% | 5.0% |
| Detection of real AI text (92 samples) | 91.3% | 97.8% |
What that does and does not claim. We are not claiming zero bias; we have not published an ESL-only error rate, and we would rather say that plainly than dress up a number we have not measured. What we can say is that second-language writing is inside the human baseline by design, and our false-positive rate on mixed real-world human text is 0.9% at the strict setting.
We are explicit about this, and it is not a legal formality: our detector is not licensed for academic integrity circumvention, not for submitting AI-written work, not for helping anyone else do so, and not as a way to argue with your institution's process. This article is about the fairness of a statistic applied to professional and creative writing. If you want to check a piece of your own prose before sending it to a client, an editor, or a platform, that is what we are for.
The free checker accepts up to 2,000 words and shows you the individual features behind the verdict, not just a number. No signup, nothing leaves your browser session.
If you need to run checks at volume, or embed this in your own product, the offline pack runs on your own machine with no word limits.
Sources
Our own accuracy figures come from held-out test data with strict train/test separation and are reproducible from the model metadata shipped in the pack. Estimates only, not proof of authorship.
More writing · Product & pricing · Try the free checker · support@prosechecker.com