Why Do AI Detectors Give Different Results on the Same Text?

AI detectors disagree because each one runs its own model, trained on its own data, cut at its own decision threshold, and applied to text chunks of its own chosen size. They're not reading the same evidence the same way. Disagreement is normal, predictable, and — once you understand the five causes — actually useful information.

If you've ever pasted one essay into five detectors and gotten five different verdicts, you already know the unsettling part. One tool says 96% AI. Another says 12%. A third refuses to commit. Your first instinct is that at least four of them must be broken, or that the whole category is a scam.

Neither is quite right. The tools are doing different things under the same label, and the differences are structural, not random. This post walks through a realistic example of how the same essay scatters across five tools, then explains the five causes of that scatter in order of how much they matter. If you want the deeper story on any single cause, we've written companion pieces on how detection thresholds get chosen and why short text breaks detectors entirely. This piece is the map; those are the territories.

One disclosure up front: we build a detector and a humanizer ourselves at HumanFlow, so we have a stake in this market. Our editorial policy is to cite published evidence, mark anything unverified, and never invent test results — judge what follows accordingly.

The scenario: one essay, five verdicts

Here's an illustration. The numbers below are hypothetical — we constructed them to show the pattern that anyone who has run this experiment will recognize. They are not results from a test we ran, and you shouldn't cite them as data. (When we publish hands-on numbers, they'll come with a dated methodology. That's our standing policy.)

Imagine a 900-word history essay. The student drafted 600 words themselves, asked ChatGPT to "tighten up" two paragraphs (about 200 words came back substantially rewritten), and wrote the conclusion at 1 a.m. in short, tired, declarative sentences. A thoroughly ordinary 2026 essay, in other words. Now run it through five detectors:

Five tools, five answers, and every one of them is behaving exactly as designed. That's the point most coverage misses. The scatter isn't malfunction. It's five different design philosophies looking at one genuinely ambiguous document — a blended human/AI text, which the Washington Post's April 2023 test of Turnitin identified as the hard case even Turnitin acknowledged struggling with.

Notice something else: the tools don't even disagree about the same thing. Tool A is saying "the AI share is too low for us to report a number responsibly." Tool C is saying "some of this text is machine-typical, so we're flagging the document." Those are different claims that happen to share a percent sign. Keep that in mind as we go through the causes.

Cause 1: thresholds — the biggest reason, and the least visible

Every statistical detector produces something like a probability, then compares it to a cutoff line the vendor chose. Above the line: "AI." Below it: "human." That line is called the decision threshold, and no two vendors put it in the same place, because no two vendors are optimizing for the same mistake.

An education-focused tool fears false accusations above all else. Flag an innocent student and you may end up in a misconduct hearing, a news story, and a canceled contract. So education tools set the line high and hedge below it. Turnitin is the clearest documented example: its headline claims — 98% accuracy, under 1% false positives — apply only to documents where more than 20% of the text is flagged, and scores from 1–19% display as an asterisk rather than a number, which is Turnitin's own admission that low-range scores aren't reliable enough to print. Turnitin's chief product officer, Annie Chechitelli, put the same trade-off in plainer words to BestColleges in April 2023: "we find about 85% of it. We let probably 15% go by".

A marketer-focused tool fears the opposite mistake. Its customers are publishers who lose money if AI slop reaches their site, so missing AI text is the expensive error and flagging a human writer is merely awkward. Originality.ai, the best-known tool in that camp, even sells the trade-off as a feature: its current model documentation cites a 0.5% false-positive rate at one setting and 1.5% at another, with true-positive rates near 99% — an explicit menu of threshold choices (Originality.ai accuracy study page, fetched during writing).

Same underlying math, opposite tuning. That's why Tool A above shows an asterisk while Tool C shouts 96%. Neither is lying. They drew the line in different places because their customers punish different mistakes. This one cause probably explains more cross-tool disagreement than everything else combined, which is why we gave it a full article of its own.

Cause 2: training-data vintage — detectors age like milk

A detector is a model trained to distinguish human text from AI text — the AI text that existed when it was trained. Frontier models change their statistical fingerprint with every release. A detector trained mostly on GPT-3.5 and early GPT-4 output is measuring yesterday's fingerprint against today's models.

The practical result: two detectors trained eighteen months apart are effectively different instruments. One might be sharp on current-generation models and rusty on older ones; another might be the reverse. When your essay contains paragraphs polished by a 2026 model, a detector last retrained in 2024 sees them differently than one retrained last quarter. Vendors rarely publish retraining dates, so you can't see this variable from the outside — you can only see its effects, as scatter.

The cautionary tale is OpenAI's own classifier. The company that built GPT retired its detector in July 2023 after it correctly identified only 26% of AI-written text while falsely flagging 9% of human writing — by OpenAI's own announcement. If the model's maker couldn't keep a classifier current against its own outputs, expect third parties to be perpetually catching up too.

Vintage also explains a specific pattern you may have seen: an essay written with a brand-new model scores "human" everywhere for a few weeks after the model launches, then starts scoring "AI" as vendors retrain. The essay didn't change. The instruments did.

Cause 3: segmentation — what counts as "the document"?

Detectors don't judge your document as one blob. They cut it into segments — sentences, sliding windows of a few hundred words, or paragraph-level chunks — classify each segment, then aggregate the results into a headline number. Every step of that pipeline involves choices, and the choices differ per vendor.

Consider our hypothetical essay: 600 human words, ~200 AI-assisted words, and a tired-sounding conclusion. A tool that classifies sentence by sentence might flag the two polished paragraphs plus a few false hits in the conclusion, yielding a mid-range score. A tool using large overlapping windows might dilute those same paragraphs across windows that are mostly human, and report a low score. A third tool might report the share of flagged segments rather than a probability, which is a different quantity entirely dressed in the same "% AI" costume.

That last point deserves a beat. "40% AI" can mean we think there's a 40% chance this was AI-written or we think 40% of this document was AI-written. Those are wildly different claims. Vendors are not consistent about which one their number means, and most users never find out. Two tools can agree perfectly about your essay's content and still print different numbers because they're answering different questions. GPTZero, for instance, documents that its accuracy is higher at the document level than at paragraph level, and higher at paragraph level than per-sentence (GPTZero FAQ) — a candid acknowledgment that granularity trades against reliability.

Cause 4: length handling — short text amplifies everything

Statistical detection needs sample size. Perplexity and burstiness — the predictability and rhythm measurements underneath most detectors (see how detectors work) — only stabilize across hundreds of words. Below that, noise dominates, and every vendor handles the noise differently.

The published minimums tell the story: Turnitin requires roughly 300 words of prose before scoring. Winston AI requires 500 characters. Copyleaks sets a 255-character floor on its web platform. Originality.ai's website scans need 100 words, and its own help documentation says accuracy diminishes below that even via API. Sapling says its detector "becomes much more accurate after 50 or so words." Five vendors, five floors — and above each floor, five different rates at which confidence ramps up with length.

So a 350-word discussion post might clear Turnitin's minimum comfortably, sit near the edge of reliable territory for one consumer tool, and be well inside the confident zone for another. Their scores on that post will scatter more than their scores on a 2,000-word essay would, purely because of how each handles marginal-length input. Short text is such a reliable disagreement generator that we wrote a separate piece on the minimum-words problem.

Cause 5: language handling — the quiet variable

Most major detectors were built and validated primarily on English. Turnitin says so explicitly. What happens with other languages — or with English written by non-native speakers — varies enormously by vendor, and it shows up as cross-tool disagreement before it shows up as anything else.

The landmark evidence is Liang et al., published in Patterns (Cell Press) in 2023: seven detectors tested against 91 human-written TOEFL essays falsely flagged them as AI 61.22% of the time on average. Eighty-nine of the 91 essays were flagged by at least one detector — while the same detectors were near-perfect on essays by native-speaking US 8th graders. (Turnitin wasn't among the seven tested, though it uses the same statistical approach.) Note what that "89 of 91 flagged by at least one" figure implies: the seven detectors disagreed with each other constantly about which essays to flag. Only 18 of 91 were flagged by all seven. Disagreement wasn't a side effect of the bias — it was the visible symptom of it.

Vendors have responded differently since. Copyleaks publishes per-language accuracy figures (99%+ claims for English, mid-90s for some other languages, per its site). Others quietly support English only. Run a Spanish essay, or an ESL student's English essay, through five tools and you're sampling five different maturity levels of multilingual handling. Scatter is the guaranteed result — and for non-native writers, so is elevated false-positive risk.

What disagreement actually means (and what it doesn't)

Here's the reframe worth keeping: disagreement between detectors is a measurement of uncertainty, not a conspiracy and not proof that detection is fake.

When five thermometers read the same pot of water five ways, you learn something about the thermometers and something about the water — maybe it's mid-boil, uneven, genuinely hard to measure. Blended human/AI text is the mid-boil case. Fully AI-generated, unedited text tends to score high nearly everywhere; it is the one case the whole category handles comparatively well. Fully human text by a native speaker on a familiar topic tends to score low nearly everywhere. The scatter zone lives in between: mixed drafts, edited AI text, short text, non-native English, technical prose.

That means the scatter itself is a signal. Broad agreement at the extremes is modestly informative. Wide disagreement usually tells you the text sits in territory where no detector's answer deserves much weight — which is precisely when acting on any single score is most dangerous. A teacher who gets 96% from one tool and an asterisk from another shouldn't average them. They should recognize the document as unmeasurable by these instruments and reach for other evidence: draft history, version control, a conversation with the student about their argument.

It also means "shopping" detectors until one clears you — or convicts you — is meaningless. With enough tools and a borderline document, you can obtain any verdict you want. That cuts both ways: a student waving one "0% AI" screenshot has proven as little as an instructor waving one "98% AI" screenshot.

How to act on conflicting scores

Practical guidance, by situation:

If you're a student checking your own work. Expect scatter on anything you've edited heavily or written under pressure. One high score among several low ones is not a prophecy of doom; it's one aggressively tuned instrument doing what it was tuned to do. What protects you in an actual dispute isn't a favorable screenshot — it's process evidence: version history, outline drafts, notes. Keep them. If you use a detector, use one that shows sentence-level results rather than a single number, so you can see which passages trip the alarm and revise them into your own voice. Our own AI detector does exactly that — with the caveat we attach to everything: it doesn't promise to beat any other detector, because nobody can honestly promise that, and we publish no accuracy percentage for it without published methodology.

If you're an instructor. Treat any single score as a hypothesis, never a finding. If you must use detectors at all, know your tool's threshold philosophy — an asterisk from Turnitin and a 96% from a marketer tool are not comparable claims. Never escalate on one number, and be especially slow with short submissions and non-native writers, where the documented error rates are worst. Vanderbilt University disabled Turnitin's AI indicator entirely in August 2023 after doing the false-positive math at scale; whatever you conclude about your own policy, their published reasoning is worth your ten minutes.

If you're a writer or editor delivering to a client. Ask which detector the client actually uses and what score they act on, because "passes AI detection" is meaningless without naming the tool, the version, and the threshold. Put that in the contract if stakes are high. And keep drafts — provenance beats scores in every dispute we've seen reported.

In all cases: two low-cost habits eliminate most conflicting-score anxiety. First, only weigh scores on text long enough for the tool to be reliable (300+ words is a reasonable floor across the industry). Second, when tools conflict sharply, downgrade all of them and switch to evidence a detector can't provide.

The honest bottom line

Detectors disagree because they are different instruments with different priorities, not because any of them is fraudulent — and not because any of them is authoritative either. Thresholds are business decisions. Training data ages. Segmentation choices change what "the score" even means. Length and language stretch every weakness wider. Once you see those five gears turning, a screen full of conflicting percentages stops being mysterious and starts being what it actually is: a rough, noisy family of estimates about how machine-typical your prose looks — never a verdict on who wrote it.

For the wider evidence on how accurate these tools really are — vendor claims versus independent studies — start with our AI detector accuracy hub.

FAQ

Why did one detector say 90% AI and another say 10% on the same text? Because they use different decision thresholds, different training data, and different ways of cutting your text into segments. A 90/10 split usually means your text sits in genuinely ambiguous territory — mixed, edited, short, or non-native — where no detector's answer is reliable.

Which detector is the "correct" one when they disagree? None of them, individually. Each is well-calibrated only for its own design goals and its own threshold. Sharp disagreement is itself the finding: it tells you the text can't be reliably measured by these tools, so the honest answer is "uncertain," not whichever score you prefer.

Does running text through more detectors give a more accurate answer? Averaging helps a little at the extremes but can't rescue a borderline document, because the tools share the same blind spots (short text, edited text, non-native English). Five noisy instruments with correlated errors don't add up to one precise instrument.

If my essay passes one detector, will it pass the others? Not necessarily. Thresholds differ, so text that scores 15% on a conservatively tuned education tool can score 70%+ on an aggressively tuned marketer tool. "Passing" only ever means passing one specific tool, at one specific version, on one specific day.

Do AI detectors give different results at different times for the same text? Yes. Vendors retrain models and adjust thresholds without notice, so the same document can score differently across weeks. This is documented behavior across the industry, not a glitch, and it's one more reason single scores shouldn't decide high-stakes outcomes.

Is detector disagreement proof that AI detection doesn't work at all? No. Unedited AI output is caught most of the time by the stronger tools, and obvious human writing usually scores low everywhere. Detection works worst exactly where disagreement is loudest: the blended middle.

What should a teacher do with conflicting detector scores? Treat the conflict as "unmeasurable," not as a menu to choose from. Move to evidence detectors can't fake or miss: document version history, drafts, and a conversation with the student about their sources and reasoning. Several universities, including Vanderbilt in August 2023, stopped using detector scores for enforcement altogether.

Key facts

• Turnitin displays scores of 1–19% as an asterisk, not a number, and its 98% accuracy / <1% false-positive claims apply only above 20% AI share (Turnitin AI writing FAQ).
• OpenAI retired its own AI classifier in July 2023 after it caught only 26% of AI text and falsely flagged 9% of human text (OpenAI announcement).
• Liang et al., Patterns, 2023: 7 detectors falsely flagged human-written TOEFL essays 61.22% of the time on average; 89 of 91 essays were flagged by at least one detector but only 18 by all seven — direct evidence of cross-tool disagreement.
• Vendor minimums vary widely: Turnitin ~300 words; Winston AI 500 characters; Copyleaks 255 characters; Originality.ai 100 words for site scans (vendor documentation, fetched 2026).
• Originality.ai publishes different false-positive rates (0.5% vs 1.5%) at different settings — a visible example of thresholds as a tunable business choice (Originality.ai accuracy page).
• Vanderbilt University disabled Turnitin's AI indicator in August 2023, publishing its false-positive reasoning.
• The Washington Post's April 2023 test found Turnitin struggled most with blended human/AI drafts — the same category where cross-tool disagreement is widest.

Sources

1. Turnitin — AI writing detection FAQ / transparency page (98% claim conditions, asterisk range, ~300-word minimum).
2. OpenAI — announcement retiring its AI text classifier, July 2023 (26% true-positive, 9% false-positive).
3. Liang, W., et al. — "GPT detectors are biased against non-native English writers," Patterns (Cell Press), 2023.
4. Originality.ai — "AI Content Detection Accuracy" study page and help-center minimum word count article (fetched during writing, 2026).
5. GPTZero — FAQ (document- vs paragraph- vs sentence-level accuracy; confidence-band figures; fetched during writing, 2026).
6. Copyleaks — AI Detector page (character minimums, per-language accuracy claims; fetched during writing, 2026).
7. Winston AI — help center, "How to Use Winston AI for Text Analysis" (500-character minimum; fetched during writing, 2026).
8. Sapling — AI Content Detector page (~50-word guidance, accuracy claims; fetched during writing, 2026).
9. Fowler, G. — Washington Post test of Turnitin's detector, April 2023.
10. Vanderbilt University — statement on disabling Turnitin's AI detection, August 2023.
11. BestColleges — interview with Annie Chechitelli, Turnitin chief product officer, April 2023, on deliberate under-flagging.