Pangram: what it measures, and what it proves

Read from pangram.com on 15 August 2026, the headline is “Detect AI-generated content with 99.98% accuracy”, with a more conservative “Pangram is over 99% accurate” in its FAQ. It attributes third-party verification to the University of Chicago and the University of Maryland, and publishes a comparison against GPTZero, Originality.ai and RoBERTa across six document types, attributed to research from Chicago Booth.

On false positives it is specific: the rate “at which human documents are incorrectly flagged as AI” is stated as “currently 1 in 10,000”. The model is described as trained on approximately one million human and AI documents.

Those are the vendor's claims about the vendor's product, and this site does not treat that kind of claim as established — a rule we apply to favourable numbers as well as unfavourable ones. What is different here is the precision. “1 in 10,000” is falsifiable in a way that “industry-leading accuracy” is not, and a company that publishes a checkable number has done something most of this category avoids.

The most current systematic comparison is an NBER working paper by Jabarian and Imas, which ran 1,992 human and 1,992 AI passages across four frontier models plus text put through a commercial humanizer. It reports Pangram reaching “essentially zero FPRs and FNRs” on medium-to-long passages, while placing GPTZero and Originality.ai in a “secondary tier” unsuitable for very short text and “susceptible to ‘humanizers.’”

That is meaningful corroboration and it carries two qualifiers. The paper is a working paper, not peer-reviewed, and states so itself. And its own framing of the field applies to Pangram as much as to anyone: “published claims about detector accuracy are hard to verify: they are often based on private data, focus on a single threshold.”

So the fair summary is that one tool has better independent evidence than the rest of a category whose evidence is generally poor. That is a real distinction and a modest one.

Short text. The documented advantage is on medium-to-long passages. Every detector loses accuracy as text shortens, because there is less to measure, and the tiering in the NBER paper was explicitly about length.

English. The evidence discussed above concerns English text. Nothing here establishes performance on other languages, and the best-known bias finding in this field — 61.22% of essays by non-native speakers flagged by seven detectors — is about writers rather than languages.

It is not what your institution runs. This is the practical limitation that matters most. If you have been flagged, it was almost certainly by Turnitin, GPTZero, Copyleaks or a similar tool, and Pangram's figures say nothing about the report in front of you.

The same thing no detection score proves: who wrote the text. A very low false positive rate makes a flag more informative than one from a weaker tool, and it does not change the category of evidence. The output remains a statistical judgement about how writing reads.

The only approach that records what actually happened at generation time is watermarking, and it covers a narrow slice of text — output from models whose operators chose to mark it.

And the procedural point holds regardless of which tool produced the number: a score is an input to a decision made by people. Whether an AI score should decide a misconduct case has the same answer for an accurate detector as an inaccurate one.