Can AI Detectors Detect Code? An Honest Answer for CS Students

Mostly, no. The AI detectors universities run — Turnitin's included — were built for prose, and they do not reliably work on source code. Turnitin says so in its own documentation: the model "does not reliably detect AI-generated text in the form of non-prose, or code," and the company has stated it is "not pursuing ChatGPT code detection at this time." What computer science departments actually run is something older and different: similarity detection, which catches copying between students, not AI generation.

If you searched for an "AI detector for code," you probably fall into one of two groups. Either you're a student wondering whether the Copilot-assisted solution you submitted last night is about to surface in an integrity report, or you're an instructor wondering whether any tool can tell you which of your 200 submissions came out of a chatbot. Both groups deserve a straight answer, and the straight answer is uncomfortable for everyone: the detection tooling that sort-of works for essays works far worse for code, for reasons baked into how code itself is written.

This post walks through why, what tools CS departments actually use, what Turnitin does and doesn't process, and — since detection software isn't the real threat — how AI-written code actually gets caught.

Why prose detectors break on code

To see why an essay detector fails on a Python file, you need to know what the detector is measuring. We've covered the mechanics in depth in how AI detectors work, but the short version is that mainstream detectors lean on two statistical signals.

The first is perplexity — how predictable each next token is, given what came before. AI-generated prose tends to be low-perplexity, because language models literally generate the statistically likely next word. Human prose is messier. We pick odd words, start sentences over, make choices a model wouldn't.

The second is burstiness — variation in sentence length and structure. Humans write in bursts: a long winding sentence, then a short one. Models, left to their defaults, produce more uniform rhythm.

Both signals assume the writer had a choice. That assumption holds, roughly, for an essay on the causes of the First World War, where there are ten thousand defensible ways to phrase any given sentence. It collapses for code.

Code is legitimately formulaic. A for loop in Java looks like every other for loop in Java because the language grammar demands it. There is close to one idiomatic way to open a file in Python, one conventional way to name a loop counter, one standard order for imports. Style guides exist precisely to remove individual variation — PEP 8 for Python, Google's style guides for C++ and Java — and auto-formatters like Black and Prettier then erase whatever variation survived. A well-taught student and a language model, given the same standard exercise, will often produce token streams that are statistically near-identical, because both are converging on the same idiom.

Run the perplexity test on that and you get noise in both directions. Clean, idiomatic human code looks "machine-typical" — low perplexity, low burstiness — and gets flagged. Meanwhile AI-generated code looks exactly like textbook code, because textbook code is what the model was trained on, so there's no distinctive machine fingerprint to find. The statistical daylight between "human" and "AI" that detectors exploit in prose narrows to almost nothing in source code.

There's a second structural problem: correctness dominates. An essay can be phrased infinitely many ways and still be a good essay. A binary search has essentially one shape. When the space of correct answers is small, everyone's answer converges — humans with humans, humans with machines. Detectors that work by measuring deviation from the expected have nothing left to measure.

None of this is speculation about edge cases. It's the same failure mode, amplified, that produces false positives on formulaic human prose. The Liang et al. study published in Patterns in 2023 found that seven GPT detectors falsely flagged an average of 61.22% of essays written by real people who were non-native English speakers — writers whose prose was more constrained and conventional than average. Code is the limiting case of constrained, conventional writing. If rigid five-paragraph essays confuse these tools, a file full of syntax-mandated structure breaks them outright.

What Turnitin actually says about code

Turnitin is unusually plain on this point, and it deserves credit for that. Its AI writing detection FAQ states the model "does not reliably detect AI-generated text in the form of non-prose, or code," excludes lists and bullet fragments from analysis, and requires at least 300 words of long-form prose before it will score a document at all. The detector was built and validated on long-form English (with Spanish and Japanese added later), in standard document formats — .docx, .pdf, .txt, .rtf. And on the direct question, Turnitin's guidance says the company is "not pursuing ChatGPT code detection at this time."

So what happens when a programming assignment goes through Turnitin anyway? Two separate things, and it's worth keeping them apart, because students routinely confuse them.

The AI writing indicator — the percentage introduced in April 2023 that we cover in detail on the Turnitin hub page — either ignores the code portions or fails to produce a meaningful score. If your submission is mostly code with a short written explanation, the prose may fall under the 300-word threshold and generate no AI score at all.

The Similarity Report — Turnitin's original product — is a different animal. It does text matching, not AI detection, and it will happily match code as raw text against other submissions and web sources. That's how a copied Stack Overflow snippet or a classmate's shared solution gets caught in Turnitin: not because any AI was detected, but because the same characters appeared in two places. Text-level matching on code is crude — rename three variables and much of the match evaporates — which is exactly why dedicated code-similarity tools exist.

Turnitin does own a serious code-checking product, just not under the Turnitin name: it acquired Gradescope in 2018, and Gradescope includes a code similarity feature built for programming assignments. Note the word choice. Similarity. Even Turnitin's purpose-built code tool measures resemblance between submissions; it makes no claim to identify AI authorship.

What CS departments actually use

Ask a computer science instructor what they run on submissions and you'll rarely hear "an AI detector." You'll hear one of a handful of similarity tools, most of them decades old.

MOSS (Measure of Software Similarity) is the field's institution. Written by Alex Aiken in 1994 and hosted at Stanford ever since, it's a free service: instructors upload a batch of submissions, MOSS returns a ranked list of suspiciously similar pairs with matched regions highlighted. Under the hood it uses document fingerprinting via the "winnowing" algorithm, published by Schleimer, Wilkerson, and Aiken in 2003, and it works on the structure of the code rather than the raw text — so renaming variables, reordering functions, and reformatting whitespace don't fool it the way they fool a text matcher.

JPlag, developed at Karlsruhe Institute of Technology and actively maintained as an open-source project, does token-based structural comparison across many languages, and has become a common MOSS alternative — partly because MOSS, as a best-effort free service run by one Stanford professor for thirty years, has had reliability stretches.

Gradescope's code similarity feature brings the same idea into a mainstream grading platform, under Turnitin's corporate umbrella.

Commercial entrants like Codequiry and Copyleaks' Codeleaks sell subscription versions, checking submissions against web sources and repositories as well as peer submissions. Copyleaks additionally markets AI-generated source code detection; treat any vendor's accuracy claims on that front with the skepticism you'd apply to prose-detector marketing, because we could find no published independent benchmark validating AI-code detection at institutional scale. Note too that Turnitin's own documentation says its model "does not reliably detect AI-generated text in the form of non-prose, such as poetry, scripts, or code."

The pattern in that last column is the honest state of the field. The tools that work, work on a different problem.

Can similarity checkers catch AI code?

Sometimes — by accident, and in one specific scenario.

Similarity tools compare submissions against each other and against known sources. AI-generated code is, from the tool's perspective, novel: it matches no database entry and no prior submission. A student who generates a solution privately and submits it will typically sail through MOSS untouched. That's not an endorsement; it's an observation about what the tool measures.

The exception is convergence. Give forty students the same tightly specified assignment, and if a dozen of them paste the same prompt into the same model, the model will hand several of them near-identical solutions — same algorithm, same structure, sometimes the same idiosyncratic comment. Those submissions then flag against each other. Students in this situation are genuinely baffled ("I never talked to him!") and technically telling the truth. They didn't collude with a classmate. They colluded with the same machine.

A 2025 paper on arXiv (2505.08244) studying competitive programming courses put the problem bluntly: widely used automated similarity checkers "can be thwarted by simple code transformations or novel AI-generated code." The authors' response is telling — they didn't recommend a better detector. They redesigned assessment around mastery checks and oral interviews, because talking to the student is the one integrity mechanism a language model can't attend on the student's behalf.

Research prototypes for true AI-code detection do exist in the academic literature — classifiers fine-tuned to distinguish Copilot or ChatGPT output from human code in controlled datasets. Reported results look impressive inside the papers and degrade when the model changes, the programming task shifts, or a student lightly edits the output. That trajectory should sound familiar: it's what happened with prose detection, where OpenAI's own classifier caught just 26% of AI text, falsely flagged 9% of human writing, and was retired in July 2023 for low accuracy. Code detection is attempting the same trick on a substrate with less individual variation to work with. Nothing deployed today at institutional scale does it credibly.

How AI code actually gets caught

Here's what should genuinely concern a student leaning on AI in a course that prohibits it — because AI code does get caught, constantly, just not by detectors.

Curriculum mismatch is the loudest signal. When a week-three intro student submits a solution using list comprehensions, decorators, or a library the course hasn't touched, the instructor notices in seconds. Language models write like the median of their training data — experienced, idiomatic, sometimes clever. In an intro course, competence beyond the syllabus is the anomaly.

Comment style gives it away. Models write chatty, tutorial-grade comments: # Initialize the counter variable to zero, docstrings on trivial helpers, section headers narrating obvious steps. Real students under deadline comment sparsely, cryptically, or not at all. A file where every third line is annotated for an imaginary reader reads as generated to anyone who has graded a hundred human submissions.

Version history tells the truth. A human solution accumulates: commits, saves, broken intermediate states, a debugging trail. A pasted solution appears fully formed at 11:47 p.m. Instructors using Git-based workflows, IDE telemetry, or platforms that log keystroke-level activity can see the difference between three hours of iteration and one paste event.

The conversation is the real detector. More CS courses are moving to short code-review interviews: explain your approach, why this data structure, what breaks if the input is empty, change this function live. A student who wrote the code answers imperfectly but recognizably. A student who submitted a machine's code hits a wall on the second question. This is where most confirmed cases actually resolve — not with a probability score, but with a student unable to account for their own work.

If you're facing that situation, or a flag on the written portion of a technical submission, our walkthrough of the academic integrity process for AI cases covers what happens next, step by step.

What this means for students in programming courses

A few honest conclusions, in descending order of importance.

The absence of a reliable AI detector for code is not a safe harbor. If your course bans AI assistance, using it is a violation whether or not software can prove it — and as the section above shows, software is not what usually proves it. We say the same thing about essays and we'll say it about code: if the rules prohibit it, disguising it is a violation, full stop.

Know the actual policy, because it varies wildly. Programming courses are all over the map right now — some ban AI outright, some allow Copilot but not full-solution generation, some require a disclosure comment at the top of the file, some encourage AI freely and test understanding in exams. "The course next door allows it" is not a defense. Read the syllabus; when it's ambiguous, ask in writing and keep the answer.

Build a provenance trail whether or not you use AI. Commit early and often. Keep your scratch work and broken versions. If you're ever wrongly flagged — and false accusations happen in CS too, particularly on the prose portions of reports where detector false positives hit formulaic technical writing hard — your commit history is the strongest evidence a student can have.

Be able to explain every line you submit. This is simultaneously the best learning practice and the best defense. If you can rebuild the logic on a whiteboard, no interview will sink you. If you can't, no clean detector report will save you.

On the flagging risk for prose-around-code: lab reports, READMEs, and design documents are long-form prose, and those do go through AI detection. Technical prose is one of the documented high-risk categories for false flags — it's structured, terminology-dense, and low-perplexity by nature. If you want to see how your own writing scores before an instructor does, a checker like our free AI detector will show you a sentence-level readout. One caveat we attach to our own tool and everyone else's: a detector score, in either direction, is evidence of statistical typicality, not proof of authorship — HumanFlow doesn't promise to predict or beat any detector, because nobody can honestly promise that.

The bottom line

The question "can AI detectors detect code?" has a cleaner answer than most questions in this field. Prose detectors: no, and Turnitin says so itself. Similarity tools: they detect copying, and catch AI only when multiple students converge on the same generated solution. Dedicated AI-code detectors: research prototypes and unverified vendor claims, nothing credible at scale.

And yet AI-written code gets identified every week, in every CS department, by instructors reading code the way they always have and by students who can't explain their own submissions. The detector was never the thing to worry about. The conversation afterward is.

FAQ

Does Turnitin detect AI-generated code? No. Turnitin's documentation states its model "does not reliably detect AI-generated text in the form of non-prose, or code," and that the company is not pursuing ChatGPT code detection at this time. Turnitin's Similarity Report can still match code as plain text against other submissions, but that's copy detection, not AI detection.

Does MOSS detect ChatGPT or Copilot code? Not directly. MOSS compares submissions against each other and flags structural similarity between them. Freshly generated AI code matches nothing — unless several students prompt the same model with the same assignment and receive near-identical solutions, in which case they flag against each other.

Can my professor tell if my code is AI-written without a detector? Often, yes. The common tells are techniques the course hasn't taught yet, tutorial-style over-commenting, a style break from your previous work, and no development history. The most reliable check is a short interview: students who wrote their code can explain and modify it, and students who didn't usually can't.

Do GPTZero or Copyleaks work on code? GPTZero and similar tools were built for prose, and their statistical assumptions (perplexity, burstiness) break down on syntax-constrained code. Copyleaks markets a dedicated AI source code detection feature, but we're not aware of independent published benchmarks validating AI-code detection accuracy at scale, so treat vendor claims cautiously.

What happens if two students submit the same AI-generated code? Similarity tools like MOSS or Gradescope will flag the pair, and both students may face a collusion allegation even though they never communicated. Using the same model on the same prompt is enough to produce matching solutions. This is currently one of the most common ways AI use in programming courses surfaces.

Is it safe to use AI on programming assignments since detectors don't work? No — "hard to detect by software" and "permitted" are different things. If your course bans AI assistance, using it is an integrity violation regardless of tooling, and interviews, version history, and style analysis catch cases regularly. If your course allows it, follow the disclosure rules exactly.

Will the written report attached to my code get AI-flagged? It can. READMEs, lab reports, and design docs are long-form prose and go through normal AI detection — and technical prose is a documented false-positive risk category because it's formulaic by nature. Keep drafts and version history for the written portions of technical work especially.

Key facts

• Turnitin's own FAQ states its model "does not reliably detect AI-generated text in the form of non-prose, or code," and that it is "not pursuing ChatGPT code detection at this time" (Turnitin AI writing detection FAQs).
• Turnitin's AI indicator requires roughly 300 words of long-form prose and was built for English (later Spanish and Japanese) in standard document formats (Turnitin).
• MOSS, the standard code-similarity tool in CS departments, was written by Alex Aiken in 1994 and is hosted at Stanford; its fingerprinting method was published by Schleimer, Wilkerson, and Aiken in 2003.
• Turnitin acquired Gradescope, which includes a code similarity feature for programming assignments, in 2018 (Turnitin press release).
• A 2025 arXiv study of competitive programming courses (2505.08244) found automated similarity checkers "can be thwarted by simple code transformations or novel AI-generated code," and recommended oral interviews over better detectors.
• OpenAI retired its own AI text classifier in July 2023 after it identified only 26% of AI-written text and falsely flagged 9% of human writing (OpenAI).
• Liang et al. (Patterns, 2023) found seven GPT detectors falsely flagged 61.22% of human-written TOEFL essays on average — evidence that formulaic, constrained writing (of which code is the extreme case) breaks detector assumptions.

Sources

1. Turnitin — "Turnitin's AI writing detection capabilities FAQs" (guides.turnitin.com).
2. Stanford University — MOSS (Measure of Software Similarity), theory.stanford.edu/~aiken/moss.
3. Schleimer, S., Wilkerson, D., Aiken, A. — "Winnowing: Local Algorithms for Document Fingerprinting," SIGMOD 2003.
4. JPlag — open-source code similarity project, Karlsruhe Institute of Technology (github.com/jplag/JPlag).
5. Turnitin — "Turnitin Acquires Gradescope" press release, 2018.
6. arXiv:2505.08244 (2025) — on the failure of plagiarism detection in competitive programming.
7. OpenAI — announcement retiring the AI text classifier, July 2023.
8. Liang, W., et al. — "GPT detectors are biased against non-native English writers," Patterns (Cell Press), 2023.