Zero-Shot AI Detection: DetectGPT and Its Descendants

Most people assume AI detectors work like spam filters: feed them a mountain of labeled AI text and human text, and let a classifier learn the difference. The most interesting family of detectors never saw a labeled example at all. The assumption has a real grain of truth, though, because the majority of commercial tools you can buy today do work roughly that way, and that shared design is exactly why they share the same failure modes.

Zero shot AI detection skips the training set. Instead of learning what machine writing looks like from examples, methods in the DetectGPT lineage measure how a passage's probability behaves when you nudge it, exploiting a geometric quirk: generated text tends to sit on a local peak of a model's probability surface, and human text usually does not.

How zero-shot AI detection actually works

Start with what a language model is. It reads a partial sentence and produces a probability for every possible next token. That is the entire machine. Text generation is just repeatedly sampling from those probabilities, usually favoring the likely options.

A trained detector ignores that machinery and looks at surface features it learned to associate with AI text. A training-free detector goes straight to the source: it asks a reference model how probable your exact passage is, token by token, and reasons from the shape of the answer.

The earliest public version of this idea was a visualization rather than a verdict. GLTR, released in 2019, colored each word by how high it ranked in the model's predicted distribution. Machine text lit up green almost everywhere, because generation keeps picking from the top of the list. Human text was speckled with reds and purples, the low-probability word choices people make constantly without noticing. No training data. Just a model and a highlighter.

The curvature intuition, without the calculus

DetectGPT's contribution was to notice that raw probability alone is a weak signal, and to find a better one hiding next to it.

Picture the model's probability estimate as a hilly surface, where every possible passage of text is a point and its height is how likely the model thinks it is. Generated text does not land randomly on this surface. Sampling pushes it toward peaks, because the generator was choosing likely tokens the whole way. Human writing lands wherever the writer's intent put it, which is usually on a slope.

Now the test. Take the passage and make many small rewrites that preserve meaning: swap a word, restructure a clause. Score each variant with the same model.

• If the original was sitting on a peak, nearly every rewrite is less probable. Move in any direction from a summit and you go downhill.
• If the original was on a slope, rewrites scatter. Some go up, some go down, and the average barely moves.

So the signal is not "this text is too probable." It is "this text is more probable than everything nearby," which is a much sharper thing to measure. That local-peak property is what curvature means here, and it is the whole idea.

Why training-free methods travel better across models

Here is the practical payoff. A trained classifier learns the habits of the specific models in its training set. Ship a new model family, or hand it output from a community fine-tune, and it is guessing about a genre it never read. We cover that drift in detail in open-source LLM detection, and it is the central weakness of the commercial mainstream.

Curvature methods have no training set to go stale. They only need a reference model whose probability estimates roughly resemble the generator's. Because transformer language models trained on overlapping web-scale data end up with broadly similar views of what ordinary English looks like, a decent reference model gives usable signal on text from a generator it has never encountered.

That is a genuinely different failure profile, which is why serious detection pipelines run both kinds of signal instead of picking a side. Our own scans combine multiple engines and report where they disagree, with the weighting written up on our methodology page.

The four costs nobody puts in the abstract

Cost	What it means in practice
Model access	You need probability outputs from a reference model, not just its text
Compute and latency	Many scored variants per document instead of one pass
Paraphrase brittleness	Rewriting moves text off the peak the method looks for
Threshold calibration	The score is continuous; someone still has to pick the cutoff

The first is the quiet one. Commercial APIs vary in how much probability detail they expose, so most implementations use an open-weight proxy model and hope its view of language is close enough to the generator's. When the proxy is a poor match, the measurement degrades in ways the output does not announce.

The third is the one that actually decides outcomes. In the RAID benchmark, across more than 10 million documents and 12 adversarial attacks, commercial detectors degraded sharply under paraphrase and homoglyph attacks. Curvature methods are not exempt; a paraphraser's entire job is to move text away from wherever it was sitting. If you want the fuller treatment, we wrote it up in do AI detectors catch paraphrased text.

The second cost has a shape worth picturing. A trained classifier reads your document once. A curvature method reads a whole neighborhood of rewritten variants, each one requiring its own pass through a language model. That is not a small constant factor, and it is why these methods show up inside enterprise pipelines and research papers rather than behind free web forms that promise instant results.

The fourth is where institutions get hurt. A continuous score means nothing until someone picks a threshold, and thresholds have body counts. Vanderbilt did that arithmetic in 2023: at a 1% false-positive rate, its 75,000 papers a year would produce roughly 750 wrongly flagged students, which is why it disabled Turnitin's AI detector. The 2025 NBER working paper by Jabarian and Imas found that only one tested commercial detector met a strict 0.5% false-positive policy cap, at per-detection costs of $0.02 to $0.06.

The counter-argument: isn't this just perplexity with extra steps?

It is a fair challenge, and worth taking seriously rather than waving away. Perplexity-based detection also measures how surprising text is to a reference model. Curvature adds a second-order comparison against neighbors. Is that really a different method or a more expensive one?

Two things separate them. Perplexity has a fixed reference point, so it inherits every bias in the reference model's idea of normal English. That is one mechanism behind the 2023 Stanford finding in Patterns that seven detectors flagged an average of 61.3% of TOEFL essays by non-native English speakers, with one flagging 97.8%, while performing near-perfectly on US eighth-grade native essays. Unusual-to-the-model gets read as machine-written, and second-language prose is unusual to the model.

Curvature is a relative measurement. It compares a passage to its own neighborhood, which partially cancels the "this writer is unusual" effect that produces those false positives. Partially. Nobody has demonstrated it eliminates the bias, and claiming otherwise would be exactly the overreach this field keeps making.

So: more expensive, yes. Also structurally different in a way that matters for the population most often harmed. Both are true, and our explainer on perplexity and burstiness covers the first-order version in depth.

Where zero-shot fits in a real pipeline

If you are evaluating tools rather than building them, this section is the one that pays. Ask a vendor these five questions and you learn more than any accuracy chart will tell you:

1. What signals do you combine, and does any of them work on models released after your last training run?
2. What reference model do you use, and what happens when the text came from something very different?
3. What is your false-positive rate on second-language and formal-register human writing, specifically?
4. What threshold does the displayed verdict use, and can I see the underlying continuous score?
5. What does your tool do when the sample is too short to score honestly?

Question five is the sleeper. A tool that scores a 90-word paragraph with the same interface it uses for a 3,000-word essay is quietly telling you it does not model its own uncertainty, and short samples are where most real disputes start.

Any vendor who answers all five plainly is worth your money. Any vendor who answers "our accuracy is 99%" to all five is selling a number. AI Detector 360 publishes sentence-level heatmaps and explicit confidence levels rather than a bare percentage for exactly this reason, and you can pressure-test it on your own writing with the free AI detector, 5,000 characters and no sign-up, or on model-specific text through our ChatGPT detector. The mechanics behind all of it are in how AI detectors work.

What nobody can verify yet

There is no public, continuously updated benchmark that scores current commercial tools on how much training-free signal they actually use, so when a vendor says "our approach generalizes to new models," you cannot check it. We include ourselves in that: the claim is testable in principle and untested in public.

Nor can anyone tell you how curvature methods perform against the newest generation of models, because the reference-model mismatch grows quietly as generators pull ahead of the open-weight models used as proxies. The direction of that drift is predictable. The size of it is not published.

One benchmark has held steady while automated methods argued among themselves. In the 2025 ACL study by Russell, Karpinska and Iyyer, expert annotators who frequently use ChatGPT identified AI-generated text with 99.3% accuracy, and stayed robust against evasion tactics that beat automated tools. The most reliable detector in the published literature is still a person who reads a lot. Machines that reason about probability surfaces are catching up, and they are not there.