AI Text Watermarking: What It Proves And What It Doesn't

AI text watermarking doesn't touch the characters you see. It lives in the choices between words - secret-keyed mathematical nudges applied at the model level that shift which token gets picked, just slightly, in a pattern invisible to human readers but detectable with the right key. Google has been watermarking text from the Gemini app and web experience since 2024, and Anthropic's Claude models will mark text at the model level starting August 2026. That makes this less a novelty and more a default condition of working with frontier models.

How The Detection Actually Works

A watermark is the opposite of a hidden string you can grep for: it's a deliberate, key-gated statistical test. To check a document, the detector uses a secret key to 're-color' words and count how often specific outcomes appear. Without the key, there's nothing to find - which is good for security and bad for independent verification, because only whoever holds the key can run the test.

The math behind it is where the practical limits show up. A 1,500-word document would flag at only about 55% green, and roughly 0.5% of windows survive as usable signal in short passages. In other words: short texts are unreliable to judge, full stop. The detector does recover given enough text - after roughly 800 tokens (about 600 words) of material, even human paraphrase becomes detectable again.

What Editing Does To The Mark

Editing erases marks wherever wording runs break, so light paraphrasing chips away at detection while full rewrites are far more effective at removing it entirely. That asymmetry matters more than any benchmark number here. Anyone motivated to launder AI output through a competent rewrite defeats the scheme; anyone who just pastes model output into a doc carries it around unknowingly.

Which points to the part worth being skeptical of: this isn't an authorship detector, and treating it like one will produce false accusations. A found mark means 'processed by', not 'written by'. If someone drafts with Claude and then heavily rewrites in their own voice, or bounces text through multiple tools, the signal degrades in ways that have nothing to do with honesty or dishonesty.

The Gap Between Signal And Claim

The real story here isn't that watermarking exists - it's that its failure modes are exactly where people will lean on it hardest. Short texts are precisely what gets submitted as homework or flagged in HR disputes, and that's where detection is weakest (55% green on 1,500 words; ~0.5% of windows surviving). Meanwhile the strongest detections land on long, lightly-edited output - which is also where nobody needed convincing.

My read: model-level watermarking is genuinely useful as provenance infrastructure - knowing what was machine-touched matters for search integrity, content supply chains, and trust signals downstream. What it isn't is an enforcement tool for individual cases. The 'processed by' framing should be printed on every detector UI until people internalize it.

If you're building anything downstream of generated text - moderation pipelines, plagiarism tooling, content licensing - plan for watermark signals to be probabilistic inputs into a decision, not verdicts. And if your workflow involves heavy editing of AI-drafted text anyway (it should), expect your own output to look increasingly like everyone else's: partially marked or not marked at all.