AI safety has a friendly-fire problem. The technology that proves where AI text came from is weakening the technology that checks whether it is true. Watermarking, or embedding a hidden signature in generated text, is used to increase transparency on AI applications (Europe’s EU AI Act Article 50; China’s AI labeling regulation). Besides, hallucination detection tries to flag parts of an answer that the model made up. Regulators are pushing both, but a hidden collision exists inside AI models between transparency and liability.

An LLM computes a probability distribution over the next token. Sometimes that distribution is sharp, like after “The capital of France is,” where “Paris” is forced. Sometimes it is flat, with multiple continuations equally plausible. The flatness is called entropy. Both technologies live at high-entropy positions. A text watermark biases the coin flips using a secret key, as in Kirchenbauer and colleagues' 2023 scheme, which splits vocabulary into green and red lists. The text reads naturally but contains more green tokens.

A detector with the key counts green tokens and computes improbability. Google DeepMind’s Synth ID-Text, in Gemini models, uses tournament sampling. Distortion-free schemes arrange bias so output distribution is unchanged on average. But a watermark can only push where there is room. At low-entropy positions, biasing would mean wrong words and quality collapse. The collision is hidden in model internals but has real consequences for AI safety.