HeadlinesBriefing favicon HeadlinesBriefing.com

Detecting AI-Generated Content Without a Model

Towards Data Science •
×

I was permabanned from a subreddit after posting an article that moderators flagged as AI‑generated, yet the piece was written two years before Chat GPT was public. The ban highlighted a problem: many detection tools have low precision and high recall, classifying almost everything as AI.

Research shows LLMs leave "tells" in vocabulary, rhetoric, punctuation, and names. In a study of 15 million PubMed abstracts, words like delve, intricate, and meticulous spiked after Chat GPT’s release, a phenomenon called excess vocabulary. Another study noted consistent use of certain names—Claude, Elena Vasquez, Marcus Chen—suggesting models develop totale name priors.

LLMs also favor contrastive phrasing such as "It isn’t X— it is Y" and hedging language like "This may indicate…" These patterns arise from RLHF training, making them hard to eliminate even with prompt tweaks.

With these cues, you can move from a vague suspicion to a more informed flag of potential slop, though certainty remains elusive.