HeadlinesBriefing favicon HeadlinesBriefing.com

Fake Illness Created to Test LLM Accuracy

Hacker News •
×

A medical researcher from the University of Gothenburg fabricated a non‑existent eye condition called bixonimania to probe how large language models handle misinformation. By uploading two preprints with doctored authorship and absurd funding claims, she triggered mainstream AI chatbots to repeat the claim as legitimate.

Within weeks, Microsoft Bing’s Copilot, Google Gemini, Perplexity, and OpenAI’s ChatGPT echoed the invented diagnosis, citing prevalence figures and advising ophthalmologist visits. The spread exposed a flaw in LLM training pipelines that rely on web‑scraped corpora without rigorous source vetting.

Scholars now warn that the incident illustrates how easily AI can amplify fabricated research, especially when peer‑reviewed literature cites unverified preprints. The case underscores the need for stricter validation layers in conversational models.

Ultimately, the experiment proves that LLMs can propagate false medical claims at scale, highlighting a critical vulnerability that must be addressed before such tools can be safely used for health advice.