HeadlinesBriefing favicon HeadlinesBriefing.com

LLMs Struggle with Keyword Search and Hallucinations

Hacker News •
×

Users report LLMs fail at generating precise keyword search queries, often producing redundant or incorrect variations like 'nhl toronto scores' or 'nhl hockey toronto scores'. Despite semantic strengths, they lack reliable keyword precision. Hallucinations persist across contexts — inventing game mechanics in OSRS and Anno 1800, fabricating business names, and misinterpreting UI layouts from screenshots.

Models struggle with spatial reasoning in ASCII maps, long-term planning in games like NetHack, and suppressing over-explanation. While tool use and iterative prompting help with compression and accuracy, LLMs still fail to self-correct vague prompts or anticipate how other models might misinterpret them. Writing effective benchmarks remains ironically difficult for them, suggesting a gap in abstraction handling or imaginative reasoning.