HeadlinesBriefing favicon HeadlinesBriefing.com

Grep Beats LSP: Why Coding Agents Prefer Lexical Search

Hacker News •
×

A study comparing lexical search (grep) with LSP-backed semantic navigation found that coding agents often prefer grep despite LSP's higher precision. Testing three Claude models (Opus 4.8, Sonnet 4.6, Haiku 4.5) across Python and TypeScript repositories, agents chose semantic tools only 0-6% of the time for simple code-location tasks. Forcing semantic-first paths reduced success rates from 100% to 89%.

However, for reference-completeness tasks like finding all callers, models chose semantic navigation 45-57% of the time, achieving 1.00 precision versus grep's 0.76 by eliminating false matches. Recall remained near 0.66 for both, limited by agent thoroughness rather than retrieval precision. Stronger models saw precision gains but higher token usage.

Codebase noise proved decisive: on clean TypeScript repos, LSP provided no F1 gain and used 16% more tokens; on noisy repos, it improved F1 by 0.246 with 12% fewer tokens. The key predictor was lexical noise, not static typing strength. Results suggest tool usability depends on context return format and model familiarity, not just precision.

A model and its familiar tool loop act as one capability surface, routing by task type rather than habit.