HeadlinesBriefing favicon HeadlinesBriefing.com

How PostHog Replaced ANTLR with a 70x Faster SQL Parser

Hacker News •
×

An engineer at PostHog replaced their existing SQL parser to eliminate the overhead of generic graph traversal. The previous system used ANTLR, which relies on a graph-walking interpreter that slows down token processing. By switching to a hand-rolled recursive-descent parser, the team achieved a 70x speed up in processing performance.

Development relied on parallel Claude Code sessions and an "oracle" for validation. The developer used Hypothesis for property-based testing to find edge cases where the new parser diverged from the original C++ version. To avoid brittle fixes, the engineer forced the AI to reload the grammar and source code into its context before every fix.

Testing involved a complex loop of code-coverage-guided generation and production query logs. This process produced 16,000 lines of parser code and 5,000 lines of tooling. The new implementation now runs in shadow mode against the original C++ parser to detect any divergences in real-time production traffic.

This transition proves that AI can maintain complex, hand-rolled parsers that were previously too expensive to build manually.