HeadlinesBriefing favicon HeadlinesBriefing.com

Rust replaces Protobuf for 5x faster PostgreSQL parsing

Hacker News: Front Page •
×

PgDog, a PostgreSQL scaling proxy, replaced Protobuf serialization with direct Rust-to-C bindings, achieving a 5.45x speedup in query parsing and 9.64x improvement in deparsing. The project uses `libpg_query` and found Protobuf overhead was the bottleneck after profiling with samply. This optimization directly benefits their sharding and query rewrite engine performance.

The team first tried caching parsed queries but faced issues with ORM bugs and legacy drivers. They then used `bindgen` and AI assistance from Claude to manually map C structs to Rust. The recursive conversion code, while using `unsafe` functions, eliminated serialization overhead and passed strict verification tests against the original Protobuf implementation.

While the gains are substantial, this approach requires deep expertise in both Rust and PostgreSQL internals. It highlights a broader trend where removing serialization layers can unlock performance, but demands careful, machine-verifiable engineering. Future work may involve extending these bindings to other languages or further optimizing the AST traversal.