HeadlinesBriefing favicon HeadlinesBriefing.com

Why ClickHouse Rewrote Postgres Backup Tools in Rust

Hacker News •
×

ClickHouse engineers developed WAL-RUS, a new open-source tool designed to replace the Go-based WAL-G for Postgres backups. While WAL-G is a reliable industry standard, its garbage-collected runtime creates unpredictable memory spikes. This makes it difficult for operators to provision resources without over-allocating memory that could otherwise be used for database queries.

By switching to Rust, the team achieved much tighter control over memory allocation and concurrency. In internal benchmarks, WAL-RUS maintained a memory footprint below 1 GB, representing a 70% reduction compared to the 2.8 GB peak seen with WAL-G. This predictable profile allows for better capacity planning in resource-constrained environments.

Architecturally, WAL-RUS uses a daemonized approach that maintains persistent connections to object storage rather than spawning new processes for every file. The tool remains fully compatible with existing WAL-G configurations and archives. ClickHouse plans to use this tool as the default backup mechanism for its managed Postgres offering.