HeadlinesBriefing favicon HeadlinesBriefing.com

PlanetScale: Scaling to 768 Servers with Sharding

Hacker News •
×

Database sharding is the proven way to scale Postgres or MySQL beyond a few terabytes. A single database server hits CPU, I/O, and write-throughput bottlenecks that read replicas cannot solve — writes remain limited to one primary, replicas don't expand data capacity, and backups become unacceptably slow.

Sharding distributes data and queries across many distinct primaries. For example, 2 TB fits on four shards; a 1 petabyte dataset requires 256 shards each with a primary and two replicas, totaling 768 servers. Without abstraction, this complexity burdens application backends: routing queries, handling cross-shard transactions, backups, and failure recovery.

The solution is a proxy layer that makes thousands of servers appear as one cohesive database. Neki for Postgres and Vitess for MySQL act as intelligent routers, parsing SQL, applying the sharding strategy (e.g., hash-based), and directing each query to the correct shard. Unlike simple connection poolers such as PgBouncer, these routers understand data distribution, enabling applications to use a single connection string while the infrastructure scales horizontally.