Pinterest rebuilt its async compute platform from scratch to handle billions of daily jobs. The original system, Pinlater, faced severe lock contention and poor isolation. A new service, Pacer, introduces a dedicated dequeue broker to eliminate database scanning and improve reliability for critical workflows.

Pinlater’s architecture scattered partitions across every database shard, causing constant lock battles. Different job queues shared worker machines, so a bug in one could crash everything. The system also couldn’t guarantee FIFO ordering. These bottlenecks made performance tuning nearly impossible as Pinterest scaled.

Pacer assigns each queue partition to a single dequeue broker instance managed by Helix. Brokers cache jobs in memory, serving workers directly and removing database competition. This redesign provides isolation between queues and supports FIFO ordering. The overhaul reflects a common industry shift toward dedicated orchestration layers for high-throughput task processing.