HeadlinesBriefing favicon HeadlinesBriefing.com

Storage Write Latency Trade-offs

Hacker News •
×

Every storage system balances speed and durability in write operations. Local writes using fdatasync() on NVMe SSDs survive host crashes but fail if the device is lost. Remote storage via object storage or network volumes enhances durability at higher latency.

The choice impacts failure resilience and performance. A local WAL accelerates writes but risks data loss if the log is unreplicated. Systems like AWS S3 Express claim single-digit millisecond latencies, while Turso reported 6.4 ms for 4 KB PUTs.

Sharing flushes across multiple writes improves throughput but complicates failure handling. The article emphasizes distinguishing operational costs from durability guarantees. Separating client PUTs from object PUTs clarifies data paths.

Key decisions involve where to fsync()—memory, local SSD, or remote storage—and how much cleanup the system can tolerate post-success. Newer designs leverage object storage immutability and LSM trees for efficiency. The fsync() syscall’s meaning varies wildly: 1 ms vs. 64x faster times depend on where durability is enforced.

Buffering writes into batches reduces flush frequency but risks data loss if the batch isn’t synced. The database must still manage versions, consistency, and crash recovery alongside storage choices. Understanding these trade-offs is critical for optimizing performance without sacrificing data safety.