HeadlinesBriefing favicon HeadlinesBriefing.com

PicoMQ: Durable HTTP Streams on Object Storage

Hacker News •
×

PicoMQ is a Rust server designed for durable streams, built directly on object storage. It offers cheap, URL-addressable, and granular stream capabilities, including create, append, read, long-poll, and Server-Sent Events (SSE). The system utilizes either the Pico Protocol or the Durable Streams Protocol as its facade layer.

At its core, S3Stream serves as the fundamental stream storage primitive. This component is notably used in AutoMQ and is available as a standalone Rust library. By leveraging this primitive, PicoMQ ensures efficient data persistence and retrieval over standard HTTP interfaces.

For coordination tasks, the system relies on a command log stored in Postgres. This architecture allows for reliable state management and synchronization across distributed nodes without requiring complex dedicated coordination services. The combination of object storage for data and Postgres for control flow provides a robust yet cost-effective solution for streaming needs.

The project was recently shared on Hacker News, where it garnered 49 points and attracted 5 comments. The discussion highlights interest in lightweight, object-storage-backed messaging solutions that simplify infrastructure requirements while maintaining durability.