HeadlinesBriefing favicon HeadlinesBriefing.com

Rayforce: Unified Analytics and Graph Pipeline

Hacker News •
×

Rayforce is an open-source, C-based engine that unifies columnar analytics and graph traversal into a single execution pipeline. Designed for microsecond latency, it eliminates data movement between separate engines by optimizing relational operators and graph traversals together.

The engine's "Lazy DAG" approach allows composition of tables, graphs, and rules, which are then optimized through multi-pass rewriting. Execution involves bytecode generation and parallel, stream-cached processing of data. Rayforce boasts a small footprint with only 16K lines of C code and no external dependencies.

It supports complex queries through the Rayfall language, accessible via an interactive interface, C API, or client connections. The system integrates columnar analytics (vectorized filters, joins, groups), a native graph engine (CSR traversal, shortest paths), and recursive Datalog. Storage options include CSV, splayed tables, memory maps, and IPC.

Rayforce is already deployed in production systems for trading and investment analytics. A managed cloud version, Rayforce Cloud, is also planned. The core engine is open, licensed under MIT, and available on GitHub.