HeadlinesBriefing favicon HeadlinesBriefing.com

Rocky Data Launches Rust-Based Control Plane for Data Warehouse Pipelines

Hacker News •
×

Rocky Data introduces a Rust-powered control plane for data warehouse pipelines, enabling teams to maintain existing warehouses like Databricks or Snowflake while modernizing orchestration. The platform provides compile-time safety, column-level lineage, and cost attribution through a DAG-centric architecture. Key entities include the Rust CLI binary, PyPI integration for Python, and VS Code extension for AI-assisted SQL development.

The system decouples governance from warehouse operations via branches (logical schema copies), replay (SQL execution reconstruction), and 8-field audit trails. Column-level lineage traces data transformations through joins and CTEs using Rust's type system, with visualization in VS Code. Compliance features include automated masking policies and retention checks integrated with Unity Catalog and Snowflake's masking frameworks.

Cost attribution is enforced via per-model metrics (bytes, duration) with budget thresholds triggering alerts. The platform supports multi-cloud warehouses through dialect-aware linting and schema-grounded AI suggestions that type-check before execution. Not a warehouse replacement, Rocky focuses on pipeline governance, with Fivetran or Airbyte handling source ingestion.

Open-source under Apache 2.0, Rocky offers prebuilt adapters for major warehouses and integrates natively with Dagster. The project emphasizes practical adoption, with playground environments for immediate testing and detailed documentation at rocky-data.dev. The control plane's design prioritizes blast-radius containment and developer productivity through compile-time validation.