HeadlinesBriefing favicon HeadlinesBriefing.com

Medallion Data Architecture: Bronze, Silver, Gold

Towards Data Science •
×

The Medallion Data Architecture is a practical approach to managing data platforms, dividing them into three distinct layers: Bronze, Silver, and Gold. This structure aims to improve data trustworthiness, maintainability, and debugging capabilities as data grows in scope.

Originating from Databricks, the Medallion architecture progressively enhances data quality. The Bronze layer stores raw, unfiltered data as it arrives from the source, typically in an immutable, append-only format. It captures comprehensive ingestion metadata. The Silver layer refines the Bronze data by cleaning, validating, and standardizing it, addressing issues like null records or invalid dates, and deduplicating entries. This layer serves as the clean, foundational data for downstream systems.

The Gold layer organizes data around specific business use cases, often featuring summarized and aggregated datasets optimized for fast querying. Examples include star schemas or data marts tailored for teams like finance or marketing. While such layered systems have existed in data engineering for a long time, Databricks popularized the Bronze, Silver, and Gold terminology. The implementation typically involves different schemas within a database or folders in object storage, utilizing tools like SQL, Python, and open table formats such as Hudi, Apache Iceberg, and Delta tables.