kafka · debezium · delta/iceberg · dbt · spark
Lakehouse modernization blueprint
Reference architecture for migrating a legacy warehouse to a bronze/silver/gold lakehouse — with an interactive diagram you can explore.
The problem
Most scaling companies hit the same wall: an overloaded warehouse doing triple duty as ingestion target, transformation engine, and serving layer. Costs climb linearly with data volume, and every new consumer makes the nightly batch window tighter.
The architecture
Drag the nodes around — this is the reference design I adapt per client:
Three ingestion paths (streaming events, CDC from OLTP, batch from SaaS APIs) land in a bronze layer, get refined through dbt-managed quality gates, and serve BI, ML, and LLM workloads from the same governed gold layer.
The result profile
A typical engagement of this shape bends the cost curve like this:
monthly warehouse spend, usd — projected vs. actual after optimization
Numbers above are illustrative of the engagement pattern; client-specific figures are shared under NDA.