AWS Databricks Lakehouse¶
Welcome to the AWS Databricks Lakehouse documentation hub. This site is the single source of truth for architecture decisions, operational runbooks, and developer guides for the lakehouse platform built on AWS and Databricks.
What is the AWS Databricks Lakehouse?¶
The Lakehouse combines the best of data warehouses and data lakes into a single, open platform:
| Capability | Description |
|---|---|
| Open formats | All data is stored as Delta Lake tables on Amazon S3 — no proprietary lock-in |
| Unified compute | Databricks clusters handle SQL analytics, ML workloads, and streaming pipelines |
| Governance | Unity Catalog provides centralised metadata, fine-grained access control, and data lineage |
| Scalability | AWS auto-scaling + Databricks autoscaling clusters handle petabyte-scale workloads |
Site map¶
- Architecture — high-level design, component diagram, and key design decisions
- OU and Networking — VPC, subnets, NAT Gateway, and security group configurations
- Getting Started — set up your local environment and connect to the platform
- Data Ingestion — patterns for batch and streaming ingestion into the Bronze layer
- Data Transformation — Silver/Gold layer pipelines with Delta Live Tables and dbt
- Security & Governance — IAM roles, Unity Catalog, encryption, and compliance
- Operations & Monitoring — cluster management, cost controls, and observability