Getting Started¶

This guide walks you through everything needed to connect to the AWS Databricks Lakehouse and run your first query.

Prerequisites¶

Requirement	Notes
AWS IAM access	You need an IAM identity with permission to assume the Databricks role
Databricks account	Request access from your workspace admin
Python 3.9+	Required for the Databricks CLI and local development
AWS CLI v2	Required to configure credentials

Step 1 — Configure AWS credentials¶

aws configure --profile lakehouse
# Enter: Access Key ID, Secret Access Key, region (e.g. us-east-1), output format (json)

Verify:

aws sts get-caller-identity --profile lakehouse

Step 2 — Install the Databricks CLI¶

pip install databricks-cli

Step 3 — Configure the Databricks CLI¶

databricks configure --token
# Host:  https://<workspace-id>.cloud.databricks.com
# Token: <personal-access-token>

Verify:

databricks clusters list

Step 4 — Clone the notebooks repo (optional)¶

If your team manages notebooks in source control:

git clone https://github.com/taapey-labs/aws-lakehouse-notebooks.git
cd aws-lakehouse-notebooks

Step 5 — Run a test query¶

Open the Databricks SQL editor or a notebook and run:

SELECT current_user(), current_catalog(), current_database();

You should see your user, the default catalog (unity_catalog), and schema.

Environments¶

Environment	Workspace URL	Catalog prefix
Development	`https://dev-<id>.cloud.databricks.com`	`dev_`
Staging	`https://staging-<id>.cloud.databricks.com`	`stg_`
Production	`https://prod-<id>.cloud.databricks.com`	`prd_`

Warning

Production credentials are managed via AWS Secrets Manager. Never store production tokens in plain text or commit them to version control.

Troubleshooting¶

PERMISSION_DENIED on a table : Request access from the Unity Catalog admin. Permissions are granted at the catalog, schema, or table level.

Cluster fails to start : Check the cluster event log in the Databricks UI. Common causes are insufficient EC2 quota or a missing IAM instance profile.

NoCredentialProviders error in AWS CLI : Run aws configure --profile lakehouse again or check that AWS_PROFILE=lakehouse is exported in your shell.