| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
| Name | Name | Last commit date | ||
|---|---|---|---|---|
Provisions a Unity Catalog metastore — the account-level root container for every catalog, schema, table, and grant in a Databricks Unity Catalog hierarchy — against the databricks/databricks provider ~> 1.117.0.
💡 Why it matters: every other Unity Catalog object in this library — catalogs, schemas, volumes, external locations, storage credentials, grants — exists inside exactly one metastore. Getting this module's secure defaults right (external access off, destroy protection on, bounded Delta Sharing token lifetimes) sets the security posture for the entire governance chain beneath it.
If these Terraform modules have been helpful to you or your organization, I'd appreciate your support in any of the following ways:
Whether it's a star, a professional connection, or a coffee, every gesture helps keep these modules actively maintained and continually improving. Thank you for being part of the community!
flowchart LR
THIS["terraform-databricks-metastore"]
style THIS fill:#FF3621,color:#fff,stroke:#1B3139,stroke-width:1px
ASSIGN["terraform-databricks-metastore-assignment"]
style ASSIGN fill:#1B3139,color:#fff,stroke:#1B3139,stroke-width:1px
CATALOG["terraform-databricks-catalog"]
SCHEMA["terraform-databricks-schema"]
VOLUME["terraform-databricks-volume"]
style CATALOG fill:#F2F2F2,color:#1B3139,stroke:#CCCCCC,stroke-width:1px
style SCHEMA fill:#F2F2F2,color:#1B3139,stroke:#CCCCCC,stroke-width:1px
style VOLUME fill:#F2F2F2,color:#1B3139,stroke:#CCCCCC,stroke-width:1px
THIS -->|"id becomes metastore_id"| ASSIGN
ASSIGN -.->|"binds metastore to workspace (no direct reference)"| CATALOG
CATALOG --> SCHEMA
SCHEMA --> VOLUME
This module is the account-plane foundation of the Unity Catalog chain (first in this library's recommended authoring order). Its nearest sibling is terraform-databricks-metastore-assignment, which consumes this module's id output directly as metastore_id. Downstream modules (terraform-databricks-catalog, -schema, -volume) depend on a metastore existing and being assigned to their workspace, but do not reference this module's output directly — they operate against whichever metastore is assigned to the workspace their provider targets.
flowchart TB
subgraph INPUTS["var.*"]
NAME["name"]
STORAGE["storage_root / storage_root_credential_id|name"]
REGION["region"]
SHARING["delta_sharing_scope / recipient_token_lifetime / organization_name"]
FLAGS["external_access_enabled / force_destroy / owner"]
end
KEYSTONE["databricks_metastore.this"]
style KEYSTONE fill:#1B3139,color:#fff,stroke:#1B3139,stroke-width:1px
subgraph OUTPUTS["outputs"]
ID["id / metastore_id"]
GLOBAL["global_metastore_id"]
REST["cloud, storage_root_credential_*, default_data_access_config_id, privilege_model_version"]
end
NAME --> KEYSTONE
STORAGE --> KEYSTONE
REGION --> KEYSTONE
SHARING --> KEYSTONE
FLAGS --> KEYSTONE
KEYSTONE --> ID
KEYSTONE --> GLOBAL
KEYSTONE --> REST
Resource inventory: one resource, databricks_metastore.this. No child collection — a metastore's storage-root and Delta Sharing settings are metastore-scoped fields, not independent objects.
| Requirement | Value |
|---|---|
| Terraform | >= 1.12.0 |
| databricks/databricks | ~> 1.117.0 |
| Provider block | None — the caller's root module configures provider "databricks" {} |
| tags / custom_tags | Not supported by databricks_metastore — none added |
| timeouts | Not supported by databricks_metastore — none added |
Schema notes that bite:
terraform-databricks-metastore/
├── providers.tf # required_providers only — no provider {} block
├── variables.tf # name, storage_root(+credential), region, owner, Delta Sharing, flags
├── main.tf # databricks_metastore.this
├── outputs.tf # id first, then name and computed metastore metadata
├── SCOPE.md # cross-module contract
├── README.md # this file
└── examples/
└── basic/
└── main.tf # smallest real, runnable call
module "metastore" {
source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore.git?ref=v1.0.0"
name = "casey-primary-metastore"
region = "eastus"
storage_root = "abfss://unity-catalog@caseyuc.dfs.core.windows.net/"
owner = "uc-admins"
}ℹ️ This is an account-plane module. The caller's root module must configure a provider with host = "https://accounts.azuredatabricks.net" (Azure) or "https://accounts.cloud.databricks.com" (AWS/GCP) and an account_id — this module accepts neither.
Consumes: none — this module is the root of the Unity Catalog chain and has no sibling dependencies.
Emits:
| Output | Description | Consumed by |
|---|---|---|
| id | System-generated metastore ID (identical to metastore_id) | terraform-databricks-metastore-assignment (metastore_id input) |
| name | Metastore name | Operator tooling / auditing |
| metastore_id | Unique identifier of the metastore | Same as id |
| global_metastore_id | Globally unique ID (cloud:region:metastore_id) | Delta Sharing configuration, cross-account tooling |
| cloud | Cloud vendor of the metastore's home shard | Auditing / drift-detection tooling |
| storage_root_credential_id, storage_root_credential_name | Echoed storage-credential reference, if set | Auditing |
| default_data_access_config_id, privilege_model_version | Metastore-managed computed attributes | Auditing |
module "metastore" {
source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore.git?ref=v1.0.0"
name = "casey-minimal-metastore"
}2 · AWS metastore with S3 storage rootℹ️ Every catalog created under this metastore must then define its own storage_root — this library's house recommendation regardless (see terraform-databricks-catalog).
module "metastore" {
source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore.git?ref=v1.0.0"
name = "casey-aws-metastore"
region = "us-east-1"
storage_root = "s3://casey-unity-catalog-metastore/root"
owner = "uc-admins"
}module "metastore" {
source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore.git?ref=v1.0.0"
name = "casey-azure-metastore"
region = "eastus"
storage_root = "abfss://unity-catalog@caseyuc.dfs.core.windows.net/"
owner = "uc-admins"
}module "metastore" {
source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore.git?ref=v1.0.0"
name = "casey-gcp-metastore"
region = "us-east1"
storage_root = "gs://casey-unity-catalog-metastore"
owner = "uc-admins"
}module "metastore" {
source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore.git?ref=v1.0.0"
name = "casey-credentialed-metastore"
region = "eastus"
storage_root = "abfss://unity-catalog@caseyuc.dfs.core.windows.net/"
storage_root_credential_id = "11111111-2222-3333-4444-555555555555"
owner = "uc-admins"
}6 · Delta Sharing enabled — internal only🔒 storage_root_credential_id and storage_root_credential_name are mutually exclusive — this module rejects a call that sets both at plan time.
module "metastore" {
source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore.git?ref=v1.0.0"
name = "casey-sharing-metastore"
region = "eastus"
storage_root = "abfss://unity-catalog@caseyuc.dfs.core.windows.net/"
owner = "uc-admins"
delta_sharing_scope = "INTERNAL"
}7 · Delta Sharing enabled — external, with a bounded token lifetime💡 delta_sharing_recipient_token_lifetime_in_seconds keeps its secure default of 604800 (7 days) even though it has no external-recipient effect under INTERNAL scope.
module "metastore" {
source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore.git?ref=v1.0.0"
name = "casey-external-sharing-metastore"
region = "eastus"
storage_root = "abfss://unity-catalog@caseyuc.dfs.core.windows.net/"
owner = "uc-admins"
delta_sharing_scope = "INTERNAL_AND_EXTERNAL"
delta_sharing_recipient_token_lifetime_in_seconds = 259200 # 3 days
}8 · Databricks-to-Databricks sharing organization name⚠️ INTERNAL_AND_EXTERNAL allows token-based sharing to recipients outside this Databricks account — reserve for a specific, reviewed integration.
module "metastore" {
source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore.git?ref=v1.0.0"
name = "casey-d2d-sharing-metastore"
region = "eastus"
storage_root = "abfss://unity-catalog@caseyuc.dfs.core.windows.net/"
owner = "uc-admins"
delta_sharing_scope = "INTERNAL_AND_EXTERNAL"
delta_sharing_organization_name = "casey-financial-partners"
}9 · External client access enabled (opt-in)⚠️ Once set, delta_sharing_organization_name cannot be removed — only changed to another valid value. Removing it requires tainting and recreating the metastore.
module "metastore" {
source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore.git?ref=v1.0.0"
name = "casey-external-clients-metastore"
region = "eastus"
storage_root = "abfss://unity-catalog@caseyuc.dfs.core.windows.net/"
owner = "uc-admins"
external_access_enabled = true
}10 · Destroy protection explicitly disabled (deliberate teardown)⚠️ Secure default is false. Only set true for a specific, reviewed non-Databricks-Runtime client integration.
module "metastore" {
source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore.git?ref=v1.0.0"
name = "casey-decommission-metastore"
region = "eastus"
force_destroy = true
}11 · Full configuration surface exercised together⚠️ Secure default is false. Only set true immediately before a reviewed, deliberate teardown — never as a standing configuration.
module "metastore" {
source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore.git?ref=v1.0.0"
name = "casey-full-config-metastore"
region = "eastus"
storage_root = "abfss://unity-catalog@caseyuc.dfs.core.windows.net/"
storage_root_credential_id = "11111111-2222-3333-4444-555555555555"
owner = "uc-admins"
delta_sharing_scope = "INTERNAL_AND_EXTERNAL"
delta_sharing_recipient_token_lifetime_in_seconds = 259200
delta_sharing_organization_name = "casey-financial-partners"
external_access_enabled = false
force_destroy = false
}module "metastore" {
source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore.git?ref=v1.0.0"
name = "casey-baseline-metastore"
region = "eastus"
storage_root = "abfss://unity-catalog@caseyuc.dfs.core.windows.net/"
owner = "uc-admins"
# delta_sharing_scope left null: sharing disabled
# external_access_enabled left false (default): no non-DBR client access
# force_destroy left false (default): no accidental teardown
}13 · Multi-region reference (documentation only — one metastore per call)💡 This is the recommended starting configuration for a new metastore in a regulated environment — every optional feature (Delta Sharing, external access) starts off, requiring a deliberate, separate change to enable.
module "metastore_us" {
source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore.git?ref=v1.0.0"
name = "casey-metastore-us-east"
region = "us-east-1"
storage_root = "s3://casey-unity-catalog-metastore-us-east/root"
owner = "uc-admins"
}
module "metastore_eu" {
source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore.git?ref=v1.0.0"
name = "casey-metastore-eu-west"
region = "eu-west-1"
storage_root = "s3://casey-unity-catalog-metastore-eu-west/root"
owner = "uc-admins"
}🏗️ 14 · End-to-end composition — metastore → assignment → catalogℹ️ Each metastore is a fully independent top-level container — there is no cross-metastore reference in this module. Multiple metastores per account are a real, supported topology (e.g. one per major region), each assigned to its own set of workspaces via terraform-databricks-metastore-assignment.
module "metastore" {
source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore.git?ref=v1.0.0"
name = "casey-primary-metastore"
region = "eastus"
storage_root = "abfss://unity-catalog@caseyuc.dfs.core.windows.net/"
owner = "uc-admins"
}
module "metastore_assignment" {
source = "git::https://github.com/microsoftexpert/terraform-databricks-metastore-assignment.git?ref=v1.0.0"
metastore_id = module.metastore.id
workspace_id = 123456789012345
}
module "analytics_catalog" {
source = "git::https://github.com/microsoftexpert/terraform-databricks-catalog.git?ref=v1.0.0"
name = "analytics"
depends_on = [module.metastore_assignment]
}ℹ️ terraform-databricks-metastore-assignment and terraform-databricks-catalog are seeded modules in this same catalog batch — this composition reflects their planned contract (per SCOPE.md), not yet a verified cross-module terraform plan. depends_on is required here because terraform-databricks-catalog has no direct Terraform reference to the metastore — Unity Catalog resolves the catalog against whichever metastore is assigned to the workspace at apply time, an implicit dependency Terraform's graph cannot see from resource references alone.
| Variable | Type | Default | Notes |
|---|---|---|---|
| name | string | — (required) | Metastore name |
| storage_root | string | null | Force-new if changed |
| storage_root_credential_id | string | null | Mutually exclusive with storage_root_credential_name |
| storage_root_credential_name | string | null | Mutually exclusive with storage_root_credential_id |
| region | string | null | Mandatory under an account-level provider |
| owner | string | null | |
| delta_sharing_scope | string | null | INTERNAL | INTERNAL_AND_EXTERNAL |
| delta_sharing_recipient_token_lifetime_in_seconds | number | 604800 | Secure default; API default is 31536000 |
| delta_sharing_organization_name | string | null | Cannot be unset once configured |
| external_access_enabled | bool | false | Secure default |
| force_destroy | bool | false | Secure default |
variable "name" {
type = string
}
variable "storage_root" {
type = string
default = null
}
variable "storage_root_credential_id" {
type = string
default = null
}
variable "storage_root_credential_name" {
type = string
default = null
}
variable "region" {
type = string
default = null
}
variable "owner" {
type = string
default = null
}
variable "delta_sharing_scope" {
type = string
default = null
# validation: must be "INTERNAL" or "INTERNAL_AND_EXTERNAL" when non-null
}
variable "delta_sharing_recipient_token_lifetime_in_seconds" {
type = number
default = 604800
# validation: must be > 0
}
variable "delta_sharing_organization_name" {
type = string
default = null
}
variable "external_access_enabled" {
type = bool
default = false
}
variable "force_destroy" {
type = bool
default = false
}| Output | Description | Sensitive? |
|---|---|---|
| id | System-generated metastore ID (same as metastore_id) | No |
| name | Metastore name | No |
| metastore_id | Unique identifier of the metastore | No |
| global_metastore_id | Globally unique ID (cloud:region:metastore_id) | No |
| cloud | Cloud vendor of the metastore's home shard | No |
| storage_root_credential_id | Echoed storage-credential ID, if set | No |
| storage_root_credential_name | Echoed storage-credential name, if set | No |
| default_data_access_config_id | Metastore's default data access configuration ID, if any | No |
| privilege_model_version | Privilege model version (major.minor) | No |
| Concern | Secure default | Opt-out (caller must set explicitly) |
|---|---|---|
| External client access | external_access_enabled = false | Explicit true, for a specific reviewed integration |
| Destroy protection | force_destroy = false | Explicit true, for a deliberate, reviewed teardown |
| Delta Sharing | Disabled (delta_sharing_scope = null) | Caller sets INTERNAL or INTERNAL_AND_EXTERNAL explicitly |
| Delta Sharing recipient token lifetime | 604800 seconds (7 days), well under the API's 1-year default | Caller sets a longer value explicitly |
These defaults follow this library's Secure-by-default conventions — this module introduced all three (external_access_enabled, force_destroy at the metastore level, and the Delta Sharing token lifetime default) as new instances of that pattern.
cd terraform-databricks-metastore
terraform init -backend=false
terraform validate
terraform fmt -checkPin consumers to an immutable tag — ?ref=v1.0.0 — never a branch. This module is plan-only; a human applies from CI after review, against a sub-production environment first.
terraform validate / terraform fmt -check catch: missing required arguments (name), type mismatches, the storage_root_credential_id/_name mutual-exclusivity validation, the delta_sharing_scope enum validation, and malformed HCL. They do not catch: whether the applying identity actually holds account-admin rights, whether region is genuinely required for the caller's provider plane, whether a referenced storage_root_credential_id exists, or any real Unity Catalog API-side constraint (naming collision, region availability). Those require an actual plan/apply against a live account, which is out of scope for this authoring process.
$ terraform output cloud = "azure" default_data_access_config_id = null global_metastore_id = "azure:eastus:12345678-90ab-cdef-1234-567890abcdef" id = "12345678-90ab-cdef-1234-567890abcdef" metastore_id = "12345678-90ab-cdef-1234-567890abcdef" name = "casey-primary-metastore" privilege_model_version = "1.0" storage_root_credential_id = null storage_root_credential_name = null
| Symptom | Cause | Fix |
|---|---|---|
| terraform validate fails with "storage_root_credential_id or storage_root_credential_name, not both" | Both mutually exclusive credential inputs were set | Remove one — they identify the same storage credential two different ways |
| terraform validate fails on delta_sharing_scope | Value other than INTERNAL / INTERNAL_AND_EXTERNAL | Correct the value; re-verify against the live provider schema if the provider added a new scope value |
| Apply fails with a permissions error even though terraform validate passed | Applying identity is not an account admin | Confirm the identity/service principal running apply holds account-admin rights — validate cannot check live permissions |
| Apply attempts to replace the metastore unexpectedly | storage_root was changed | storage_root is force-new; treat any change as a deliberate, full-hierarchy migration, not a routine edit |
| terraform destroy fails with a "metastore not empty" style error | force_destroy is false (the secure default) and the metastore still has registered catalogs/schemas | Confirm the teardown is intentional, then set force_destroy = true explicitly for that run only |
💙 "Infrastructure as Code should be standardized, consistent, and secure."
| Back | FazBrowse Home | New Git URL |