{"id":25693246,"url":"https://github.com/data-platform-hq/terraform-databricks-databricks-runtime","last_synced_at":"2025-04-24T06:26:31.930Z","repository":{"id":65608611,"uuid":"554886461","full_name":"data-platform-hq/terraform-databricks-databricks-runtime","owner":"data-platform-hq","description":"Terraform module for managing Databricks Workspace using Databricks provider","archived":false,"fork":false,"pushed_at":"2024-09-10T08:22:42.000Z","size":71,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-30T07:41:26.713Z","etag":null,"topics":["azure","databricks","databricks-workspace","terraform-modules"],"latest_commit_sha":null,"homepage":"https://registry.terraform.io/modules/data-platform-hq/databricks-runtime/databricks/latest","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/data-platform-hq.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-10-20T15:09:57.000Z","updated_at":"2024-09-10T08:20:24.000Z","dependencies_parsed_at":"2024-02-26T10:26:21.822Z","dependency_job_id":"c2a6165b-9566-4898-b588-f3c73fbe3d0b","html_url":"https://github.com/data-platform-hq/terraform-databricks-databricks-runtime","commit_stats":null,"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/data-platform-hq%2Fterraform-databricks-databricks-runtime","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/data-platform-hq%2Fterraform-databricks-databricks-runtime/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/data-platform-hq%2Fterraform-databricks-databricks-runtime/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/data-platform-hq%2Fterraform-databricks-databricks-runtime/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/data-platform-hq","download_url":"https://codeload.github.com/data-platform-hq/terraform-databricks-databricks-runtime/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250575583,"owners_count":21452795,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure","databricks","databricks-workspace","terraform-modules"],"created_at":"2025-02-24T23:34:57.967Z","updated_at":"2025-04-24T06:26:31.884Z","avatar_url":"https://github.com/data-platform-hq.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Databricks Workspace Terraform module\nTerraform module used for Databricks Workspace configuration and Resources creation\n\n## Usage\nThis module provides an ability for Databricks Workspace configuration and Resources creation, for example:\n1. Default Shared Autoscaling cluster\n2. ADLS Gen2 Mount\n3. Secret scope and its secrets\n4. Users for Standard SKU Worspaces\n\n```hcl\n# Prerequisite resources\ndata \"azurerm_databricks_workspace\" \"example\" {\n  name                = \"example-workspace\"\n  resource_group_name = \"example-rg\"\n}\n\n# Databricks Provider configuration\nprovider \"databricks\" {\n  alias                       = \"main\"\n  host                        = data.azurerm_databricks_workspace.example.workspace_url\n  azure_workspace_resource_id = data.azurerm_databricks_workspace.example.id\n}\n\n# Key Vault which contains Service Principal credentials (App ID and Secret) for mounting ADLS Gen 2\ndata \"azurerm_key_vault\" \"example\" {\n  name                = \"example-key-vault\"\n  resource_group_name = \"example-rg\"\n}\n\ndata \"azurerm_storage_account\" \"example\" {\n  name                = \"examplestorage\"\n  resource_group_name = \"example-rg\"\n}\n\n# Databricks Runtime module usage example\nmodule \"databricks_runtime_core\" {\n  source  = \"data-platform-hq/databricks-runtime/databricks\"\n\n  sku          = \"standard\"\n  workspace_id = data.azurerm_databricks_workspace.example.workspace_id\n  \n  # Databricks user\n  users        = [\"user1\", \"user2\"]  \n\n  # Default cluster parameters\n  custom_default_cluster_name  = \"databricks_example_custer\"\n  cluster_nodes_availability   = \"SPOT_AZURE\" # it required to increase Regional Spot quotas  \n  cluster_log_conf_destination = \"dbfs:/cluster-logs\"\n\n  # Parameters of Service principal used for ADLS mount\n  # Imports App ID and Secret of Service Principal from target Key Vault\n  key_vault_id             =  data.azurerm_key_vault.example.id\n  sp_client_id_secret_name = \"sp-client-id\" # secret's name that stores Service Principal App ID\n  sp_key_secret_name       = \"sp-key\" # secret's name that stores Service Principal Secret Key\n  tenant_id_secret_name    = \"infra-arm-tenant-id\" # secret's name that stores tenant id value\n\n  # Additional Secret Scope\n  secret_scope = [{\n    scope_name = \"extra-scope\"\n    acl        = null # Only group names are allowed. If left empty then only Workspace admins could access these keys\n    secrets    = [\n      { key = \"secret-name\", string_value = \"secret-value\"}\n    ]\n  }]\n\n  mountpoints = {\n    storage_account_name = data.azurerm_storage_account.example.name\n    container_name       = \"example_container\"\n  }\n\n  providers = {\n    databricks = databricks.main\n  }\n}\n```\n\u003c!-- BEGIN_TF_DOCS --\u003e\n## Requirements\n\n| Name | Version |\n|------|---------|\n| \u003ca name=\"requirement_terraform\"\u003e\u003c/a\u003e [terraform](#requirement\\_terraform) | \u003e=1.0.0 |\n| \u003ca name=\"requirement_azurerm\"\u003e\u003c/a\u003e [azurerm](#requirement\\_azurerm) | \u003e= 4.0.1 |\n| \u003ca name=\"requirement_databricks\"\u003e\u003c/a\u003e [databricks](#requirement\\_databricks) | \u003e=1.9.2 |\n\n## Providers\n\n| Name | Version |\n|------|---------|\n| \u003ca name=\"provider_azurerm\"\u003e\u003c/a\u003e [azurerm](#provider\\_azurerm) | \u003e= 4.0.1 |\n| \u003ca name=\"provider_databricks\"\u003e\u003c/a\u003e [databricks](#provider\\_databricks) | \u003e=1.9.2 |\n\n## Modules\n\nNo modules.\n\n## Resources\n\n| Name | Type |\n|------|------|\n| [azurerm_role_assignment.this](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/role_assignment) | resource |\n| [databricks_cluster.this](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/cluster) | resource |\n| [databricks_mount.adls](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/mount) | resource |\n| [databricks_secret.main](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/secret) | resource |\n| [databricks_secret.this](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/secret) | resource |\n| [databricks_secret_scope.main](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/secret_scope) | resource |\n| [databricks_secret_scope.this](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/secret_scope) | resource |\n| [databricks_token.pat](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/token) | resource |\n| [databricks_user.this](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/user) | resource |\n| [azurerm_key_vault_secret.sp_client_id](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/data-sources/key_vault_secret) | data source |\n| [azurerm_key_vault_secret.sp_key](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/data-sources/key_vault_secret) | data source |\n| [azurerm_key_vault_secret.tenant_id](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/data-sources/key_vault_secret) | data source |\n\n## Inputs\n\n| Name | Description | Type | Default | Required |\n|------|-------------|------|---------|:--------:|\n| \u003ca name=\"input_autotermination_minutes\"\u003e\u003c/a\u003e [autotermination\\_minutes](#input\\_autotermination\\_minutes) | Automatically terminate the cluster after being inactive for this time in minutes. If not set, Databricks won't automatically terminate an inactive cluster. If specified, the threshold must be between 10 and 10000 minutes. You can also set this value to 0 to explicitly disable automatic termination. | `number` | `15` | no |\n| \u003ca name=\"input_cluster_log_conf_destination\"\u003e\u003c/a\u003e [cluster\\_log\\_conf\\_destination](#input\\_cluster\\_log\\_conf\\_destination) | Provide a dbfs location to push all cluster logs to certain location | `string` | `\"\"` | no |\n| \u003ca name=\"input_cluster_nodes_availability\"\u003e\u003c/a\u003e [cluster\\_nodes\\_availability](#input\\_cluster\\_nodes\\_availability) | Availability type used for all subsequent nodes past the first\\_on\\_demand ones: [SPOT\\_AZURE\\|SPOT\\_WITH\\_FALLBACK\\_AZURE\\|ON\\_DEMAND\\_AZURE] | `string` | `null` | no |\n| \u003ca name=\"input_custom_default_cluster_name\"\u003e\u003c/a\u003e [custom\\_default\\_cluster\\_name](#input\\_custom\\_default\\_cluster\\_name) | Databricks cluster name, which does not have to be unique | `string` | `null` | no |\n| \u003ca name=\"input_data_security_mode\"\u003e\u003c/a\u003e [data\\_security\\_mode](#input\\_data\\_security\\_mode) | Security features of the cluster | `string` | `\"NONE\"` | no |\n| \u003ca name=\"input_first_on_demand\"\u003e\u003c/a\u003e [first\\_on\\_demand](#input\\_first\\_on\\_demand) | The first first\\_on\\_demand nodes of the cluster will be placed on on-demand instances: [[:number]] | `number` | `0` | no |\n| \u003ca name=\"input_key_vault_id\"\u003e\u003c/a\u003e [key\\_vault\\_id](#input\\_key\\_vault\\_id) | ID of the Key Vault instance where the Secret resides | `string` | n/a | yes |\n| \u003ca name=\"input_max_workers\"\u003e\u003c/a\u003e [max\\_workers](#input\\_max\\_workers) | The maximum number of workers to which the cluster can scale up when overloaded. max\\_workers must be strictly greater than min\\_workers. | `number` | `2` | no |\n| \u003ca name=\"input_min_workers\"\u003e\u003c/a\u003e [min\\_workers](#input\\_min\\_workers) | The minimum number of workers to which the cluster can scale down when underutilized. It is also the initial number of workers the cluster will have after creation. | `number` | `1` | no |\n| \u003ca name=\"input_mountpoints\"\u003e\u003c/a\u003e [mountpoints](#input\\_mountpoints) | Mountpoints for databricks | \u003cpre\u003emap(object({\u003cbr\u003e    storage_account_name = string\u003cbr\u003e    container_name       = string\u003cbr\u003e  }))\u003c/pre\u003e | `{}` | no |\n| \u003ca name=\"input_node_type\"\u003e\u003c/a\u003e [node\\_type](#input\\_node\\_type) | Databricks\\_node\\_type id | `string` | `\"Standard_D3_v2\"` | no |\n| \u003ca name=\"input_pat_token_lifetime_seconds\"\u003e\u003c/a\u003e [pat\\_token\\_lifetime\\_seconds](#input\\_pat\\_token\\_lifetime\\_seconds) | The lifetime of the token, in seconds. If no lifetime is specified, the token remains valid indefinitely | `number` | `315569520` | no |\n| \u003ca name=\"input_permissions\"\u003e\u003c/a\u003e [permissions](#input\\_permissions) | Databricks Workspace permission maps | `list(map(string))` | \u003cpre\u003e[\u003cbr\u003e  {\u003cbr\u003e    \"object_id\": null,\u003cbr\u003e    \"role\": null\u003cbr\u003e  }\u003cbr\u003e]\u003c/pre\u003e | no |\n| \u003ca name=\"input_secret_scope\"\u003e\u003c/a\u003e [secret\\_scope](#input\\_secret\\_scope) | Provides an ability to create custom Secret Scope, store secrets in it and assigning ACL for access management\u003cbr\u003escope\\_name - name of Secret Scope to create;\u003cbr\u003eacl - list of objects, where 'principal' custom group name, this group is created in 'Premium' module; 'permission' is one of \"READ\", \"WRITE\", \"MANAGE\";\u003cbr\u003esecrets - list of objects, where object's 'key' param is created key name and 'string\\_value' is a value for it; | \u003cpre\u003elist(object({\u003cbr\u003e    scope_name = string\u003cbr\u003e    acl = optional(list(object({\u003cbr\u003e      principal  = string\u003cbr\u003e      permission = string\u003cbr\u003e    })))\u003cbr\u003e    secrets = optional(list(object({\u003cbr\u003e      key          = string\u003cbr\u003e      string_value = string\u003cbr\u003e    })))\u003cbr\u003e  }))\u003c/pre\u003e | \u003cpre\u003e[\u003cbr\u003e  {\u003cbr\u003e    \"acl\": null,\u003cbr\u003e    \"scope_name\": null,\u003cbr\u003e    \"secrets\": null\u003cbr\u003e  }\u003cbr\u003e]\u003c/pre\u003e | no |\n| \u003ca name=\"input_single_user_name\"\u003e\u003c/a\u003e [single\\_user\\_name](#input\\_single\\_user\\_name) | single user cluster mode | `string` | `null` | no |\n| \u003ca name=\"input_sp_client_id_secret_name\"\u003e\u003c/a\u003e [sp\\_client\\_id\\_secret\\_name](#input\\_sp\\_client\\_id\\_secret\\_name) | The name of Azure Key Vault secret that contains ClientID of Service Principal to access in Azure Key Vault | `string` | n/a | yes |\n| \u003ca name=\"input_sp_key_secret_name\"\u003e\u003c/a\u003e [sp\\_key\\_secret\\_name](#input\\_sp\\_key\\_secret\\_name) | The name of Azure Key Vault secret that contains client secret of Service Principal to access in Azure Key Vault | `string` | n/a | yes |\n| \u003ca name=\"input_spark_conf\"\u003e\u003c/a\u003e [spark\\_conf](#input\\_spark\\_conf) | Map with key-value pairs to fine-tune Spark clusters, where you can provide custom Spark configuration properties in a cluster configuration. | `map(any)` | `{}` | no |\n| \u003ca name=\"input_spark_env_vars\"\u003e\u003c/a\u003e [spark\\_env\\_vars](#input\\_spark\\_env\\_vars) | Map with environment variable key-value pairs to fine-tune Spark clusters. Key-value pairs of the form (X,Y) are exported (i.e., X='Y') while launching the driver and workers. | `map(any)` | `{}` | no |\n| \u003ca name=\"input_spark_version\"\u003e\u003c/a\u003e [spark\\_version](#input\\_spark\\_version) | Runtime version | `string` | `\"11.3.x-scala2.12\"` | no |\n| \u003ca name=\"input_spot_bid_max_price\"\u003e\u003c/a\u003e [spot\\_bid\\_max\\_price](#input\\_spot\\_bid\\_max\\_price) | The max price for Azure spot instances. Use -1 to specify lowest price. | `number` | `-1` | no |\n| \u003ca name=\"input_tenant_id_secret_name\"\u003e\u003c/a\u003e [tenant\\_id\\_secret\\_name](#input\\_tenant\\_id\\_secret\\_name) | The name of Azure Key Vault secret that contains tenant ID secret of Service Principal to access in Azure Key Vault | `string` | n/a | yes |\n| \u003ca name=\"input_users\"\u003e\u003c/a\u003e [users](#input\\_users) | List of users to access Databricks | `list(string)` | `[]` | no |\n| \u003ca name=\"input_workspace_id\"\u003e\u003c/a\u003e [workspace\\_id](#input\\_workspace\\_id) | Databricks Workspace ID | `string` | n/a | yes |\n\n## Outputs\n\n| Name | Description |\n|------|-------------|\n| \u003ca name=\"output_cluster_id\"\u003e\u003c/a\u003e [cluster\\_id](#output\\_cluster\\_id) | Databricks Cluster Id |\n| \u003ca name=\"output_token\"\u003e\u003c/a\u003e [token](#output\\_token) | Databricks Personal Authorization Token |\n\u003c!-- END_TF_DOCS --\u003e\n\n## License\n\nApache 2 Licensed. For more information please see [LICENSE](https://github.com/data-platform-hq/terraform-databricks-databricks-runtime/blob/main/LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdata-platform-hq%2Fterraform-databricks-databricks-runtime","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdata-platform-hq%2Fterraform-databricks-databricks-runtime","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdata-platform-hq%2Fterraform-databricks-databricks-runtime/lists"}