{"id":26575888,"url":"https://github.com/tomarv2/terraform-databricks-aws-workspace","last_synced_at":"2025-03-23T02:35:41.085Z","repository":{"id":42075720,"uuid":"342073273","full_name":"tomarv2/terraform-databricks-aws-workspace","owner":"tomarv2","description":"Terraform module to create Databricks AWS E2 workspace","archived":false,"fork":false,"pushed_at":"2022-07-27T17:25:05.000Z","size":1199,"stargazers_count":9,"open_issues_count":0,"forks_count":7,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-22T15:49:52.628Z","etag":null,"topics":["aws","databricks","databricks-account","databricks-e2-workspace","databricks-e2-workspaces","terraform","terraform-module"],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tomarv2.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-02-25T00:20:45.000Z","updated_at":"2023-09-27T04:19:01.000Z","dependencies_parsed_at":"2022-08-12T04:10:32.284Z","dependency_job_id":null,"html_url":"https://github.com/tomarv2/terraform-databricks-aws-workspace","commit_stats":null,"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomarv2%2Fterraform-databricks-aws-workspace","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomarv2%2Fterraform-databricks-aws-workspace/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomarv2%2Fterraform-databricks-aws-workspace/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomarv2%2Fterraform-databricks-aws-workspace/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tomarv2","download_url":"https://codeload.github.com/tomarv2/terraform-databricks-aws-workspace/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245047996,"owners_count":20552431,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","databricks","databricks-account","databricks-e2-workspace","databricks-e2-workspaces","terraform","terraform-module"],"created_at":"2025-03-23T02:35:40.414Z","updated_at":"2025-03-23T02:35:41.044Z","avatar_url":"https://github.com/tomarv2.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://github.com/tomarv2/terraform-databricks-aws-workspace/actions/workflows/pre-commit.yml\" alt=\"Pre Commit\"\u003e\n        \u003cimg src=\"https://github.com/tomarv2/terraform-databricks-aws-workspace/actions/workflows/pre-commit.yml/badge.svg?branch=main\" /\u003e\u003c/a\u003e\n    \u003ca href=\"https://www.apache.org/licenses/LICENSE-2.0\" alt=\"license\"\u003e\n        \u003cimg src=\"https://img.shields.io/github/license/tomarv2/terraform-databricks-aws-workspace\" /\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/tomarv2/terraform-databricks-aws-workspace/tags\" alt=\"GitHub tag\"\u003e\n        \u003cimg src=\"https://img.shields.io/github/v/tag/tomarv2/terraform-databricks-aws-workspace\" /\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/tomarv2/terraform-databricks-aws-workspace/pulse\" alt=\"Activity\"\u003e\n        \u003cimg src=\"https://img.shields.io/github/commit-activity/m/tomarv2/terraform-databricks-aws-workspace\" /\u003e\u003c/a\u003e\n    \u003ca href=\"https://stackoverflow.com/users/6679867/tomarv2\" alt=\"Stack Exchange reputation\"\u003e\n        \u003cimg src=\"https://img.shields.io/stackexchange/stackoverflow/r/6679867\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://discord.gg/XH975bzN\" alt=\"chat on Discord\"\u003e\n        \u003cimg src=\"https://img.shields.io/discord/813961944443912223?logo=discord\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://twitter.com/intent/follow?screen_name=varuntomar2019\" alt=\"follow on Twitter\"\u003e\n        \u003cimg src=\"https://img.shields.io/twitter/follow/varuntomar2019?style=social\u0026logo=twitter\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n## Terraform module for [Databricks AWS Workspace E2 (Part 1)](https://registry.terraform.io/providers/databrickslabs/databricks/latest/docs/guides/aws-workspace)\n\n\u003e ❗️ **Important**\n\u003e\n\u003e :point_right: This Terraform module assumes you have access to: [https://accounts.cloud.databricks.com](https://accounts.cloud.databricks.com)\n\u003e\n\u003e :point_right: Databricks account username: `databricks_account_username`\n\u003e\n\u003e :point_right: Databricks account password: `databricks_account_password`\n\u003e\n\u003e :point_right: Databricks account id, `databricks_account_id` can be found on the bottom left corner of the page, once you're logged in.\n\u003e\n\u003e :point_right: Part 2: Terraform module for [Databricks Workspace management](https://github.com/tomarv2/terraform-databricks-workspace-management)\n\n---\n![Databricks deployment](https://github.com/tomarv2/terraform-databricks-aws-workspace/raw/main/docs/images/databricks_deployment.png)\n---\n\n## Versions\n\n- Module tested for Terraform 1.0.1.\n- `databrickslabs/databricks` provider version [0.4.7](https://registry.terraform.io/providers/databrickslabs/databricks/latest)\n- AWS provider version [3.47](https://registry.terraform.io/providers/hashicorp/aws/latest).\n- `main` branch: Provider versions not pinned to keep up with Terraform releases.\n- `tags` releases: Tags are pinned with versions (use \u003ca href=\"https://github.com/tomarv2/terraform-databricks-aws-workspace/tags\" alt=\"GitHub tag\"\u003e\n        \u003cimg src=\"https://img.shields.io/github/v/tag/tomarv2/terraform-databricks-aws-workspace\" /\u003e\u003c/a\u003e).\n\n---\n## Usage\n\n### Option 1:\n\n```\nterrafrom init\nterraform plan -var='teamid=tryme' -var='prjid=project1'\nterraform apply -var='teamid=tryme' -var='prjid=project1'\nterraform destroy -var='teamid=tryme' -var='prjid=project1'\n```\n**Note:** With this option please take care of remote state storage\n\n### Option 2:\n\n#### Recommended method (stores remote state in S3 using `prjid` and `teamid` to create directory structure):\n\n- Create python 3.6+ virtual environment\n```\npython3 -m venv \u003cvenv name\u003e\n```\n\n- Install package:\n```\npip install tfremote --upgrade\n```\n\n- Set below environment variables:\n```\nexport TF_AWS_BUCKET=\u003cremote state bucket name\u003e\nexport TF_AWS_BUCKET_REGION=us-west-2\nexport TF_AWS_PROFILE=\u003cprofile from ~/.ws/credentials\u003e\n```\n\nor\n\n- Set below environment variables:\n```\nexport TF_AWS_BUCKET=\u003cremote state bucket name\u003e\nexport TF_AWS_BUCKET_REGION=us-west-2\nexport AWS_ACCESS_KEY_ID=\u003caws_access_key_id\u003e\nexport AWS_SECRET_ACCESS_KEY=\u003caws_secret_access_key\u003e\n```\n\n- Update [main.tf](examples/sample/main.tf) file with required values.\n\n- Run and verify the output before deploying:\n```\ntf -c=aws plan -var='teamid=foo' -var='prjid=bar'\n```\n\n- Run below to deploy:\n```\ntf -c=aws apply -var='teamid=foo' -var='prjid=bar'\n```\n\n- Run below to destroy:\n```\ntf -c=aws destroy -var='teamid=foo' -var='prjid=bar'\n```\n\n**NOTE:**\n\n- Read more on [tfremote](https://github.com/tomarv2/tfremote)\n\n### Databricks workspace creation with new role\n```\nmodule \"databricks_workspace\" {\n  source = \"git::git@github.com:tomarv2/terraform-databricks-aws-workspace.git\"\n\n  # NOTE: One of the below is required:\n  # - 'profile_for_iam' - for IAM creation (if none is provided 'default' is used)\n  # - 'existing_role_name'\n  profile_for_iam             = \"iam-admin\"\n\n  databricks_account_username = \"example@example.com\"\n  databricks_account_password = \"sample123!\"\n  databricks_account_id       = \"1234567-1234-1234-1234-1234567\"\n  # -----------------------------------------\n  # Do not change the teamid, prjid once set.\n  teamid = var.teamid\n  prjid  = var.prjid\n}\n```\n\n### Databricks workspace creation with existing role\n```\nmodule \"databricks_workspace\" {\n  source = \"git::git@github.com:tomarv2/terraform-databricks-aws-workspace.git\"\n\n  # NOTE: One of the below is required:\n  # - 'profile_for_iam' - for IAM creation (if none is provided 'default' is used)\n  # - 'existing_role_name'\n  existing_role_arn          = \"arn:aws:iam::123456789012:role/demo-role\"\n\n  databricks_account_username = \"example@example.com\"\n  databricks_account_password = \"sample123!\"\n  databricks_account_id       = \"1234567-1234-1234-1234-1234567\"\n  # -----------------------------------------\n  # Do not change the teamid, prjid once set.\n  teamid = var.teamid\n  prjid  = var.prjid\n}\n```\n\nPlease refer to examples directory [link](examples) for references.\n\n## Coming up:\n\n- **Use** [**Customer Managed VPC**](https://docs.databricks.com/administration-guide/cloud-configurations/aws/customer-managed-vpc.html)\n\n## Troubleshooting:\n\n### IAM policy error\n\nIf you notice below error:\n\n```\nError: MALFORMED_REQUEST: Failed credentials validation checks: Spot Cancellation, Create Placement Group, Delete Tags, Describe Availability Zones, Describe instances, Describe Instance Status, Describe Placement Group, Describe Route Tables, Describe Security Groups, Describe Spot Instances, Describe Spot Price History, Describe Subnets, Describe Volumes, Describe Vpcs, Request Spot Instances\n```\n\n- Try creating workspace from UI:\n\n![create_workspace_error](https://github.com/tomarv2/terraform-databricks-aws-workspace/raw/main/docs/images/create_workspace_error.png)\n\n- Verify if the role and policy exists (assume role should allow external id)\n\n![iam_role_trust_error](https://github.com/tomarv2/terraform-databricks-aws-workspace/raw/main/docs/images/iam_role_trust_error.png)\n\n\n\u003c!-- BEGIN_TF_DOCS --\u003e\n## Requirements\n\n| Name | Version |\n|------|---------|\n| \u003ca name=\"requirement_terraform\"\u003e\u003c/a\u003e [terraform](#requirement\\_terraform) | \u003e= 1.0.1 |\n| \u003ca name=\"requirement_aws\"\u003e\u003c/a\u003e [aws](#requirement\\_aws) | ~\u003e 3.63 |\n| \u003ca name=\"requirement_databricks\"\u003e\u003c/a\u003e [databricks](#requirement\\_databricks) | 0.5.1 |\n| \u003ca name=\"requirement_random\"\u003e\u003c/a\u003e [random](#requirement\\_random) | ~\u003e 3.1 |\n| \u003ca name=\"requirement_time\"\u003e\u003c/a\u003e [time](#requirement\\_time) | ~\u003e 0.7 |\n\n## Providers\n\n| Name | Version |\n|------|---------|\n| \u003ca name=\"provider_aws\"\u003e\u003c/a\u003e [aws](#provider\\_aws) | ~\u003e 3.63 |\n| \u003ca name=\"provider_databricks\"\u003e\u003c/a\u003e [databricks](#provider\\_databricks) | 0.5.1 |\n| \u003ca name=\"provider_databricks.created_workspace\"\u003e\u003c/a\u003e [databricks.created\\_workspace](#provider\\_databricks.created\\_workspace) | 0.5.1 |\n| \u003ca name=\"provider_databricks.mws\"\u003e\u003c/a\u003e [databricks.mws](#provider\\_databricks.mws) | 0.5.1 |\n| \u003ca name=\"provider_random\"\u003e\u003c/a\u003e [random](#provider\\_random) | ~\u003e 3.1 |\n| \u003ca name=\"provider_time\"\u003e\u003c/a\u003e [time](#provider\\_time) | ~\u003e 0.7 |\n\n## Modules\n\n| Name | Source | Version |\n|------|--------|---------|\n| \u003ca name=\"module_iam_policies\"\u003e\u003c/a\u003e [iam\\_policies](#module\\_iam\\_policies) | git::git@github.com:tomarv2/terraform-aws-iam-policies.git | v0.0.4 |\n| \u003ca name=\"module_iam_role\"\u003e\u003c/a\u003e [iam\\_role](#module\\_iam\\_role) | git::git@github.com:tomarv2/terraform-aws-iam-role.git//modules/iam_role_external | v0.0.7 |\n| \u003ca name=\"module_s3\"\u003e\u003c/a\u003e [s3](#module\\_s3) | git::git@github.com:tomarv2/terraform-aws-s3.git | v0.0.8 |\n| \u003ca name=\"module_vpc\"\u003e\u003c/a\u003e [vpc](#module\\_vpc) | git::git@github.com:tomarv2/terraform-aws-vpc.git | v0.0.6 |\n\n## Resources\n\n| Name | Type |\n|------|------|\n| [aws_s3_bucket_policy.root_bucket_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_policy) | resource |\n| [databricks_mws_credentials.this](https://registry.terraform.io/providers/databrickslabs/databricks/0.5.1/docs/resources/mws_credentials) | resource |\n| [databricks_mws_networks.this](https://registry.terraform.io/providers/databrickslabs/databricks/0.5.1/docs/resources/mws_networks) | resource |\n| [databricks_mws_storage_configurations.this](https://registry.terraform.io/providers/databrickslabs/databricks/0.5.1/docs/resources/mws_storage_configurations) | resource |\n| [databricks_mws_workspaces.this](https://registry.terraform.io/providers/databrickslabs/databricks/0.5.1/docs/resources/mws_workspaces) | resource |\n| [databricks_token.pat](https://registry.terraform.io/providers/databrickslabs/databricks/0.5.1/docs/resources/token) | resource |\n| [random_string.naming](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/string) | resource |\n| [time_sleep.wait](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep) | resource |\n| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |\n| [databricks_aws_assume_role_policy.this](https://registry.terraform.io/providers/databrickslabs/databricks/0.5.1/docs/data-sources/aws_assume_role_policy) | data source |\n| [databricks_aws_bucket_policy.this](https://registry.terraform.io/providers/databrickslabs/databricks/0.5.1/docs/data-sources/aws_bucket_policy) | data source |\n| [databricks_aws_crossaccount_policy.cross_account_iam_policy](https://registry.terraform.io/providers/databrickslabs/databricks/0.5.1/docs/data-sources/aws_crossaccount_policy) | data source |\n\n## Inputs\n\n| Name | Description | Type | Default | Required |\n|------|-------------|------|---------|:--------:|\n| \u003ca name=\"input_cidr_block\"\u003e\u003c/a\u003e [cidr\\_block](#input\\_cidr\\_block) | The CIDR block for the VPC | `string` | `\"10.4.0.0/16\"` | no |\n| \u003ca name=\"input_custom_tags\"\u003e\u003c/a\u003e [custom\\_tags](#input\\_custom\\_tags) | Extra custom tags | `any` | `null` | no |\n| \u003ca name=\"input_databricks_account_id\"\u003e\u003c/a\u003e [databricks\\_account\\_id](#input\\_databricks\\_account\\_id) | External ID provided by third party. | `string` | n/a | yes |\n| \u003ca name=\"input_databricks_account_password\"\u003e\u003c/a\u003e [databricks\\_account\\_password](#input\\_databricks\\_account\\_password) | databricks account password | `string` | n/a | yes |\n| \u003ca name=\"input_databricks_account_username\"\u003e\u003c/a\u003e [databricks\\_account\\_username](#input\\_databricks\\_account\\_username) | databricks account username | `string` | n/a | yes |\n| \u003ca name=\"input_databricks_hostname\"\u003e\u003c/a\u003e [databricks\\_hostname](#input\\_databricks\\_hostname) | databricks hostname | `string` | `\"https://accounts.cloud.databricks.com\"` | no |\n| \u003ca name=\"input_existing_role_name\"\u003e\u003c/a\u003e [existing\\_role\\_name](#input\\_existing\\_role\\_name) | If you want to use existing role name, else a new role will be created | `string` | `null` | no |\n| \u003ca name=\"input_prjid\"\u003e\u003c/a\u003e [prjid](#input\\_prjid) | Name of the project/stack e.g: mystack, nifieks, demoaci. Should not be changed after running 'tf apply' | `string` | n/a | yes |\n| \u003ca name=\"input_profile\"\u003e\u003c/a\u003e [profile](#input\\_profile) | profile to use for resource creation | `string` | `\"default\"` | no |\n| \u003ca name=\"input_profile_for_iam\"\u003e\u003c/a\u003e [profile\\_for\\_iam](#input\\_profile\\_for\\_iam) | profile to use for IAM | `string` | `null` | no |\n| \u003ca name=\"input_region\"\u003e\u003c/a\u003e [region](#input\\_region) | AWS region to deploy resources | `string` | `\"us-east-1\"` | no |\n| \u003ca name=\"input_teamid\"\u003e\u003c/a\u003e [teamid](#input\\_teamid) | Name of the team/group e.g. devops, dataengineering. Should not be changed after running 'tf apply' | `string` | n/a | yes |\n\n## Outputs\n\n| Name | Description |\n|------|-------------|\n| \u003ca name=\"output_databricks_credentials_id\"\u003e\u003c/a\u003e [databricks\\_credentials\\_id](#output\\_databricks\\_credentials\\_id) | databricks credentials id |\n| \u003ca name=\"output_databricks_deployment_name\"\u003e\u003c/a\u003e [databricks\\_deployment\\_name](#output\\_databricks\\_deployment\\_name) | databricks deployment name |\n| \u003ca name=\"output_databricks_host\"\u003e\u003c/a\u003e [databricks\\_host](#output\\_databricks\\_host) | databricks hostname |\n| \u003ca name=\"output_databricks_mws_credentials_id\"\u003e\u003c/a\u003e [databricks\\_mws\\_credentials\\_id](#output\\_databricks\\_mws\\_credentials\\_id) | databricks mws credentials id |\n| \u003ca name=\"output_databricks_mws_network_id\"\u003e\u003c/a\u003e [databricks\\_mws\\_network\\_id](#output\\_databricks\\_mws\\_network\\_id) | databricks mws network id |\n| \u003ca name=\"output_databricks_mws_storage_bucket_name\"\u003e\u003c/a\u003e [databricks\\_mws\\_storage\\_bucket\\_name](#output\\_databricks\\_mws\\_storage\\_bucket\\_name) | databricks mws storage bucket name |\n| \u003ca name=\"output_databricks_mws_storage_id\"\u003e\u003c/a\u003e [databricks\\_mws\\_storage\\_id](#output\\_databricks\\_mws\\_storage\\_id) | databricks mws storage id |\n| \u003ca name=\"output_databricks_token\"\u003e\u003c/a\u003e [databricks\\_token](#output\\_databricks\\_token) | Value of the newly created token |\n| \u003ca name=\"output_databricks_token_lifetime_hours\"\u003e\u003c/a\u003e [databricks\\_token\\_lifetime\\_hours](#output\\_databricks\\_token\\_lifetime\\_hours) | Token validity |\n| \u003ca name=\"output_iam_role_arn\"\u003e\u003c/a\u003e [iam\\_role\\_arn](#output\\_iam\\_role\\_arn) | iam role arn |\n| \u003ca name=\"output_inline_policy_id\"\u003e\u003c/a\u003e [inline\\_policy\\_id](#output\\_inline\\_policy\\_id) | inline policy id |\n| \u003ca name=\"output_nonsensitive_databricks_token\"\u003e\u003c/a\u003e [nonsensitive\\_databricks\\_token](#output\\_nonsensitive\\_databricks\\_token) | Value of the newly created token (nonsensitive) |\n| \u003ca name=\"output_s3_bucket_arn\"\u003e\u003c/a\u003e [s3\\_bucket\\_arn](#output\\_s3\\_bucket\\_arn) | s3 bucket arn |\n| \u003ca name=\"output_s3_bucket_id\"\u003e\u003c/a\u003e [s3\\_bucket\\_id](#output\\_s3\\_bucket\\_id) | s3 bucket id |\n| \u003ca name=\"output_s3_bucket_name\"\u003e\u003c/a\u003e [s3\\_bucket\\_name](#output\\_s3\\_bucket\\_name) | s3 bucket name |\n| \u003ca name=\"output_storage_configuration_id\"\u003e\u003c/a\u003e [storage\\_configuration\\_id](#output\\_storage\\_configuration\\_id) | databricks storage configuration id |\n| \u003ca name=\"output_vpc_id\"\u003e\u003c/a\u003e [vpc\\_id](#output\\_vpc\\_id) | vpc id |\n| \u003ca name=\"output_vpc_route_table_ids\"\u003e\u003c/a\u003e [vpc\\_route\\_table\\_ids](#output\\_vpc\\_route\\_table\\_ids) | list of VPC route tables IDs |\n| \u003ca name=\"output_vpc_security_group_id\"\u003e\u003c/a\u003e [vpc\\_security\\_group\\_id](#output\\_vpc\\_security\\_group\\_id) | list of VPC security group ID |\n| \u003ca name=\"output_vpc_subnet_ids\"\u003e\u003c/a\u003e [vpc\\_subnet\\_ids](#output\\_vpc\\_subnet\\_ids) | list of subnet ids within VPC |\n| \u003ca name=\"output_workspace_url\"\u003e\u003c/a\u003e [workspace\\_url](#output\\_workspace\\_url) | databricks workspace url |\n\u003c!-- END_TF_DOCS --\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftomarv2%2Fterraform-databricks-aws-workspace","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftomarv2%2Fterraform-databricks-aws-workspace","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftomarv2%2Fterraform-databricks-aws-workspace/lists"}