{"id":18400700,"url":"https://github.com/databricks/terraform-databricks-mlops-aws-project","last_synced_at":"2025-04-12T16:57:59.804Z","repository":{"id":39623355,"uuid":"506788727","full_name":"databricks/terraform-databricks-mlops-aws-project","owner":"databricks","description":"This module creates and configures service principals with appropriate permissions and entitlements to run CI/CD for a project, and creates a workspace directory as a container for project-specific resources for the Databricks AWS staging and prod workspaces.","archived":false,"fork":false,"pushed_at":"2023-02-15T22:23:25.000Z","size":17,"stargazers_count":5,"open_issues_count":0,"forks_count":3,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-04-11T21:49:42.584Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/databricks.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-06-23T20:54:31.000Z","updated_at":"2025-04-09T21:12:38.000Z","dependencies_parsed_at":"2024-11-06T03:04:35.605Z","dependency_job_id":"1954162d-ea0c-4224-9775-7ae8c960ad21","html_url":"https://github.com/databricks/terraform-databricks-mlops-aws-project","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databricks%2Fterraform-databricks-mlops-aws-project","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databricks%2Fterraform-databricks-mlops-aws-project/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databricks%2Fterraform-databricks-mlops-aws-project/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databricks%2Fterraform-databricks-mlops-aws-project/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/databricks","download_url":"https://codeload.github.com/databricks/terraform-databricks-mlops-aws-project/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248601813,"owners_count":21131612,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T02:36:12.275Z","updated_at":"2025-04-12T16:57:59.768Z","avatar_url":"https://github.com/databricks.png","language":"HCL","readme":"# MLOps AWS Project Module\n\nIn both of the specified staging and prod workspaces, this module:\n* Creates and configures a service principal with appropriate permissions and entitlements to run CI/CD for a project. \n* Creates a workspace directory as a container for project-specific resources\n\nThe service principals are granted `CAN_MANAGE` permissions on the created workspace directories.\n\n**_NOTE:_** \n1. This module is in preview so it is still experimental and subject to change. Feedback is welcome!\n2. The [Databricks providers](https://registry.terraform.io/providers/databricks/databricks/latest/docs) that are passed into the module must be configured with workspace admin permissions.\n3. The module assumes that the [MLOps AWS Infrastructure Module](https://registry.terraform.io/modules/databricks/mlops-aws-infrastructure/databricks/latest) has already been applied, namely that service principal groups with token usage permissions have been created with the default name `\"mlops-service-principals\"` or by specifying the `service_principal_group_name` field.\n4. The service principal tokens are created with a default expiration of 100 days (8640000 seconds), and the module will need to be re-applied after this time to refresh the tokens.\n\n## Usage\n```hcl\nprovider \"databricks\" {\n  alias = \"staging\"     # Authenticate using preferred method as described in Databricks provider\n}\n\nprovider \"databricks\" {\n  alias = \"prod\"     # Authenticate using preferred method as described in Databricks provider\n}\n\nmodule \"mlops_aws_project\" {\n  source = \"databricks/mlops-aws-project/databricks\"\n  providers = {\n    databricks.staging = databricks.staging\n    databricks.prod = databricks.prod\n  }\n  service_principal_name = \"example-name\"\n  project_directory_path = \"/dir-name\"\n}\n```\n\n### Usage example with [MLOps AWS Infrastructure Module](https://registry.terraform.io/modules/databricks/mlops-aws-infrastructure/databricks/latest)\n```hcl\nprovider \"databricks\" {\n  alias = \"dev\" # Authenticate using preferred method as described in Databricks provider\n}\n\nprovider \"databricks\" {\n  alias = \"staging\"     # Authenticate using preferred method as described in Databricks provider\n}\n\nprovider \"databricks\" {\n  alias = \"prod\"     # Authenticate using preferred method as described in Databricks provider\n}\n\nmodule \"mlops_aws_infrastructure\" {\n  source = \"databricks/mlops-aws-infrastructure/databricks\"\n  providers = {\n    databricks.dev     = databricks.dev\n    databricks.staging = databricks.staging\n    databricks.prod    = databricks.prod\n  }\n  staging_workspace_id          = \"123456789\"\n  prod_workspace_id             = \"987654321\"\n  additional_token_usage_groups = [\"users\"]     # This field is optional.\n}\n\n\nmodule \"mlops_aws_project\" {\n  source = \"databricks/mlops-aws-project/databricks\"\n  providers = {\n    databricks.staging = databricks.staging\n    databricks.prod    = databricks.prod\n  }\n  service_principal_name = \"example-name\"\n  project_directory_path = \"/dir-name\"\n  service_principal_group_name = module.mlops_aws_infrastructure.service_principal_group_name \n  # The above field is optional, especially since in this case service_principal_group_name will be mlops-service-principals either way, \n  # but this also serves to create an implicit dependency. Can also be replaced with the following line to create an explicit dependency:\n  # depends_on             = [module.mlops_aws_infrastructure]\n}\n```\n\n### Usage example with Git credentials for service principal\nThis can be helpful for common use cases such as Git authorization for [Remote Git Jobs](https://docs.databricks.com/repos/jobs-remote-notebook.html).\n```hcl\ndata \"databricks_current_user\" \"staging_user\" {\n  provider = databricks.staging\n}\n\ndata \"databricks_current_user\" \"prod_user\" {\n  provider = databricks.prod\n}\n\nprovider \"databricks\" {\n  alias = \"staging_sp\"\n  host  = data.databricks_current_user.staging_user.workspace_url\n  token = module.mlops_aws_project.staging_service_principal_token\n}\n\nprovider \"databricks\" {\n  alias = \"prod_sp\"\n  host  = data.databricks_current_user.prod_user.workspace_url\n  token = module.mlops_aws_project.prod_service_principal_token\n}\n\nresource \"databricks_git_credential\" \"staging_git\" {\n  provider              = databricks.staging_sp\n  git_username          = var.git_username\n  git_provider          = var.git_provider\n  personal_access_token = var.git_token    # This should be configured with `repo` scope for Databricks Repos.\n}\n\nresource \"databricks_git_credential\" \"prod_git\" {\n  provider              = databricks.prod_sp\n  git_username          = var.git_username\n  git_provider          = var.git_provider\n  personal_access_token = var.git_token    # This should be configured with `repo` scope for Databricks Repos.\n}\n```\n\n## Requirements\n| Name | Version |\n|------|---------|\n|[terraform](https://registry.terraform.io/)|\\\u003e=1.1.6|\n|[databricks](https://registry.terraform.io/providers/databricks/databricks/0.5.8)|\\\u003e=0.5.8|\n\n## Inputs\n| Name | Description | Type | Default | Required |\n|------|-------------|------|---------|:--------:|\n|service_principal_name|The display name for the service principals.|string|N/A|yes|\n|project_directory_path|Path/Name of Databricks workspace directory to be created for the project. NOTE: The parent directories in the path must already be created.|string|N/A|yes|\n|service_principal_group_name|The name of the service principal group in the staging and prod workspace. The created service principals will be added to this group.|string|`\"mlops-service-principals\"`|no|\n\n## Outputs\n| Name | Description | Type | Sensitive |\n|------|-------------|------|---------|\n|project_directory_path|Path/Name of Databricks workspace directory created for the project.|string|no|\n|staging_service_principal_application_id|Application ID of the created Databricks service principal in the staging workspace.|string|no|\n|staging_service_principal_token|Sensitive personal access token (PAT) value of the created Databricks service principal in the staging workspace.|string|yes|\n|prod_service_principal_application_id|Application ID of the created Databricks service principal in the prod workspace.|string|no|\n|prod_service_principal_token|Sensitive personal access token (PAT) value of the created Databricks service principal in the prod workspace.|string|yes|\n\n## Providers\n| Name | Authentication | Use |\n|------|-------------|----|\n|databricks.staging|Provided by the user.|Create group, directory, and service principal module in the staging workspace.|\n|databricks.prod|Provided by the user.|Create group, directory, and service principal module in the prod workspace.|\n\n## Resources\n| Name | Type |\n|------|------|\n|databricks_group.staging_sp_group|data source|\n|databricks_group.prod_sp_group|data source|\n|databricks_directory.staging_directory|resource|\n|databricks_permissions.staging_directory_usage|resource|\n|databricks_directory.prod_directory|resource|\n|databricks_permissions.prod_directory_usage|resource|\n|aws-service-principal.staging_sp|module|\n|aws-service-principal.prod_sp|module|","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatabricks%2Fterraform-databricks-mlops-aws-project","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatabricks%2Fterraform-databricks-mlops-aws-project","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatabricks%2Fterraform-databricks-mlops-aws-project/lists"}