{"id":26167543,"url":"https://github.com/open-metadata/terraform-aws-openmetadata","last_synced_at":"2026-04-23T06:11:30.987Z","repository":{"id":265831295,"uuid":"890979398","full_name":"open-metadata/terraform-aws-openmetadata","owner":"open-metadata","description":"OpenMetadata Terraform module for AWS","archived":false,"fork":false,"pushed_at":"2026-02-16T14:46:43.000Z","size":155,"stargazers_count":9,"open_issues_count":9,"forks_count":5,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-16T22:34:05.003Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/open-metadata.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-11-19T14:16:09.000Z","updated_at":"2026-02-16T14:09:54.000Z","dependencies_parsed_at":"2024-12-04T12:22:40.000Z","dependency_job_id":"40f0afc9-ffc7-491c-b265-6ff9d3a34a87","html_url":"https://github.com/open-metadata/terraform-aws-openmetadata","commit_stats":null,"previous_names":["open-metadata/openmetadata-terraform","open-metadata/terraform-aws-openmetadata"],"tags_count":55,"template":false,"template_full_name":null,"purl":"pkg:github/open-metadata/terraform-aws-openmetadata","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/open-metadata%2Fterraform-aws-openmetadata","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/open-metadata%2Fterraform-aws-openmetadata/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/open-metadata%2Fterraform-aws-openmetadata/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/open-metadata%2Fterraform-aws-openmetadata/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/open-metadata","download_url":"https://codeload.github.com/open-metadata/terraform-aws-openmetadata/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/open-metadata%2Fterraform-aws-openmetadata/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29609524,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-19T06:47:36.664Z","status":"ssl_error","status_checked_at":"2026-02-19T06:45:47.551Z","response_time":117,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-11T17:39:04.874Z","updated_at":"2026-04-23T06:11:30.959Z","avatar_url":"https://github.com/open-metadata.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OpenMetadata Terraform module for AWS\n\n# Usage\n\nThe following examples show how to use the module with different provisioners. Even though each example use the same provisioner for all components, you can use different provisioners for any component if you prefer.\n\n## Helm - for testing\n\nUsing `helm` as provisioner for all components:\n\n```hcl\nmodule \"omd\" {\n  source  = \"open-metadata/openmetadata/aws\"\n  version = \"1.12.6\"\n\n  # Namespace where OpenMetadata and dependencies will be deployed\n  app_namespace    = \"example\"\n\n  # Security group IDs assigned to the EKS nodes, required for the Airflow's EFS security groups\n  eks_nodes_sg_ids = [\"sg-1234abcd5678efgh\"]\n\n  # Subnet IDs, required for the Airflow's EFS mount targets\n  subnet_ids       = [\"subnet-1a2b3c4d\", \"subnet-5e6f7g8h\", \"subnet-9i0j1k2l\"]\n\n  # VPC ID for the security groups of the EFS volumes\n  vpc_id = \"vpc-1a2b3c4d\"\n\n}\n```\n\n## Accessing OpenMetadata\n\nOpenMetadata is exposed via the `openmetadata` service. To access it, follow these steps:\n\n1. Run the following command to set up port forwarding:\n\n    ```bash\n    kubectl port-forward service/openmetadata 8585:8585\n    ```\n\n2. Open your web browser and navigate to:\n\n    ```\n    http://localhost:8585\n    ```\n\n3. You should now see the OpenMetadata interface.\n\n## Provisioners\n\nThis module enables you to choose from multiple provisioners to deploy the components and dependencies of OpenMetadata on AWS. The available provisioners for each component are:\n\n|                             | Helm  |  AWS  | Existing | None | Default provisioner|\n| :-------------------------- | :---: | :---: |  :---:  | :--: | :-----------------: |\n| **OpenMetadata**            |  ✅   |  🟥   |    🟥   |  🟥  |      Helm           |\n| **OpenMetadata database**   |  ✅   |  ✅   |    ✅   |  🟥  |      Helm           |\n| **Airflow**                 |  ✅   |  🟥   |    ✅   |  ✅  |      Helm           |\n| **Airflow database**        |  ✅   |  ✅   |    ✅   |  🟥  |      Helm           |\n| **OpenSearch**              |  ✅   |  ✅   |    ✅   |  🟥  |      Helm           |\n\n\u003e [!NOTE]\n\u003e If you select `existing` as the provisioner for Airflow, we expect the service to be fully functional, including its database.\n\u003e The Airflow database will not be deployed in this scenario.\n\n\u003e [!NOTE]\n\u003e If you select `none` as the provisioner for Airflow, no Airflow instance or database will be deployed.\n\u003e OpenMetadata will use Kubernetes Jobs for ingestion via the OMJob operator, which is installed automatically.\n\n## AWS - production ready\n\nUsing `aws` as provisioner for all possible components:\n\n```hcl\nmodule \"omd\" {\n  source  = \"open-metadata/openmetadata/aws\"\n  version = \"1.12.6\"\n\n  # Namespace where OpenMetadata and dependencies will be deployed\n  app_namespace = \"example\"\n\n  # Security group IDs assigned to the EKS nodes, the RDS instances, EFS volumes, and OpenSearch domain will allow inbound traffic from them\n  eks_nodes_sg_ids = [\"sg-1234abcd5678efgh\", \"sg-8765ijkl4321mnop\"]\n\n  # ARN of the KMS key used to encrypt resources \n  kms_key_id = \"arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012\"\n\n  # Subnet IDs, used for:\n  # the Airflow's EFS mount targets\n  # the subnet group for the RDS instances\n  # the OpenSearch domain\n  subnet_ids = [\"subnet-1a2b3c4d\", \"subnet-5e6f7g8h\", \"subnet-9i0j1k2l\"]\n\n  # VPC ID for the security groups of the EFS volumes, the RDS instances, and the OpenSearch domain\n  vpc_id = \"vpc-1a2b3c4d\"\n\n  # OpenMetadata database settings\n  db = {\n    provisioner = \"aws\"\n  }\n\n  # Airflow settings\n  airflow = {\n    db = {\n      provisioner = \"aws\"\n    }\n  }\n\n  # OpenSearch settings\n  opensearch = {\n    provisioner = \"aws\"\n  }\n}\n```\n\n## Existing\n\nUsing `existing` as provisioner for all possible components:\n\n```hcl\nmodule \"omd\" {\n  source  = \"open-metadata/openmetadata/aws\"\n  version = \"1.12.6\"\n\n  # Namespace where OpenMetadata and dependencies will be deployed\n  app_namespace = \"example\"\n\n  # OpenMetadata database settings\n  db = {\n    provisioner = \"existing\"\n    host        = \"omd-db.postgres.example\"\n    port        = \"5432\"\n    db_name     = \"openmetadata_db\"\n    engine = {\n      name = \"postgres\"\n    }\n    credentials = {\n      username = \"dbadmin\"\n      password = {\n        secret_ref = \"db-secrets\"\n        secret_key = \"password\"\n      }\n    }\n  }\n\n  # Airflow settings\n  airflow = {\n    provisioner = \"existing\"\n    endpoint = \"http://airflow.example:8080\"\n    credentials = {\n      username = \"admin\"\n      password = {\n        secret_ref = \"airflow-auth\"\n        secret_key = \"password\"\n      }\n    }\n  }\n\n  # OpenSearch settings\n  opensearch = {\n    provisioner = \"existing\"\n    host        = \"opensearch.example\"\n    port        = \"443\"\n    scheme      = \"https\"\n  }\n}\n```\n\n## Without Airflow — using the OMJob operator\n\nSet `airflow.provisioner = \"none\"` to deploy OpenMetadata without Airflow. No Airflow instance, EFS volumes, or Airflow database will be created. OpenMetadata will be configured to use Kubernetes Jobs for ingestion, managed by the OMJob operator:\n\n```hcl\nmodule \"omd\" {\n  source  = \"open-metadata/openmetadata/aws\"\n  version = \"1.12.6\"\n\n  app_namespace    = \"example\"\n  eks_nodes_sg_ids = [\"sg-1234abcd5678efgh\"]\n  subnet_ids       = [\"subnet-1a2b3c4d\", \"subnet-5e6f7g8h\", \"subnet-9i0j1k2l\"]\n  vpc_id           = \"vpc-1a2b3c4d\"\n\n  airflow = {\n    provisioner = \"none\"\n  }\n}\n```\n\nThe module will automatically set the following in the OpenMetadata chart:\n- `pipelineServiceClientConfig.type: \"k8s\"`\n- `pipelineServiceClientConfig.k8s.useOMJobOperator: true`\n- `omjobOperator.enabled: true`\n\n# Examples\n\n## AWS\n\n- [Complete](examples/complete)\n\n# Terraform docs README files\n\n- [OpenMetadata deployment](modules/openmetadata-deployment)\n- [OpenMetadata dependencies](modules/openmetadata-dependencies)\n- [Airflow EFS module](modules/airflow-efs)\n- [RDS module](modules/rds)\n- [OpenSearch module](modules/opensearch)\n\n# How we manage settings\n\nComponents have default values for each provisioner, which are defined in the `defaults.tf` file.\nThe final settings for each component are determined by checking whether a value has been provided for each parameter. If a value is not provided for a parameter, the default one is used. This process is handled in the `component_conf.tf` files.\n\n# Adding extra environment variables\n\nYou can add extra environment variables by using the parameter `extra_envs`:\n\n```hcl\nmodule \"omd\" {\n  source  = \"open-metadata/openmetadata/aws\"\n  version = \"1.12.6\"\n\n  # Namespace where OpenMetadata and dependencies will be deployed\n  app_namespace    = \"example\"\n\n  # Security group IDs assigned to the EKS nodes, required for the Airflow's EFS security groups\n  eks_nodes_sg_ids = [\"sg-1234abcd5678efgh\"]\n\n  # Subnet IDs, required for the Airflow's EFS mount targets\n  subnet_ids       = [\"subnet-1a2b3c4d\", \"subnet-5e6f7g8h\", \"subnet-9i0j1k2l\"]\n\n  # VPC ID for the security groups of the EFS volumes\n  vpc_id = \"vpc-1a2b3c4d\"\n\n  # Extra environment variables for the OpenMetadata pod\n  extra_envs = {\n    \"VAR_1\" = \"foo\"\n    \"VAR_2\" = \"bar\"\n  }\n}\n```\n\nYou can also add extra environment variables from Kubernetes secrets by using the parameter `env_from`:\n\n```hcl\nmodule \"omd\" {\n  source  = \"open-metadata/openmetadata/aws\"\n  version = \"1.12.6\"\n\n  # Namespace where OpenMetadata and dependencies will be deployed\n  app_namespace    = \"example\"\n\n  # Security group IDs assigned to the EKS nodes, required for the Airflow's EFS security groups\n  eks_nodes_sg_ids = [\"sg-1234abcd5678efgh\"]\n\n  # Subnet IDs, required for the Airflow's EFS mount targets\n  subnet_ids       = [\"subnet-1a2b3c4d\", \"subnet-5e6f7g8h\", \"subnet-9i0j1k2l\"]\n\n  # VPC ID for the security groups of the EFS volumes\n  vpc_id = \"vpc-1a2b3c4d\"\n\n  # Extra environment variables for the OpenMetadata pod from Kubernetes secrets\n  env_from = [\"my-other-secret-1\", \"my-other-secret-2\"]\n}\n```\n\n# Customizing Helm values\n\nYou can pass arbitrary Helm chart values for each component using the `*_helm_values` variables. These are merged on top of the values generated by the module, so they can be used to override defaults or configure options not exposed as Terraform variables.\n\n| Variable | Chart |\n| :--- | :--- |\n| `openmetadata_helm_values` | OpenMetadata |\n| `airflow_helm_values` | Airflow (inside the deps chart) |\n| `mysql_helm_values` | MySQL (inside the deps chart) |\n| `opensearch_helm_values` | OpenSearch (inside the deps chart) |\n\nExample — increase the OpenMetadata replica count and tune Airflow's scheduler:\n\n```hcl\nmodule \"omd\" {\n  source  = \"open-metadata/openmetadata/aws\"\n  version = \"1.12.6\"\n\n  app_namespace    = \"example\"\n  eks_nodes_sg_ids = [\"sg-1234abcd5678efgh\"]\n  subnet_ids       = [\"subnet-1a2b3c4d\", \"subnet-5e6f7g8h\", \"subnet-9i0j1k2l\"]\n  vpc_id           = \"vpc-1a2b3c4d\"\n\n  openmetadata_helm_values = {\n    \"replicaCount\" = \"2\"\n  }\n\n  airflow_helm_values = {\n    \"scheduler.replicas\" = \"2\"\n  }\n}\n```\n\n## Airflow database — `existing` provisioner\n\nWhen using `existing` as the Airflow database provisioner, you must supply a Kubernetes secret that contains the full SQLAlchemy connection URI for the Airflow metadata database. Create the secret in the same namespace as OpenMetadata:\n\n```bash\nkubectl create secret generic airflow-db-connection \\\n  --namespace \u003capp_namespace\u003e \\\n  --from-literal=connection='postgresql+psycopg2://user:password@host:5432/airflow'\n```\n\nThen reference it via `airflow_helm_values`:\n\n```hcl\nmodule \"omd\" {\n  source  = \"open-metadata/openmetadata/aws\"\n  version = \"1.12.6\"\n\n  app_namespace    = \"example\"\n  eks_nodes_sg_ids = [\"sg-1234abcd5678efgh\"]\n  subnet_ids       = [\"subnet-1a2b3c4d\", \"subnet-5e6f7g8h\", \"subnet-9i0j1k2l\"]\n  vpc_id           = \"vpc-1a2b3c4d\"\n\n  airflow = {\n    db = {\n      provisioner = \"existing\"\n    }\n  }\n\n  airflow_helm_values = {\n    \"airflow.data.metadataConnectionSecretName\" = \"airflow-db-connection\"\n  }\n}\n```\n\n\u003e [!NOTE]\n\u003e When using `aws` as the Airflow database provisioner, this secret is created automatically by the module.\n\n# Accessing Airflow using port forwarding\n\nThis section explains how to access **Airflow** running in your Kubernetes cluster using port forwarding.\n\nIf you deployed Airflow using our Helm chart for dependencies, it will be exposed via the `openmetadata-deps-web` service. To access it, follow these steps:\n\n1. Run the following command to set up port forwarding:\n\n    ```bash\n    kubectl port-forward service/openmetadata-deps-web 8080:8080\n    ```\n\n2. Open your web browser and navigate to:\n\n    ```\n    http://localhost:8080\n    ```\n\n3. You should now see the Airflow interface.\n\n## Notes\n\n- Ensure that the required services (`openmetadata-deps-web` and `openmetadata`) are active.\n- The `kubectl port-forward` command maps a local port on your machine to the service's port in the Kubernetes cluster. This allows you to access the service as though it were running locally.\n- If a service is already running on your machine using one of the ports in the examples, you can modify the local port (the first number in the mapping, e.g., 8585:8585) to an available port of your choice.\n- Keep the terminal session with the `kubectl port-forward` command open while you are accessing the services.\n\n\n# Development\n\n## pre-commit\n\nYou can use [pre-commit](https://pre-commit.com/) to run checks on the code before committing. Checks are defined in the `.pre-commit-config.yaml` file and currently include:\n\n - terraform-docs\n - terraform fmt\n\nTo install the pre-commit hooks, run:\n\n```bash\ndevops@collate:~/projects/collate/openmetadata-terraform/openmetadata-aws$ pre-commit install\npre-commit installed at .git/hooks/pre-commit\n```\n\nThen the checks will run automatically before each commit. If any check fails, the commit will be aborted and you will need to fix the issues before committing again:\n\n```bash\ndevops@collate:~/projects/collate/openmetadata-terraform/openmetadata-aws$ git add variables.tf \ndevops@collate:~/projects/collate/openmetadata-terraform/openmetadata-aws$ git commit -m \"GEN-1521 test pre-commit\"\n[WARNING] Unstaged files detected.\n[INFO] Stashing unstaged files to /home/devops/.cache/pre-commit/patch1732213728-159233.\nterraform-docs...........................................................Passed\nterraform fmt............................................................Failed\n- hook id: terraform-fmt\n- files were modified by this hook\n\nopenmetadata-aws/variables.tf\n\n[INFO] Restored changes from /home/devops/.cache/pre-commit/patch1732213728-159233.\ndevops@collate:~/projects/collate/openmetadata-terraform/openmetadata-aws$ git status -sb\n## GEN-1521-aws-initial-version...origin/GEN-1521-aws-initial-version [ahead 1]\nMM variables.tf\ndevops@collate:~/projects/collate/openmetadata-terraform/openmetadata-aws$ git diff variables.tf\ndiff --git a/openmetadata-aws/variables.tf b/openmetadata-aws/variables.tf\nindex ee4af93..a3e3f57 100644\n--- a/openmetadata-aws/variables.tf\n+++ b/openmetadata-aws/variables.tf\n@@ -1,5 +1,5 @@\n variable \"app_helm_chart_version\" {\n-  type      = string\n+  type        = string\n   description = \"Version of the OpenMetadata Helm chart to deploy. If not specified, the variable `app_version` will be used.\"\n   default     = null\n }\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopen-metadata%2Fterraform-aws-openmetadata","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopen-metadata%2Fterraform-aws-openmetadata","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopen-metadata%2Fterraform-aws-openmetadata/lists"}