{"id":19772171,"url":"https://github.com/zerodha/nomad-cluster-setup","last_synced_at":"2026-02-10T11:04:11.877Z","repository":{"id":160377434,"uuid":"633326306","full_name":"zerodha/nomad-cluster-setup","owner":"zerodha","description":"Terraform modules for creating Nomad servers and clients nodes on AWS.","archived":false,"fork":false,"pushed_at":"2025-03-27T13:08:57.000Z","size":403,"stargazers_count":149,"open_issues_count":7,"forks_count":19,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-03-29T07:04:34.891Z","etag":null,"topics":["aws","nomad","terraform"],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zerodha.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-27T09:00:51.000Z","updated_at":"2025-02-05T12:38:34.000Z","dependencies_parsed_at":"2024-01-23T07:25:13.197Z","dependency_job_id":"e964ad08-a8aa-45c7-b92d-daf66ca98b0c","html_url":"https://github.com/zerodha/nomad-cluster-setup","commit_stats":null,"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zerodha%2Fnomad-cluster-setup","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zerodha%2Fnomad-cluster-setup/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zerodha%2Fnomad-cluster-setup/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zerodha%2Fnomad-cluster-setup/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zerodha","download_url":"https://codeload.github.com/zerodha/nomad-cluster-setup/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247305933,"owners_count":20917208,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","nomad","terraform"],"created_at":"2024-11-12T05:05:38.601Z","updated_at":"2026-02-10T11:04:11.871Z","avatar_url":"https://github.com/zerodha.png","language":"HCL","funding_links":[],"categories":["Infrastructure setup"],"sub_categories":["Deployment and Cluster Setup"],"readme":"\u003ca href=\"https://zerodha.tech\"\u003e\u003cimg src=\"https://zerodha.tech/static/images/github-badge.svg\" align=\"right\" /\u003e\u003c/a\u003e\n\n# Nomad Cluster Setup\n\nTerraform modules to deploy a [HashiCorp Nomad]((https://www.nomadproject.io/)) cluster on AWS using an Auto Scaling Group (ASG). The modules are designed to provision Nomad servers and clients in ASG, making it easy to manage the infrastructure for Nomad cluster. Additionally, the repository includes Packer scripts to build a custom Amazon Machine Image (AMI) with Nomad pre-installed.\n\n![Nomad architecture](./docs/architecture.png)\n\n- [Nomad Cluster Setup](#nomad-cluster-setup)\n  - [AMI](#ami)\n  - [AWS Resources](#aws-resources)\n    - [Auto Scaling Group (ASG)](#auto-scaling-group-asg)\n    - [Security Group](#security-group)\n    - [IAM Role](#iam-role)\n    - [ALB](#alb)\n  - [Nomad Server](#nomad-server)\n    - [Terraform Module Reference](#terraform-module-reference)\n  - [Nomad Client](#nomad-client)\n    - [Terraform Module Reference](#terraform-module-reference-1)\n  - [Example Usage](#example-usage)\n    - [Nomad Servers](#nomad-servers)\n    - [Nomad Clients](#nomad-clients)\n    - [Other Examples](#other-examples)\n  - [Changelog](#changelog)\n  - [Contributors](#contributors)\n  - [Contributing](#contributing)\n  - [LICENSE](#license)\n\n## AMI\n\nThe repository includes a [Packer file](./packer/ami.pkr.hcl), to build a custom Amazon Machine Image (AMI) with Nomad and `docker` pre-installed. This AMI is used by the Terraform modules when creating the ASG instances.\n\nTo build the AMI, run:\n\n```bash\ncd packer\nmake build\n```\n\nNOTE: `dry_run` mode is toggled as true by default. To build the AMI, set the `dry_run` variable in [`Makefile`](./packer/Makefile) to `false`.\n\n## AWS Resources\n\nThe key resources provisioned by this module are:\n\n1. Auto Scaling Group (ASG)\n2. Security Group\n3. IAM Role\n4. Application Load Balancer (ALB) (optional)\n\n### Auto Scaling Group (ASG)\n\nThe module deploys Nomad on top of an Auto Scaling Group (ASG). For optimal performance and fault tolerance, it is recommended to run the Nomad server ASG with 3 or 5 EC2 instances distributed across multiple Availability Zones. Each EC2 instance should utilize an AMI built using the provided Packer script.\n\n**NOTE:** The Nomad Client terraform module allows setting up EC2 instances instead of ASGs. Check out the [`nomad_clients` Terraform Module Reference](./modules/nomad-clients/README.mkdn) for more information.\n\n### Security Group\n\nEach EC2 instance within the ASG is assigned a Security Group that permits:\n\n- All outbound requests\n- All inbound ports specified in the [Nomad documentation](https://developer.hashicorp.com/nomad/docs/install/production/requirements#ports-used)\n\nThe common Security Group is attached to both client and server nodes, enabling the Nomad agent to communicate and discover other agents within the cluster. The Security Group ID is exposed as an output variable for adding additional rules as needed. Furthermore, you can provide your own list of security groups as a variable to the module.\n\n### IAM Role\n\nAn IAM Role is attached to each EC2 instance within the ASG. This role is granted a minimal set of IAM permissions, allowing each instance to automatically discover other instances in the same ASG and form a cluster with them.\n\n### ALB\n\nAn internal Application Load Balancer (ALB) is _optionally_ created for the Nomad servers. The ALB is configured to listen on port 80/443 and forward requests to the Nomad servers on port 4646. The ALB is exposed as an output variable for adding additional rules as needed.\n\n## Nomad Server\n\nThe [`setup_server`](./modules/nomad-servers/scripts/setup_server.tftpl.sh) script included in this project configures and bootstraps Nomad server nodes in an AWS Auto Scaling group. The script performs the following steps:\n\n- Configures the Nomad agent as a server on the EC2 instances and uses the `nomad_join_tag_value` tag to auto-join the cluster. Once all the server instances discover each other, they elect a leader.\n- Bootstraps the Nomad ACL system with a pre-configured token on the first server.\n  - It waits for the cluster leader to get elected before bootstrapping ACL.\n  - The token must be passed as the `nomad_acl_bootstrap_token` variable.\n\n### Terraform Module Reference\n\nCheck out [`nomad_servers`](./modules/nomad-servers/README.mkdn) documentation for module reference.\n\n## Nomad Client\n\nThe [`setup_client`](./modules/nomad-clients/scripts/setup_client.tftpl.sh) script included in this project configures Nomad client nodes in an AWS Auto Scaling group. The script performs the following steps:\n\n- Configures the Nomad agent as a client on the EC2 instances and uses the `nomad_join_tag_value` tag to auto-join the cluster.\n- Configures DNS resolution for the Nomad cluster inside `exec` driver.\n- Prepares configurations for different task drivers.\n\n### Terraform Module Reference\n\nCheck out [`nomad_clients`](./modules/nomad-clients/README.mkdn) documentation for module reference.\n\n## Example Usage\n\n### Nomad Servers\n\n```hcl\nmodule \"nomad_servers\" {\n  source = \"git::https://github.com/zerodha/nomad-cluster-setup//modules/nomad-servers?ref=v1.5.0\"\n\n  cluster_name         = \"demo-nomad\"\n  nomad_join_tag_value = \"demo\"\n  instance_count       = 3\n  ami                  = \"ami-xyz\"\n  vpc                  = \"vpc-xyz\"\n  subnets              = \"subnet-xyz\"\n  create_alb           = true\n  nomad_alb_hostname   = \"nomad.example.internal\"\n\n  nomad_gossip_encrypt_key  = var.nomad_gossip_encrypt_key\n  nomad_acl_bootstrap_token = var.nomad_acl_bootstrap_token\n}\n```\n\n### Nomad Clients\n\n```hcl\nmodule \"nomad_client_demo\" {\n  source = \"git::https://github.com/zerodha/nomad-cluster-setup//modules/nomad-clients?ref=v1.5.0\"\n\n  cluster_name              = \"demo-nomad\"\n  nomad_join_tag_value      = \"demo\"\n  client_name               = \"example-app\"\n  enable_docker_plugin      = true\n  ami                       = \"ami-abc\"\n  instance_type             = \"c6a.xlarge\"\n  instance_desired_count    = 10\n  vpc                       = \"vpc-xyz\"\n  subnets                   = \"subnet-xyz\"\n  route_53_resolver_address = \"10.0.0.2\"\n}\n```\n\n**NOTE:** This module does not set up an ALB for accessing applications running on Nomad Clients. This is left up to the user to configure. Check out [`terraform-aws-alb`](https://github.com/terraform-aws-modules/terraform-aws-alb) or [Other Examples](#other-examples) for more information. You may also need to set [`target_group_arns`](./modules/nomad-clients#input_target_group_arns) if Auto-Scaling Groups are used.\n\n### Other Examples\n\n* [Complete Cluster Setup](./examples)\n\n## Changelog\n\nSee [CHANGELOG.md](./CHANGELOG.md) for a detailed list of changes across versions.\n\n## Contributors\n\n- [Karan Sharma](https://github.com/mr-karan)\n- [Chinmay Pai](https://github.com/thunderbottom)\n\n\n## Contributing\n\nContributions to this repository are welcome. Please submit a pull request or open an issue to suggest improvements or report bugs.\n\n\n## LICENSE\n\n[LICENSE](./LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzerodha%2Fnomad-cluster-setup","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzerodha%2Fnomad-cluster-setup","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzerodha%2Fnomad-cluster-setup/lists"}