{"id":19107287,"url":"https://github.com/govtechsg/fargate-gitlab-runner-terraform","last_synced_at":"2025-09-23T12:46:55.091Z","repository":{"id":78493640,"uuid":"557240227","full_name":"GovTechSG/fargate-gitlab-runner-terraform","owner":"GovTechSG","description":null,"archived":false,"fork":false,"pushed_at":"2023-05-29T01:53:35.000Z","size":136,"stargazers_count":19,"open_issues_count":1,"forks_count":5,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-02-14T16:45:07.454Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GovTechSG.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-25T10:30:46.000Z","updated_at":"2025-01-08T07:58:48.000Z","dependencies_parsed_at":"2024-11-09T04:12:08.685Z","dependency_job_id":"16df0ac3-82b9-43af-9cbb-82211bd195eb","html_url":"https://github.com/GovTechSG/fargate-gitlab-runner-terraform","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GovTechSG%2Ffargate-gitlab-runner-terraform","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GovTechSG%2Ffargate-gitlab-runner-terraform/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GovTechSG%2Ffargate-gitlab-runner-terraform/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GovTechSG%2Ffargate-gitlab-runner-terraform/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GovTechSG","download_url":"https://codeload.github.com/GovTechSG/fargate-gitlab-runner-terraform/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240161827,"owners_count":19757896,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T04:12:00.729Z","updated_at":"2025-09-23T12:46:50.047Z","avatar_url":"https://github.com/GovTechSG.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Terraform module for deploying ECS service for Fargate GitLab runner\n\nThis repo is part of a set of repos for the complete setup of ECS Service for managers and workers:\n* [fargate-gitlab-runner](../../../fargate-gitlab-runner): Docker image for the ECS task for all runner managers\n* [fargate-gitlab-runner-worker](../../../fargate-gitlab-runner-worker): Sample docker images for the worker ECS tasks\n* [fargate-gitlab-runner-terraform](../../../fargate-gitlab-terraform): Terraform code to set up complete ECS Service for managers and workers\n\n## Architecture\n![ECS Fargate GitLab runner Architecture](assets/ECS%20Fargate%20GitLab%20runner%20Architecture.png)\n\n## Folder structure\n```\n|_ environments\n   |_ sample-dev                               # Sample dev environment\n      |_ ...\n      |_ ecs-fargate-gitlab-runner-service     # Specification of the ECS service module for this environment\n         |_ terragrunt.hcl\n      |_ env_inputs.hcl                        # Environment-specific parameters\n   |\n   |_ sample-prd                               # Sample prd environment\n      |_ ...\n      |_ env_inputs.hcl\n   |\n   |_ terragrunt.hcl                           # Common Terragrunt config for all environments\n|\n|_ terraform_modules\n   |_ ecs-fargate-gitlab-runner-service        # Module for ECS service for Fargate GitLab runner\n```\n\nThis folder structure follows [Terragrunt's](https://terragrunt.gruntwork.io/docs/getting-started/quick-start/#promote-immutable-versioned-terraform-modules-across-environments)'s recommendation to keep Terraform configurations DRY:\n* Terraform modules are kept in `terraform_modules`\n* Common Terragrunt settings for all environments are kept at `environments/terragrunt.hcl`\n* Each environment can provide its own parameters in `environments/\u003cenv\u003e/env_inputs.hcl`\n* Live modules for deployment are kept in each environment's folder with a `terragrunt.hcl` to provide link to the module in `terraform_modules` and provide variables' values in `inputs` block\n\n\n## Usage\n### Pre-requisites:\nFollowing resources are required to be set up either manually or via another set of Terraform code (recommended):\n* Secret for GitLab Token: Obtain token for runners from GitLab, create a new secret in either AWS Secret Manager or System Manager Parameter Store. Take note of the KMS key used.\n* VPC, subnets and security groups required for both managers and workers. Take note that SSH communication via port 22 need to be allowed between managers' and workers' network ACLs and security groups.\n* Workers' ECS Task roles with appropriate policies for the worker tasks to access AWS services.\n\n### Setup\n\n* Copy from `environments/sample-dev` to a new env `environments/\u003cenv\u003e`\n* Update your environment settings at `environments/\u003cenv\u003e/env_inputs.hcl`.\n* If there's no existing ECS Cluster for either managers or workers, create new one(s) following the samples at `environments/\u003cenv\u003e/ecs-cluster-for-managers` or `environments/\u003cenv\u003e/ecs-cluster-for-workers`.\n* Update variable values in `environments/\u003cenv\u003e/ecs-fargate-gitlab-runner-service/terragrunt.hcl` matching your environment. Refer to [environments/sample-dev/ecs-fargate-gitlab-runner-service/terragrunt.hcl](environments/sample-dev/ecs-fargate-gitlab-runner-service/terragrunt.hcl) for sample values. Snippet:\n```hcl\ninputs = merge(local.vars.inputs,\n  {\n    project_code = local.vars.inputs.project_name\n    environment  = local.vars.inputs.env\n    service_name = \"--SERVICE-NAME--\"\n\n    manager_instance_count     = 1\n    manager_ecs_cluster_arn    = \"arn:aws:ecs:${local.vars.inputs.aws_region}:${local.vars.inputs.aws_account_id}:cluster/\u003cMANAGER_CLUSTER_NAME\u003e\"\n    manager_docker_image       = \"${local.vars.inputs.aws_account_id}.dkr.ecr.${local.vars.inputs.aws_region}.amazonaws.com/\u003cMANAGER_DOCKER_IMAGE\u003e\"\n    manager_subnet_ids         = [\"\u003cMANAGER_SUBNET_1\u003e\", \"\u003cMANAGER_SUBNET_2\u003e\", \"\u003cMANAGER_SUBNET_3\u003e\"]\n    manager_security_group_ids = [\"\u003cMANAGER_SECURITY_GROUP_1\u003e\"]\n    gitlab_token_secret_arn    = \"arn:aws:secretsmanager:${local.vars.inputs.aws_region}:${local.vars.inputs.aws_region}:secret:PATH_TO_GITLAB_TOKEN\"\n\n    gitlab_url                = \"\u003cGITLAB_FULL_URL\u003e\"\n    gitlab_runner_concurrency = 10\n    gitlab_runner_name_prefix = \"\u003cRUNNER_NAME_PREFIX\u003e\"\n\n    managers_configs = {\n      dev_tool1 : {\n        tags : [\"dev\", \"tool1\"]\n        limit : 10 # Check the available IPs in worker subnet\n        worker_docker_image : \"${local.vars.inputs.aws_account_id}.dkr.ecr.${local.vars.inputs.aws_region}.amazonaws.com/\u003cWORKER_DOCKER_IMAGE_1\u003e\"\n        worker_cpu : 256\n        worker_memory : 512\n        worker_ecs_cluster_arn : \"arn:aws:ecs:${local.vars.inputs.aws_region}:${local.vars.inputs.aws_account_id}:cluster/\u003cECS_CLUSTER_NAME\u003e\"\n        worker_aws_region : local.vars.inputs.aws_region\n        worker_subnet_id : \"WORKER_SUBNET_1\"\n        worker_security_group_id : \"WORKER_SECURITY_GROUP_1\"\n        worker_ssh_user : \"user_1\"\n        worker_task_role_arn : \"arn:aws:iam::${local.vars.inputs.aws_account_id}:role/\u003cWORKER_IAM_ROLE_1\u003e\"\n      }\n      dev_tool2_tool3 : {\n        tags : [\"dev\", \"tool2\", \"tool3\"]\n        limit : 5 # Check the available IPs in worker subnet\n        worker_docker_image : \"${local.vars.inputs.aws_account_id}.dkr.ecr.${local.vars.inputs.aws_region}.amazonaws.com/\u003cWORKER_DOCKER_IMAGE_2\u003e\"\n        worker_cpu : 512\n        worker_memory : 2048\n        worker_ecs_cluster_arn : \"arn:aws:ecs:${local.vars.inputs.aws_region}:${local.vars.inputs.aws_account_id}:cluster/\u003cECS_CLUSTER_NAME\u003e\"\n        worker_aws_region : local.vars.inputs.aws_region\n        worker_subnet_id : \"WORKER_SUBNET_2\"\n        worker_security_group_id : \"WORKER_SECURITY_GROUP_2\"\n        worker_ssh_user : \"user_2\"\n        worker_task_role_arn : \"arn:aws:iam::${local.vars.inputs.aws_account_id}:role/\u003cWORKER_IAM_ROLE_2\u003e\"\n      }\n    }\n\n    iam_permissions_boundary = \"\" # modify as required\n  }\n)\n```\n\n## Deployment with `terragrunt`\n```shell\ncd environments/\u003cenv\u003e/ecs-fargate-gitlab-runner-service\nterragrunt apply\n```\n\nIf you prefer to use Terraform instead of Terragrunt for a quick test,\n- Create `terraform.tfvars` file in folder terraform_modules/ecs-fargate-gitlab-runner-service with all variable values found in `environments/\u003cenv\u003e/ecs-fargate-gitlab-runner-service/terragrunt.hcl`\n- Copy content of `versions.tf` and `provider.tf` from `environments/terragrunt.hcl` into their respective files in `terraform_modules/ecs-fargate-gitlab-runner-service`.\n- Finally, run `terraform apply`.\n\n\n## Test Gitlab job\nUse this code for a simple test of the created GitLab runner(s):\n```yaml\ntest:\n  tags:\n    # these should match all the tags set in the manager configs or a subset (note that a subset may mean other non-Fargate runners can pick up the job, depending on your setup)\n    - dev\n    - tool1\n  script:\n    - echo \"It works!\"\n    - for i in $(seq 1 30); do echo \".\"; sleep 1; done\n```\n\n\n## More info on `ecs-fargate-gitlab-runner-service` Terraform module\nRefer to [terraform_modules/ecs-fargate-gitlab-runner-service/README.md](terraform_modules/ecs-fargate-gitlab-runner-service/README.md) for detailed documentation of the module.\n\n**Important parameters:**\n\n| Name | Description | Type | Default | Required |\n|------|-------------|------|---------|:--------:|\n| \u003ca name=\"input_gitlab_runner_concurrency\"\u003e\u003c/a\u003e [gitlab\\_runner\\_concurrency](#input\\_gitlab\\_runner\\_concurrency) | Number of jobs that can run concurrently. Refer to the guide at https://www.howtogeek.com/devops/how-to-manage-gitlab-runner-concurrency-for-parallel-ci-jobs/ | `number` | `10` | no |\n| \u003ca name=\"input_gitlab_token_secret_arn\"\u003e\u003c/a\u003e [gitlab\\_token\\_secret\\_arn](#input\\_gitlab\\_token\\_secret\\_arn) | ARN of the secret in either Secret Manager or Parameter Store which stores the GitLab token for runner registration | `string` | n/a | yes |\n| \u003ca name=\"input_manager_docker_image\"\u003e\u003c/a\u003e [manager\\_docker\\_image](#input\\_manager\\_docker\\_image) | n/a | `string` | n/a | yes |\n| \u003ca name=\"input_manager_ecs_cluster_arn\"\u003e\u003c/a\u003e [manager\\_ecs\\_cluster\\_arn](#input\\_manager\\_ecs\\_cluster\\_arn) | ARN of the ECS Cluster for managers | `string` | n/a | yes |\n| \u003ca name=\"input_manager_instance_count\"\u003e\u003c/a\u003e [manager\\_instance\\_count](#input\\_manager\\_instance\\_count) | Number of ECS Fargate instances. Final number of GitLab runners is manager\\_instance\\_count * length(keys(managers\\_configs)) | `number` | `1` | no |\n| \u003ca name=\"input_managers_configs\"\u003e\u003c/a\u003e [managers\\_configs](#input\\_managers\\_configs) | Map of managers' names and their worker configs. Final number of GitLab runners is manager\\_instance\\_count * length(keys(managers\\_configs)) | \u003cpre\u003emap(object({\u003cbr\u003e    tags : list(string)\u003cbr\u003e    limit : number, # Note that this is limited by the number of available IPs in the worker subnet\u003cbr\u003e    worker_docker_image : string\u003cbr\u003e    worker_cpu : number\u003cbr\u003e    worker_memory : number\u003cbr\u003e    worker_ecs_cluster_arn : string\u003cbr\u003e    worker_aws_region : string\u003cbr\u003e    worker_subnet_id : string\u003cbr\u003e    worker_security_group_id : string\u003cbr\u003e    worker_ssh_user : string\u003cbr\u003e    worker_task_role_arn : string\u003cbr\u003e  }))\u003c/pre\u003e | \u003cpre\u003e{\u003cbr\u003e  \"r1\": {\u003cbr\u003e    \"limit\": 10,\u003cbr\u003e    \"tags\": [\u003cbr\u003e      \"env1\",\u003cbr\u003e      \"tool1\",\u003cbr\u003e      \"tool2\"\u003cbr\u003e    ],\u003cbr\u003e    \"worker_aws_region\": \"region-1\",\u003cbr\u003e    \"worker_cpu\": 256,\u003cbr\u003e    \"worker_docker_image\": \"image_1\",\u003cbr\u003e    \"worker_ecs_cluster_arn\": \"ecs_cluster_arn_1\",\u003cbr\u003e    \"worker_memory\": 512,\u003cbr\u003e    \"worker_security_group_id\": \"sg-321\",\u003cbr\u003e    \"worker_ssh_user\": \"user_1\",\u003cbr\u003e    \"worker_subnet_id\": \"subnet-123\",\u003cbr\u003e    \"worker_task_role_arn\": \"task_role_arn_1\"\u003cbr\u003e  }\u003cbr\u003e}\u003c/pre\u003e | no |\n\n\n### Troubleshooting\nFollowings are some issues that can occur:\n\n**Unable to start Fargate Task due to No Container Instances were found in your cluster error**\n* Check your ECS Cluster for workers and make sureDefault capacity provider strategy is set to FARGATE\n\n**Manager unable to connect to ECS to start a task**\n* If the managers are hosted in private subnets, create VPC endpoints for ECS and ECR and make sure the managers can access them.\n\n**Manager unable to connect to worker ECS task via ssh**\n* Make sure your worker container image has openssh installed and `SSH_PUBLIC_KEY` is added to the right user's `~/.ssh/authorized_keys`.\n* Check that the subnets and security groups of both managers and workers allow traffic on port 22. Use the VPC Reachability Analyzer to confirm.\n* If the error is `signature algorithm ssh-rsa not in PubkeyAcceptedAlgorithms`, enable `ssh-rsa` by adding this to worker container image: `RUN echo \"PubkeyAcceptedKeyTypes +ssh-rsa\" \u003e\u003e /etc/ssh/sshd_config`.\n\n**Worker ECS task has no credentials to access AWS**\n* Share the variable `AWS_CONTAINER_CREDENTIALS_RELATIVE_URI` with the SSH session by adding this to sshd run:\n`-o \"SetEnv=AWS_CONTAINER_CREDENTIALS_RELATIVE_URI=\\\"$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI\\\"\"`\n\n### Known Limitation:\n* The Fargate driver doesn't support ECS Exec yet. For more info: https://gitlab.com/gitlab-org/ci-cd/custom-executor-drivers/fargate/-/issues/49\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgovtechsg%2Ffargate-gitlab-runner-terraform","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgovtechsg%2Ffargate-gitlab-runner-terraform","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgovtechsg%2Ffargate-gitlab-runner-terraform/lists"}