{"id":13989740,"url":"https://github.com/turnerlabs/compose-to-batch","last_synced_at":"2025-04-09T18:50:19.973Z","repository":{"id":57204894,"uuid":"94587468","full_name":"turnerlabs/compose-to-batch","owner":"turnerlabs","description":"A CLI tool to transform a docker-compose.yml into an AWS Batch job definition","archived":false,"fork":false,"pushed_at":"2017-08-11T12:55:23.000Z","size":6,"stargazers_count":17,"open_issues_count":1,"forks_count":1,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-23T20:43:48.371Z","etag":null,"topics":["aws","batch","docker","docker-compose","job"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/turnerlabs.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-06-16T23:09:35.000Z","updated_at":"2023-07-29T02:28:17.000Z","dependencies_parsed_at":"2022-09-18T00:42:13.643Z","dependency_job_id":null,"html_url":"https://github.com/turnerlabs/compose-to-batch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/turnerlabs%2Fcompose-to-batch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/turnerlabs%2Fcompose-to-batch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/turnerlabs%2Fcompose-to-batch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/turnerlabs%2Fcompose-to-batch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/turnerlabs","download_url":"https://codeload.github.com/turnerlabs/compose-to-batch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248092127,"owners_count":21046416,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","batch","docker","docker-compose","job"],"created_at":"2024-08-09T13:02:01.422Z","updated_at":"2025-04-09T18:50:19.947Z","avatar_url":"https://github.com/turnerlabs.png","language":"JavaScript","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"readme":"compose-to-batch\n==============\n\nA CLI tool to transform a [`docker-compose.yml`](https://docs.docker.com/compose/compose-file/compose-file-v2/) into an [`AWS Batch job definition`](http://docs.aws.amazon.com/batch/latest/userguide/job_definitions.html).\n\nThis tool allows you to develop your job code on your `laptop` using docker:\n```\n$ docker-compose up\n```\n\nAnd then when you're ready to deploy to `AWS Batch`:\n```\n$ docker-compose config | compose-to-batch \u003e job-definition.json\n$ aws batch register-job-definition --cli-input-json file://job-definition.json\n```\n\n### usage\n\ninstall\n```\nnpm install -g compose-to-batch\n```\n\ngiven the following `docker-compose.yml`\n\n```yaml\nversion: \"2\"\nservices:\n  my-job:\n    build: .\n    image: 12345678910.dkr.ecr.us-east-1.amazonaws.com/my-job:0.1.0\n    environment:\n      REGION: us-east-1\n```\n\nrunning the following command\n\n```bash\ndocker-compose config | compose-to-batch\n```\n\nwill output the following\n\n```json\n{\n  \"jobDefinitionName\": \"my-job\",\n  \"type\": \"container\",\n  \"containerProperties\": {\n    \"image\": \"12345678910.dkr.ecr.us-east-1.amazonaws.com/my-job:0.1.0\",\n    \"environment\": [\n      {\n        \"name\": \"REGION\",\n        \"value\": \"us-east-1\"\n      }\n    ],\n    \"vcpus\": \"2\",\n    \"memory\": \"2000\"\n  }\n}\n```\n\n### mapping\n\n- service -\u003e jobDefinitionName\n- service.image -\u003e containerProperties.image\n- service.command -\u003e containerProperties.command\n- service.environment -\u003e containerProperties.environment\n- service.labels.composeToBatch.vcpus -\u003e containerProperties.vcpus\n- service.labels.composeToBatch.memory -\u003e containerProperties.memory\n- service.labels.composeToBatch.jobRoleArn -\u003e containerProperties.jobRoleArn\n\n\n### local development using IAM keys and Batch with jobRoleArn\n\nA common scenario is to do local development using IAM keys specified as environment variables in your container.  You typically don't want to check in your secrets to source control so a `.env` file is a good option here.  When you're ready to deploy to Batch, you also typically don't want to expose your secret IAM keys.  With this in mind,  `compose-to-batch` will exclude any environment variables that reference dynamic variables.  For example:\n\n```\nAWS_ACCESS_KEY_ID=xyz\nAWS_SECRET_ACCESS_KEY=xyz\n```\n\n```yaml\nversion: \"2\"\nservices:\n  my-job:\n    build: .\n    image: 12345678910.dkr.ecr.us-east-1.amazonaws.com/my-job:0.1.0\n    environment:\n      REGION: us-east-1\n      S3_BUCKET: some-bucket\n      AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID} \n      AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}       \n    labels:      \n      composeToBatch.jobRoleArn: arn:aws:iam::12345678910:role/my-role\n```\n\nWhen you're ready to deploy to Batch, the following command will safely exclude your secret keys from the generated batch job definition and use the job role instead.\n\n```\n$ cat docker-compose.yml | compose-to-batch\n```\n\n```json\n{\n  \"jobDefinitionName\": \"my-job\",\n  \"type\": \"container\",\n  \"containerProperties\": {\n    \"image\": \"12345678910.dkr.ecr.us-east-1.amazonaws.com/my-job:0.1.0\",\n    \"environment\": [\n      {\n        \"name\": \"REGION\",\n        \"value\": \"us-east-1\"\n      },\n      {\n        \"name\": \"S3_BUCKET\",\n        \"value\": \"some-bucket\"\n      }      \n    ],\n    \"vcpus\": \"2\",\n    \"memory\": \"2000\",\n    \"jobRoleArn\": \"arn:aws:iam::12345678910:role/my-role\"\n  }\n}\n```\n\nHowever, if you DO want to include substituted environment variables, you can use `docker-compose config` instead since it returns the result of the variable substitution.\n\n```\n$ docker-compose.config | compose-to-batch\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fturnerlabs%2Fcompose-to-batch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fturnerlabs%2Fcompose-to-batch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fturnerlabs%2Fcompose-to-batch/lists"}