{"id":20971841,"url":"https://github.com/marco-m/concourse-in-a-box","last_synced_at":"2025-05-14T11:33:58.734Z","repository":{"id":38300053,"uuid":"367672383","full_name":"marco-m/concourse-in-a-box","owner":"marco-m","description":"All-in-one Concourse CI/CD system based on Docker Compose, with Minio S3-compatible storage and HashiCorp Vault secret manager.","archived":false,"fork":false,"pushed_at":"2024-05-06T08:18:02.000Z","size":95,"stargazers_count":12,"open_issues_count":0,"forks_count":7,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-05-07T11:47:49.794Z","etag":null,"topics":["ci-cd","concourse","concourse-demo-pipeline"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/marco-m.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-05-15T15:54:22.000Z","updated_at":"2024-10-18T09:28:08.000Z","dependencies_parsed_at":"2023-01-16T18:01:30.168Z","dependency_job_id":null,"html_url":"https://github.com/marco-m/concourse-in-a-box","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marco-m%2Fconcourse-in-a-box","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marco-m%2Fconcourse-in-a-box/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marco-m%2Fconcourse-in-a-box/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marco-m%2Fconcourse-in-a-box/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/marco-m","download_url":"https://codeload.github.com/marco-m/concourse-in-a-box/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254131941,"owners_count":22020049,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ci-cd","concourse","concourse-demo-pipeline"],"created_at":"2024-11-19T04:05:25.163Z","updated_at":"2025-05-14T11:33:58.721Z","avatar_url":"https://github.com/marco-m.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# concourse-in-a-box\n\nAll-in-one [Concourse] CI/CD system based on Docker Compose, with Minio S3-compatible storage and HashiCorp Vault secret manager. This enables to:\n\n1. Learn Concourse pipelines from scratch in a simple environment.\n2. Troubleshoot production Concourse pipelines in a stand-alone environment.\n3. Write Concourse pipelines that can be reused as-is in your production environment, since it comes with S3 and secret store.\n\n# Security considerations\n\nThis project is NOT adapted for production or networked use.\n\nAmong other non-production ready settings, it contains hard-coded secrets, stored in the git repo. For production use, all secrets must be regenerated and must not be stored in the git repo!\n\n# What's in the box\n\n* [Concourse] v7.8.2 (ATC and web UI)\n* Concourse worker (platform: Linux)\n* [PostgreSQL] v13.2 (needed by Concourse web)\n* [Minio] latest stable S3-compatible object storage. With this, you can learn writing real-world Concourse pipelines using the [concourse-s3-resource] without the need of setting up an AWS S3 (or any other cloud provider) account.\n* [HashiCorp Vault] v1.7.1 secret and credential manager. With this, you can learn writing real-world Concourse pipelines following security and operations best practices. See also [Concourse credential management] for how Concourse uses Vault.\n* the incomplete [Concourse primer](doc/concourse-primer.md) tutorial.\n\n# Usage\n\nThe various credentials are in file [env](./.env) and can be changed if you wish. They will be read automatically by `docker compose`.\n\n## Common setup and teardown\n\n### Setup\n\n* Download the images:\n  ```\n  $ docker compose pull\n  ```\n\n* Start the containers:\n  ```\n  $ docker compose up\n  ```\n\n### Verify setup\n\nThe docker-compose file uses some short-lived containers to perform initialization. Given the amount of log output from `docker compose up`, failures can be hard to notice.\n\nRun `docker compose ps` and confirm that the containers ending with `-setup` have exited with a `0` state. If any of them exited with a different code, then look back at the logs from `docker compose up` and identify the problem.\n\nFor example:\n\n```\n$ docker compose ps | grep setup\nconcourse-in-a-box_minio-setup_1   /scripts/minio-setup.sh          Exit 1\nconcourse-in-a-box_vault-setup_1   /scripts/vault-setup.sh          Exit 0\n```\n\nThe minio setup failed.\n\n### Teardown\n\n* When done, remember to stop the containers:\n  ```\n  $ docker compose stop\n  ```\n* If you want to also delete the persistent volumes, in order to delete the Concourse build history and the contents of the Minio S3 buckets:\n  ```\n  $ docker compose down\n  ```\n\n## Concourse setup\n\n* Point your web browser to http://localhost:8080 and follow the instructions there:\n  * Download the `fly` command-line tool and put it in your $PATH.\n  * Login to the web interface.\n* In another terminal, login with `fly` (will open the web browser to finish authentication):\n  ```\n  $ fly --target=main login --concourse-url=http://localhost:8080 --open-browser\n  ```\n* You can use anything as the value for `--target`, it is an alias for the connection to the given Concourse with the given credentials (see file `$HOME/.flyrc`).\n\n## Minio S3 setup\n\n* The `minio-setup` container creates a bucket named `concourse`.\n* Optional: point your browser to http://localhost:9000 and login.\n* Optional: follow [mc documentation] and install the command-line client `mc`.\n* If you want to create additional buckets, you can add to [scripts/minio-setup.sh](scripts/minio-setup.sh).\n\n## Vault setup\n\n* For the time being vault is configured in dev mode, which means that the storage backend is in memory and will not be persisted to disk.\n* The `vault-setup` container adds the S3 secrets to vault.\n* Optional: point your browser to http://localhost:8200 and login.\n* Optional: follow [vault download], install the command-line utility `vault` and login.\n* If you want to create more secrets, see [scripts/vault-setup.sh](scripts/vault-setup.sh).\n\n# Concourse primer\n\nHave a look at [Concourse incomplete primer](./doc/concourse-primer.md).\n\n# Known issues\n\n* The scheduling of Concourse 7.x is slow, it takes 5-10 seconds to decide what to do next. There are various open tickets about this behavior.\n\n# History and credits\n\nThis project builds upon what I learned in my previous approach, VM-based: [concourse-ci-formula](https://github.com/marco-m/concourse-ci-formula).\n\nThis project is just an humble collection of great open source software.\n\n# License\n\n[MIT](LICENSE).\n\n\n[concourse]: https://concourse-ci.org/\n[concourse credential management]: https://concourse-ci.org/creds.html\n[concourse-s3-resource]: https://github.com/concourse/s3-resource/\n[minio]: https://min.io/\n[mc documentation]: https://docs.min.io/minio/baremetal/reference/minio-cli/minio-mc.html\n[HashiCorp Vault]: https://www.hashicorp.com/products/vault\n[vault download]: https://www.vaultproject.io/downloads\n[PostgreSQL]: https://www.postgresql.org/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarco-m%2Fconcourse-in-a-box","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarco-m%2Fconcourse-in-a-box","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarco-m%2Fconcourse-in-a-box/lists"}