{"id":19259248,"url":"https://github.com/concourse/pool-resource","last_synced_at":"2026-05-02T21:04:43.999Z","repository":{"id":33253079,"uuid":"36897472","full_name":"concourse/pool-resource","owner":"concourse","description":"atomically manages the state of the world (e.g. external environments)","archived":false,"fork":false,"pushed_at":"2025-03-26T18:05:22.000Z","size":488,"stargazers_count":57,"open_issues_count":17,"forks_count":36,"subscribers_count":25,"default_branch":"master","last_synced_at":"2025-04-04T12:05:03.593Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/concourse.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-06-04T21:56:21.000Z","updated_at":"2025-03-13T15:54:11.000Z","dependencies_parsed_at":"2024-06-18T18:11:59.531Z","dependency_job_id":"4489a159-ef1c-4b56-9fe7-c505f9d0ef6f","html_url":"https://github.com/concourse/pool-resource","commit_stats":null,"previous_names":[],"tags_count":21,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/concourse%2Fpool-resource","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/concourse%2Fpool-resource/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/concourse%2Fpool-resource/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/concourse%2Fpool-resource/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/concourse","download_url":"https://codeload.github.com/concourse/pool-resource/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248478000,"owners_count":21110613,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T19:15:50.969Z","updated_at":"2026-05-02T21:04:43.992Z","avatar_url":"https://github.com/concourse.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pool-resource\n\n*a pool of locks (modeling semaphores)*\n\n\u003ca href=\"https://ci.concourse-ci.org/teams/main/pipelines/resource/jobs/build?vars.type=%22pool%22\"\u003e\n  \u003cimg src=\"https://ci.concourse-ci.org/api/v1/teams/main/pipelines/resource/jobs/build/badge?vars.type=%22pool%22\" alt=\"Build Status\"\u003e\n\u003c/a\u003e\n\nAllows you to lock environments, pipeline flow, or other entities which have to\nbe interacted with in a serial manner.\n\nThis resource is backed by a Git repository, so the configuration is largely the\nsame as the git-resource.\n\n## Git Repository Structure\n\n```\n.\n├── aws\n│   ├── claimed\n│   │   ├── .gitkeep\n│   │   └── env-2\n│   └── unclaimed\n│       ├── .gitkeep\n│       └── env-1\n├── ping-pong-tables\n│   ├── claimed\n│   │   └── .gitkeep\n│   └── unclaimed\n│       ├── .gitkeep\n│       └── north-table\n└── vsphere\n    ├── claimed\n    │   ├── .gitkeep\n    │   └── f3cb3823-a45a-49e8-ab41-e43268494205\n    └── unclaimed\n        ├── .gitkeep\n        └── 83ed9977-3a10-4c49-a818-2d7a37693da7\n```\n\nThis structure represents 3 pools of locks, `aws`, `ping-pong-tables`, and\n`vsphere`. The `.gitkeep` files are required to keep the `unclaimed` and\n`claimed` directories track-able by Git if there are no files in them.\n\nYou will need to mirror this structure in your own lock repository. In other words, initialize an empty repository\nand create one directory in the root of the repository for each pool of locks (e.g. `aws`).  Inside of each lock pool directory, create one directory\nnamed `claimed` and one directory named `unclaimed`. Inside each `claimed` and `unclaimed` directory, create an empty\nfiled named `.gitkeep`. Finally, create individual locks by making an empty file inside of the `unclaimed` directory\n(assuming you want your lock to be unclaimed by default) with the desired name of the lock (e.g. `env-1`).\n\n## Source Configuration\n\n* `uri`: *Required.* The location of the repository.\n\n* `branch`: *Required.* The branch to track.\n\n* `pool`: *Required.* The logical name of your pool of things to lock.\n\n* `private_key`: *Optional.* Private key to use when pulling/pushing. Ensure it does not require a password.\n    Example:\n    ```\n    private_key: |\n      -----BEGIN RSA PRIVATE KEY-----\n      MIIEowIBAAKCAQEAtCS10/f7W7lkQaSgD/mVeaSOvSF9ql4hf/zfMwfVGgHWjj+W\n      \u003cLots more text\u003e\n      DWiJL+OFeg9kawcUL6hQ8JeXPhlImG6RTUffma9+iGQyyBMCGd1l\n      -----END RSA PRIVATE KEY-----\n    ```\n\n* `private_key_user`: *Optional.* Enables setting `User` in the ssh config.\n\n* `private_key_passphrase`: *Optional.* To unlock `private_key` if it is protected by a passphrase.\n\n* `username`: *Optional.* Username for HTTP(S) auth when pulling/pushing.\n  This is needed when only HTTP/HTTPS protocol for git is available (which does not support private key auth) and auth is required.\n\n* `password`: *Optional.* Password for HTTP(S) auth when pulling/pushing.\n\n* `git_config`: *Optional.* If specified as (list of pairs `name` and `value`)\n  it will configure git global options, setting each name with each value.\n\n  This can be useful to set options like `credential.helper` or similar.\n\n  See the [`git-config(1)` manual page](https://www.kernel.org/pub/software/scm/git/docs/git-config.html)\n  for more information and documentation of existing git options.\n\n* `retry_delay`: *Optional.* If specified, dictates how long to wait until\n  retrying to acquire a lock or release a lock. The default is 10 seconds.\n  Valid values: `60s`, `90m`, `1h`.\n\n* `https_tunnel`: *Optional.* Information about an HTTPS proxy that will be used to tunnel SSH-based git commands over.\n  Has the following sub-properties:\n  * `proxy_host`: *Required.* The host name or IP of the proxy server\n  * `proxy_port`: *Required.* The proxy server's listening port\n  * `proxy_user`: *Optional.* If the proxy requires authentication, use this username\n  * `proxy_password`: *Optional.* If the proxy requires authentication, use this password\n\n* `forward_agent`: *Optional.* Enables ForwardAgent SSH option when set to true. Useful when using proxy/jump hosts. Defaults to false.\n\n* `skip_ssl_verification`: *Optional.* Skips git ssl verification by exporting `GIT_SSL_NO_VERIFY=true`.\n\n### Example\n\nFetching a repo with only 100 commits of history:\n\n``` yaml\n- get: source-code\n  params: {depth: 100}\n```\n\n## Behavior\n\n### `check`: Check for changes to the pool.\n\nThe repository is cloned (or pulled if already present), and any commits made to the specified pool from the given version on are returned. If no version is\ngiven, the ref for `HEAD` is returned.\n\n\n### `in`: Fetch an acquired lock.\n\n\nOutputs 2 files:\n\n* `metadata`: Contains the contents of whatever was in your lock file. This is\n  useful for environment configuration settings.\n\n* `name`: Contains the name of lock that was acquired.\n\n#### Parameters\n\n* `depth`: *Optional.* If a positive integer is given, *shallow* clone the\n  repository using the `--depth` option.\n\n### `out`: Acquire, release, add, or remove a lock.\n\nPerforms one of the following actions to change the state of the pool.\n\n#### Parameters\n\nOne of the following is required.\n\n* `acquire`: If true, we will attempt to move a randomly chosen lock from the\n  pool's unclaimed directory to the claimed directory. Acquiring will retry\n  until a lock becomes available.\n\n* `claim`: If set, the specified lock from the pool will be acquired, rather\n  than a random one (as in `acquire`). Like `acquire`, claiming will retry\n  until the specific lock becomes available.\n\n* `release`: If set, we will release the lock by moving it from claimed to\n  unclaimed. The value is the path of the lock to release (a directory\n  containing `name` and `metadata`), which typically is just the step that\n  provided the lock (either a `get` to pass one along or a `put` to acquire).\n\n  Note: the lock must be available in your job before you can release it. In\n  other words, a `get` step to fetch metadata about the lock is necessary\n  before a `put` step can release the lock.\n\n* `add`: If set, we will add a new lock to the pool in the unclaimed state. The\n  value is the path to a directory containing the files `name` and `metadata`\n  which should contain the name of your new lock and the contents you would like\n  in the lock, respectively.\n\n* `add_claimed`: Exactly the same as the `add` param, but adds a lock to the\n  pool in the *claimed* state. \n\n* `remove`: If set, we will remove the given lock from the pool. The value is\n  the same as `release`. This can be used for e.g. tearing down an environment,\n  or moving a lock between pools by using `add` with a different pool in a\n  second step.\n\n* `update`: If set, we will update an existing lock in the pool.\n\n  * If the existing lock is in the unclaimed state we will update it with the\n    contents of the `metadata` file.\n  * If no such lock is present in either the claimed or unclaimed state we add a\n    new lock to the pool in the unclaimed state.\n  * If the lock is in the claimed state we will wait for it to be unclaimed and\n    proceed to update it as above.\n\n  The value is the path to a directory containing the files `name` and\n  `metadata` which should contain the name of your new lock and the contents you\n  would like in the lock, respectively.\n\n* `check`: If set, we will check for an existing claimed lock in the pool and\n  wait until it becomes unclaimed.\n\n  * If there is an existing lock in claimed state: wait until lock is unclaimed\n  * If there is an existing lock in unclaimed state: no-op\n  * If no lock exists: fail\n\n  The purpose is to simply block until a given lock in a pool is moved from a\n  claimed state to an unclaimed state. This functionality allows us to build\n  dependencies between disparate pipelines without the need to `acquire` locks.\n\n  Note: the lock must be present to perform a check. In other words, a `get`\n  step to fetch metadata about the lock is necessary before a `put` step can\n  check the existence of the lock.\n\n* `check_unclaimed`: If set, we will check for an existing unclaimed lock in the pool and\n  wait until it becomes claimed.\n\n  * If there is an existing lock in unclaimed state: wait until lock is unclaimed\n  * If there is an existing lock in claimed state: no-op\n  * If no lock exists: fail\n\n  The purpose is to simply block until a given lock in a pool is moved from an\n  unclaimed state to a claimed state. This functionality allows us to build\n  dependencies between disparate pipelines without the need to `acquire` locks.\n\n  Note: the lock must be present to perform a check. In other words, a `get`\n  step to fetch metadata about the lock is necessary before a `put` step can\n  check the existence of the lock.\n\n## Example Concourse Configuration\n\nThe following example pipeline models acquiring, passing through, and releasing\na lock based on the example git repository structure:\n\n```\nresources:\n- name: aws-environments\n  type: pool\n  source:\n    uri: git@github.com:concourse/locks.git\n    branch: master\n    pool: aws\n    private_key: |\n      -----BEGIN RSA PRIVATE KEY-----\n      MIIEowIBAAKCAQEAtCS10/f7W7lkQaSgD/mVeaSOvSF9ql4hf/zfMwfVGgHWjj+W\n      \u003cLots more text\u003e\n      DWiJL+OFeg9kawcUL6hQ8JeXPhlImG6RTUffma9+iGQyyBMCGd1l\n      -----END RSA PRIVATE KEY-----\n\njobs:\n- name: deploy-aws\n  plan:\n    - put: aws-environments\n      params: {acquire: true}\n    - task: deploy-aws\n      file: my-scripts/deploy-aws.yml\n\n- name: test-aws\n  plan:\n    - get: aws-environments\n      passed: [deploy-aws]\n    - task: test-aws\n      file: my-scripts/test-aws.yml\n    - put: aws-environments\n      params: {release: aws-environments}\n```\n\n\n### Managing multiple locks\n\nThe parameter for `release` is the name of the step whose lock to release.\nNormally this is just the same as the name of the resource, but if you have\ncustom names for `put` (for example if you need to acquire multiple instances\nof the same pool), you would put that name instead. For example:\n\n```\n- name: test-multi-aws\n  plan:\n    - put: environment-1\n      resource: aws-environments\n      params: {acquire: true}\n    - put: environment-2\n      resource: aws-environments\n      params: {acquire: true}\n    - task: test-multi-aws\n      file: my-scripts/test-multi-aws.yml\n    - put: aws-environments\n      params: {release: environment-1}\n    - put: aws-environments\n      params: {release: environment-2}\n```\n\n### Claiming a specific lock from a pool, and releasing\n\nThe parameter for `claim` takes the name of a lock, and releasing the\nlock requires the name of the step which acquired the lock. For\nexample:\n\n```\n- name: test-specific-aws\n  plan:\n    - put: specific-aws-env\n      resource: aws-environments\n      params: {claim: env-1}\n    - put: aws-environments\n      params: {release: specific-aws-env}\n```\n\n### Configuring resource to proxy SSH commands through an HTTP proxy\n\n```\nresources:\n- name: aws-environments\n  type: pool\n  source:\n    uri: git@github.com:concourse/locks.git\n    branch: master\n    pool: aws\n    private_key: |\n      -----BEGIN RSA PRIVATE KEY-----\n      MIIEowIBAAKCAQEAtCS10/f7W7lkQaSgD/mVeaSOvSF9ql4hf/zfMwfVGgHWjj+W\n      \u003cLots more text\u003e\n      DWiJL+OFeg9kawcUL6hQ8JeXPhlImG6RTUffma9+iGQyyBMCGd1l\n      -----END RSA PRIVATE KEY-----\n    https_tunnel:\n      proxy_host: proxy-server.mycorp.com\n      proxy_port: 3128\n      proxy_user: myuser\n      proxy_password: myverysecurepassword\n```\n\n## Development\n\n### Prerequisites\n\n* golang is *required* - version 1.9.x is tested; earlier versions may also\n  work.\n* docker is *required* - version 17.06.x is tested; earlier versions may also\n  work.\n* godep is used for dependency management of the golang packages.\n\n### Running the tests\n\nThe tests have been embedded with the `Dockerfile`; ensuring that the testing\nenvironment is consistent across any `docker` enabled platform. When the docker\nimage builds, the test are run inside the docker container, on failure they\nwill stop the build.\n\nRun the tests with the following command:\n\n```sh\ndocker build -t pool-resource --target tests .\n```\n\n#### Note about the integration tests\n\nIf you want to run the integration tests, a bit more work is required. You will require\nan actual git repo to which you can push and pull, configured for SSH access. To do this,\nadd two files to `integration-tests/ssh` (note that names **are** important):\n* `test_key`: This is the private key used to authenticate against your repo.\n* `test_repo`: This file contains one line of the form `test_repo_url[#test_branch]`.\n  If the branch is not specified, it defaults to `main`. For example,\n  `git@github.com:concourse-git-tester/git-resource-integration-tests.git` or\n  `git@github.com:concourse-git-tester/git-resource-integration-tests.git#testing`\n\nTo set up or reset the contents of the repo, use the `integration-tests/ssh/init-repo.sh` script. \nThe script clones the configured repository, (re-)creates the relevant directories,\ncommits and pushes the changes. If you'd rather execute the commands yourself, view the script\ncontents to understand the directory structure expected by the integration tests. \n\nThen run the tests:\n\n```sh\ndocker build -t pool-resource --target integrationtests .\n```\n\n### Contributing\n\nPlease make all pull requests to the `master` branch and ensure tests pass\nlocally.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconcourse%2Fpool-resource","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fconcourse%2Fpool-resource","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconcourse%2Fpool-resource/lists"}