{"id":18400455,"url":"https://github.com/docksal/ci-agent","last_synced_at":"2025-10-25T01:43:26.968Z","repository":{"id":15423613,"uuid":"73348054","full_name":"docksal/ci-agent","owner":"docksal","description":"CI agent Docker image for Docksal Sandboxes","archived":false,"fork":false,"pushed_at":"2024-02-16T17:37:08.000Z","size":237,"stargazers_count":11,"open_issues_count":9,"forks_count":16,"subscribers_count":3,"default_branch":"develop","last_synced_at":"2025-09-06T00:39:05.564Z","etag":null,"topics":["bitbucket-pipelines","circleci","docksal-sandboxes"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/docksal.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null},"funding":{"github":"docksal","open_collective":"docksal"}},"created_at":"2016-11-10T04:48:26.000Z","updated_at":"2025-07-27T00:38:32.000Z","dependencies_parsed_at":"2023-02-10T09:01:34.380Z","dependency_job_id":"d68ec9d5-10ad-4983-89eb-b608ff5dca85","html_url":"https://github.com/docksal/ci-agent","commit_stats":{"total_commits":229,"total_committers":16,"mean_commits":14.3125,"dds":"0.14847161572052403","last_synced_commit":"91de49edbb910d121596e4cc0b8f810ec730fb95"},"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"purl":"pkg:github/docksal/ci-agent","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/docksal%2Fci-agent","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/docksal%2Fci-agent/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/docksal%2Fci-agent/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/docksal%2Fci-agent/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/docksal","download_url":"https://codeload.github.com/docksal/ci-agent/tar.gz/refs/heads/develop","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/docksal%2Fci-agent/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279006839,"owners_count":26084204,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bitbucket-pipelines","circleci","docksal-sandboxes"],"created_at":"2024-11-06T02:33:30.047Z","updated_at":"2025-10-11T10:14:32.997Z","avatar_url":"https://github.com/docksal.png","language":"Shell","funding_links":["https://github.com/sponsors/docksal","https://opencollective.com/docksal"],"categories":[],"sub_categories":[],"readme":"# CI agent Docker image for Docksal Sandboxes\n\nAn agent used to connect CI providers to a [Docksal Sandbox Server](https://github.com/docksal/sandbox-server).\n\nSupported CI providers:\n\n- Bitbucket Pipelines (with build status integration)\n- CircleCI (with build status integration)\n- GitLab\n- Github Actions\n\nThis image(s) is part of the [Docksal](http://docksal.io) image library.\n\n\n## Docksal Sandboxes\n\nDocksal Sandboxes are continuous integration environments powered by Docksal and deployed to a [Docksal Sandbox Server](https://github.com/docksal/sandbox-server).\nThey can be provisioned for any git branch and are feature-wise identical to project's local Docksal environment.\n\nUse cases:\n\n- automated testing (full stack)\n- manual testing\n- enhanced pull request review experience\n- demos\n\nBuild status updates (and sandbox URLs) can be posted to Github and Bitbucket via respective build status APIs.  \nURLs to sandbox environments can also be published to a Slack channel.  \n\n\n## Image variants and versions\n\n### Stable\n\n- `base` - basic (bash, curl, git, etc.), latest version\n- `php`, `latest` - basic + php stack tools (composer, drush, drupal console, wp-cli, etc.), latest version\n- `1.2-base` - basic, a specific stable version\n- `1.2-php` - php, a specific stable version\n\n### Development\n\n- `edge-base` - base, latest development version\n- `edge-php`, php, latest development version\n\n\n## Global Configuration\n\n### Required\n\nThe following required variables are usually configured at the organization level. This way, all project repos will \nhave access to them. They can as well be configured at the repo level.\n\n`DOCKSAL_HOST` or `DOCKSAL_HOST_IP`\n\nThe address of the sandbox server. Configure one or the other.  \nIf using `DOCKSAL_HOST`, make sure the domain is configured as a wildcard DNS entry.  \nIf using `DOCKSAL_HOST_IP`, the agent will use `nip.io` for dynamic wildcard domain names for sandboxes. \n\n`DOCKSAL_HOST_SSH_KEY`\n\nA base64 encoded private SSH key, used to access the sandbox server.\n\nNote: on macOS `cat /path/to/\u003cprivate_key_file\u003e | base64` can be used to create a base64 encoded string from a private SSH key, while on Linux and in WSL on Windows 10 `cat /path/to/\u003cprivate_key_file\u003e | base64 -w 0` should be used to avoid output wrapping of the `base64` command).\n\n### Optional\n\n`BUILD_ENVIRONMENT`\n\nUsed to set the environment built against. Defaults to `local`.\n\n`CI_SSH_KEY`\n\nA base64 encoded private SSH key, used by default for all hosts (set as `Host *` in `~/.ssh/config`).\nThis key will be used to clone/push to git, run commands over SSH on a remote deployment environment, etc.\n\n`DOCKSAL_HOST_SSH_PORT`\n\nCan be used to override the SSH port of the sandbox server where the agent connects (defaults to `22`).\n\n`DOCKSAL_DOMAIN`\n\nCan be used to set the base URL for sandbox builds (defaults to `DOCKSAL_HOST` if not set), individually from `DOCKSAL_HOST`.  \nThis is useful when working with CDNs/ELBs/WAFs/etc (when `DOCKSAL_DOMAIN` is different from the `DOCKSAL_HOST`).\n\n`DOCKSAL_HOST_USER`\n\nThe user name used as the build user on the sandbox server. Defaults to `build-agent`. Must have SSH access to the server.\n\n`REMOTE_BUILD_BASE`\n\nThe default directory location on the sandbox server where the repositories should be cloned down to and built. \nDefaults to `/home/build-agent/builds`.\n\n`REMOTE_CODEBASE_METHOD`\n\nPick between `rsync` (default) and `git` for the codebase initialization method on the sandbox server.\n\nThe codebase is initialized on the sandbox server by the `sandbox-init` (or `build-init`) command.\n\n`git` - code is checkout on the sandbox server via git. The server must have access to checkout from the repo. \nAny build settings and necessary code manipulations must happen on the sandbox server using `build-exec` commands.\n\n`rsync` - code is rsync-ed to the sandbox server from the build agent. You can perform necessary code adjustments in the \nbuild agent after running `build-env` and before running `sandbox-init` (or `build-init`), which pushes the code to the \nsandbox server.\n\n`REMOTE_BUILD_DIR_CLEANUP`\n\nWhether or not the remote build directory is reset during the build. Only supported with `REMOTE_CODEBASE_METHOD=git`.\n\nDefaults to `1` which wipes the remote build directory and produces a \"clean build\".    \nSet to `0` to produce \"dirty builds\", when file changes in the remote codebase should be preserved.\n\nNote: Switching `REMOTE_CODEBASE_METHOD` mode will result in a clean build. \n\n`SANDBOX_PERMANENT`\n\nSet `SANDBOX_PERMANENT=true` to have a permanent sandbox provisioned.\n\nPermanent sandboxes are exempt from scheduled garbage collection on the sandbox server. They would still hibernate after\nthe configured period of inactivity, but won't be removed from the server after becoming dangling.\nSee https://github.com/docksal/service-vhost-proxy#advanced-proxy-configuration for more information. \n\nThis variable is usually set at the branch level in the build settings to designate a specific (one or multiple) \nbranch environments as permanent.\n\n`SANDBOX_DOMAIN`\n\nSets a custom domain for a sandbox. Takes precedence over the automatic (branch name based) domain generation.\n\nThis can be used for sandbox environments which need a custom (nice) domain name.\n\n`GITHUB_TOKEN` and `BITBUCKET_TOKEN`\n\nUsed for access to post sandbox URLs via build status API as well as comments on pull requests.  \n\nFor Github, the token can be generated from the [user's account](https://github.com/settings/tokens).  \nSet access to \"repo\" (http://take.ms/nMqcW).\n\nFor Bitbucket, the token can be generated from the user's settings. Instructions on creating an [app password](https://confluence.atlassian.com/bitbucket/app-passwords-828781300.html).  \nSet access to \"Repositories: Write\", \"Pull requests: Write\" (http://take.ms/98BG5).  \nWhen storing the app password it is in the format: `USER:PASSWORD`.\n\n`GIT_USER_EMAIL`\n\nThe user's email to perform Git operations as. Defaults to `ci@docksal.io`\n\n`GIT_USER_NAME`\n\nThe user's name to perform Git operations as. Defaults to `Docksal CI`\n\n`DOCKSAL_HOST_TUNNEL`\n\nIf not empty, `localhost:2374` in the agent is mapped to `docker.sock` on the remote `DOCKSAL_HOST` via a secure SSH tunnel.  \nThe agent can then run `docker` commands against the remote `DOCKSAL_HOST`.\n\n\nOther features and integrations are usually configured at the repo level. See below.\n\n\n## Project configuration\n\n### Bitbucket Pipelines\n\nHere's the most basic configuration for Bitbucket Pipelines. Save it into `bitbucket-pipelines.yml` in your project repo.\n\n```yaml\nimage: docksal/ci-agent:base\n\npipelines:\n  default:\n    - step:\n        script:\n          - source build-env \u0026\u0026 sandbox-init\n```\n\nFor a more advanced example see [bitbucket-pipelines.yml](examples/bitbucket-pipelines/bitbucket-pipelines.yml).\n\n### CircleCI\n\nHere's the most basic configuration for CircleCI. Save it into `.circleci/config.yml` in your project repo.\n\n```yaml\nversion: 2\n\njobs:\n  build:\n    working_directory: /home/agent/build\n    docker:\n      - image: docksal/ci-agent:base\n    steps:\n      - run:\n          name: Configure agent environment\n          command: echo 'source build-env' \u003e\u003e $BASH_ENV\n      - checkout\n      - run:\n          name: Build sandbox\n          command: sandbox-init\n```\n\nFor a more advanced example see [config.yml](examples/.circleci/config.yml).\n\n### GitLab\n\nHere's the most basic configuration for GitLab. Save it into `.gitlab-ci.yml` in your project repo.\n\n```yaml\nstages:\n  - sandbox\n\nsandbox-launch:\n  stage: sandbox\n  image: docksal/ci-agent:base\n  script:\n    - export SANDBOX_DOMAIN=$CI_ENVIRONMENT_SLUG--$CI_PROJECT_NAME.$DOCKSAL_HOST\n    - source build-env\n    - sandbox-init\n  environment:\n    name: $CI_COMMIT_REF_NAME\n    url: https://$CI_ENVIRONMENT_SLUG--$CI_PROJECT_NAME.$DOCKSAL_HOST\n```\n\nFor a more advanced example see [.gitlab-ci.yml](examples/gitlab/.gitlab-ci.yml).\n\n### Github Actions\n\nHere's the most basic configuration for Github Actions. Save it into `.github/workflows/sandbox.yml` in your project repo.\n\nYou might need to add your ssh key to EC2 `/home/build-agent/.ssh` and the deploy(public) key to Github Actions for cloning the codebase via git.\n\n```yaml\nname: Docksal sandbox\n\non:\n  workflow_dispatch:\n\njobs:\n  build:\n    runs-on: ubuntu-latest\n    container:\n      image: docksal/ci-agent:base\n    env:\n      CI_SSH_KEY: ${{ secrets.CI_SSH_KEY }}\n      DOCKSAL_HOST_IP: ${{ secrets.DOCKSAL_HOST_IP }}\n      DOCKSAL_HOST_SSH_KEY: ${{ secrets.DOCKSAL_HOST_SSH_KEY }}\n      GITHUB_TOKEN: ${{ github.token }}\n      REMOTE_CODEBASE_METHOD: git\n    steps:\n      - name: Build sandbox\n        shell: bash\n        run: |\n          source build-env\n          sandbox-init\n```\n\nFor a more advanced example see [.sandbox.yml](examples/.github/workflows/sandbox.yml).\n\n## Build commands\n\nFor a complete list of built-in commands see [base/bin](base/bin).\n\n- `build-env` - configures build settings on the agent. Usage: `source build-env` (or `DEBUG=1 source build-env`)\n- `build-init`- initializes the sandbox codebase and settings on the sandbox server. Usage: `build-init`\n- `build-exec` - executes a shell command within the build directory on the sandbox server. Usage: `build-init pwd`\n- `build-notify` - see \"Build status notifications\" docs below\n- `sandbox-init` - a convenient shortcut to provision a basic sandbox. See [sandbox-init](base/bin/sandbox-init) \n\n\n## Build environment variables\n\nThe following variables are derived from the respective Bitbucket Pipelines, Circle CI, GitLab CI, and Github Actions build variables. \n\n- `GIT_REPO_OWNER` - git repo machine owner/slug name\n- `GIT_REPO_NAME` - git repo machine name\n- `GIT_REPO_URL` - git repo URL\n- `GIT_BRANCH_NAME` - git branch name\n- `GIT_COMMIT_HASH` - git commit hash\n- `GIT_PR_NUMBER` - git pull request / merge request number\n- `GIT_REPO_SERVICE` - `github`, `bitbucket` or `gitlab` (makes sense mostly for CircleCI)\n- `BUILD_ID` - The unique identifier for a build\n- `BUILD_DIR` - The full path where the repository is cloned and where the job is run in the agent container\n\n`REMOTE_BUILD_DIR`\n\nThe directory location on the sandbox server where current build will happen. Defaults to:\n\n```\n${REMOTE_BUILD_BASE}/${REPO_NAME_SAFE}-${BRANCH_NAME_SAFE}\n```\n\n\n## Feature: Basic HTTP Auth\n\nProtect sandboxes from public access using Basic HTTP authentication.\n\n### Configuration\n\nSet the following environment variables at the repo level:\n\n- `HTTP_USER`\n- `HTTP_PASS`\n\n\n## Feature: Build status notifications\n\nThis integration allows the agent to post build status updates and sandbox URL via Github/Bitbucket build status API.  \nFor CircleCI, it is also possible to enable posting the sandbox URL as a comment in pull requests. \n\n### Configuration\n\n`GITHUB_TOKEN` or `BITBUCKET_TOKEN` must be configured respectively (either globally or at the repo level). \n\n### Usage\n\n`build-notify \u003cpending|success|failure\u003e`\n\nPlace the triggers right before and right after `fin init` call in your build script, e.g.,\n\n```bash\nbuild-notify pending \nssh docker-host \"cd $REMOTE_BUILD_DIR \u0026\u0026 fin init\" || ( build-notify failure \u0026\u0026 exit 1 )\nbuild-notify success\n```\n\nTo enable posting sandbox URLs in comments on pull requests, do `export PR_COMMENT=1` prior to calling `build-notify`\n\n\n## Feature: Slack notifications\n\nThis integrations allows the agent to post messages to a given Slack channel.  \nIt can be used for notification purposes when a build is started, completed, failed, etc.\n\n### Configuration\n\n`SLACK_WEBHOOK_URL`\n\nThe Incoming Webhook integration URL from Slack, \ne.g., `SLACK_WEBHOOK_URL https://hooks.slack.com/services/XXXXXXXXX/XXXXXXXXX/XXxxXXXXxxXXXXxxXXXXxxXX`\n\n`SLACK_CHANNEL`\n\nA public or private channel in Slack, e.g., `SLACK_CHANNEL #project-name-bots`\n\n`SLACK_USER`\n\nThe username the message should post to Slack as. Defaults to `Docksal CI`\n\n`SLACK_ICON`\n\nThe icon the message should use to accompany the message: Defaults to `:desktop_computer:`\n\n### Usage\n\n```bash\nslack 'message' ['#channel'] ['webhook_url'] ['slack_user'] ['slack_icon']\n```\n\nChannel and webhook url can be passed via environment variables. See above. \n\n### Limitations\n\nIncoming Webhook integration won't work for private channels, which the owner of the integration does not belong to.\n\n\n## Feature: Build artifact storage\n\nBuild artifacts can be stored in an AWS S3 bucket.  \n\n### Configuration\n\nSet the following environment variables at the organization or repo level:\n\n- `AWS_ACCESS_KEY_ID`\n- `AWS_SECRET_ACCESS_KEY`\n- `ARTIFACTS_BUCKET_NAME`\n- `ARTIFACTS_BASE_URL` (optional)\n- `GITHUB_TOKEN` (optional)\n- `BITBUCKET_TOKEN` (optional)\n\n### Usage\n\nTo upload artifacts to the configured S3 bucket use the `build-acp` command.\n\n```bash\nbuild-acp /source/path/\n```\n\nThere is no file browsing capability available for private S3 buckets.  \nAn `index.html` file is used as the directory index, however it has to be created/generated manually.  \nWhen uploading a prepared artifacts folder with the `index.html` file in it, add a trailing slash to the source path to\nupload the contents of the source folder vs the folder itself.\n\nYou can upload additional folders/files by running the command multiple times.\n\nThe optional `destination` argument can be used to define a subdirectory at the destination in the bucket.  \n\n```bash\nbuild-acp /source/path/ destination/path\n```\n\n### Advanced usage\n\n**Base URL**\n\nThe base URL is derived from `ARTIFACTS_BUCKET_NAME` as follows (assuming AWS S3 `us-east-1` region by default):\n\n```\nhttps://${ARTIFACTS_BUCKET_NAME}.s3.amazonaws.com\n```\n\nIt can be overridden via the optional `ARTIFACTS_BASE_URL` configuration variable at the organization/repo level:\n\n```\nARTIFACTS_BASE_URL = https://artifacts.example.com\n```\n\n**Upload path**\n\nThe upload path is unique for each commit and is derived as follows: \n\n```bash\n${REPO_NAME_SAFE}/${BRANCH_NAME_SAFE}-${GIT_COMMIT_HASH}\n```\n\nIn certain cases you may want to store build artifacts per branch instead of per commit.  \nTo do this, override the `ARTIFACTS_BUCKET_PATH` variable before calling the `build-acp` command: \n\n```bash\nexport ARTIFACTS_BUCKET_PATH=\"${REPO_NAME_SAFE}/${BRANCH_NAME_SAFE}\"\nbuild-acp my-artifacts/\n```\n\n**Posting build artifact URLs to Bitbucket**\n\nIf `BITBUCKET_TOKEN` is set, the URL to the artifacts will be posted back to Bitbucket via \n[Bitbucket Build Status API](https://blog.bitbucket.org/2015/11/18/introducing-the-build-status-api-for-bitbucket-cloud/).\n\n### Security\n\nIf a bucket does not exist, it will be created automatically (with no public access). Existing bucket access permissions \nare not automatically adjusted. It's up to you whether you want to keep them open or not.\n\nWhen artifacts are uploaded, the destination artifact folder in the bucket is set to be publicly accessible. \nAnyone with the direct link will be able to access the artifacts, but will not be able to browse the list of all \navailable artifact folders in the bucket (so long as the bucket itself is set to be private). \n\nThe URL by default includes a git commit hash, which serves as an authentication token (the URL is impossible to guess). \nThis provides a simple yet efficient level of security for artifacts.\n\nTo add an additional level of security follow [this guide](https://medium.com/@lmakarov/serverless-password-protecting-a-static-website-in-an-aws-s3-bucket-bfaaa01b8666) \nto set up username/password access to S3 via CloudFront and Lambda@Edge.\n\n\n## Feature: Non-volatile environments\n\nBy combining the following configuration options you can get low overhead non-volatile environments.\n\n```\nSANDBOX_DOMAIN=\u003cnice-domain\u003e\nSANDBOX_PERMANENT=true\nREMOTE_CODEBASE_METHOD=git\nREMOTE_BUILD_DIR_CLEANUP=0\n``` \n\nSuch environments can be used for non-critical production-ish workloads, whenever an on-demand delayed start \n(5-10s delay) is not a concern.\n\n## Feature: Secrets in environment variables\n\nIt is best security practice not to store secrets such as API keys in a code repository. Many CI systems already have the ability to set such environment variables during the build process. Any environment variables set at build time whose name starts with `SECRET_` will be forwarded as-is to the built environment.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdocksal%2Fci-agent","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdocksal%2Fci-agent","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdocksal%2Fci-agent/lists"}