{"id":13472068,"url":"https://github.com/travis-ci/worker","last_synced_at":"2025-05-16T01:04:35.603Z","repository":{"id":24025601,"uuid":"27410275","full_name":"travis-ci/worker","owner":"travis-ci","description":"Worker runs your Travis CI jobs","archived":false,"fork":false,"pushed_at":"2025-05-09T20:04:15.000Z","size":6011,"stargazers_count":278,"open_issues_count":34,"forks_count":86,"subscribers_count":27,"default_branch":"master","last_synced_at":"2025-05-09T21:22:39.845Z","etag":null,"topics":["docker","go","google-cloud","travis-ci","vmware-esxi"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/travis-ci.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2014-12-02T02:02:55.000Z","updated_at":"2025-01-01T16:39:20.000Z","dependencies_parsed_at":"2023-01-14T00:15:42.030Z","dependency_job_id":"f70426e5-0c1f-4ffa-ab99-a1c2d41400b2","html_url":"https://github.com/travis-ci/worker","commit_stats":{"total_commits":1865,"total_committers":51,"mean_commits":"36.568627450980394","dds":0.6648793565683646,"last_synced_commit":"f0b6bce00ed2d90a88a3e6f5dbd070b53e4ffb33"},"previous_names":[],"tags_count":107,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/travis-ci%2Fworker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/travis-ci%2Fworker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/travis-ci%2Fworker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/travis-ci%2Fworker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/travis-ci","download_url":"https://codeload.github.com/travis-ci/worker/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254448579,"owners_count":22072764,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","go","google-cloud","travis-ci","vmware-esxi"],"created_at":"2024-07-31T16:00:51.608Z","updated_at":"2025-05-16T01:04:35.579Z","avatar_url":"https://github.com/travis-ci.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# Worker [![Build Status](https://travis-ci.org/travis-ci/worker.svg?branch=master)](https://travis-ci.org/travis-ci/worker)\n\nWorker is the component of Travis CI that will run a CI job on some form of\ncompute instance.\n\nIt's responsible for getting the bash script from\n[travis-build](https://github.com/travis-ci/travis-build), spinning up the\ncompute instance (VM, Docker container, LXD container, or maybe something different),\nuploading the bash script, running it, and streaming the logs back to\n[travis-logs](https://github.com/travis-ci/travis-logs). It also sends state\nupdates to [travis-hub](https://github.com/travis-ci/travis-hub).\n\n## Installing\n\n### from binary\n\nFind the version you wish to install on the [GitHub Releases\npage](https://github.com/travis-ci/worker/releases) and download either the\n`darwin-amd64` binary for macOS or the `linux-amd64` binary for Linux. No other\noperating systems or architectures have pre-built binaries at this time.\n\n### from package\n\nUse the [`./bin/travis-worker-install`](./bin/travis-worker-install) script,\nor take a look at the [packagecloud\ninstructions](https://packagecloud.io/travisci/worker/install).\n\n### from snap\n\nUsing a linux distribution which supports [Snaps](https://snapcraft.io/store)\nyou can run: `sudo snap install travis-worker --edge`\n\n### from source\n\n1. install [Go](http://golang.org) `v1.7+`\n1. clone this down into your `$GOPATH`\n  * `mkdir -p $GOPATH/src/github.com/travis-ci`\n  * `git clone https://github.com/travis-ci/worker $GOPATH/src/github.com/travis-ci/worker`\n  * `cd $GOPATH/src/github.com/travis-ci/worker`\n1. install [gometalinter](https://github.com/alecthomas/gometalinter):\n  * `go get -u github.com/alecthomas/gometalinter`\n  * `gometalinter --install`\n1. install [shellcheck](https://github.com/koalaman/shellcheck)\n1. `make`\n\n\n## Configuring Travis Worker\n\nTravis Worker is configured with environment variables or command line flags via\nthe [urfave/cli](https://github.com/urfave/cli) library.  A list of\nthe non-dynamic flags and environment variables may be found by invoking the\nbuilt-in help system:\n\n``` bash\ntravis-worker --help\n```\n\n### Environment-based image selection configuration\n\nSome backend providers support image selection based on environment variables.\nThe required format uses keys that are prefixed with the provider-specific\nprefix:\n\n- `TRAVIS_WORKER_{UPPERCASE_PROVIDER}_IMAGE_{UPPERCASE_NAME}`: contains an image name\n  string to be used by the backend provider\n\nThe following example is for use with the Docker backend:\n\n``` bash\n# matches on `dist: trusty`\nexport TRAVIS_WORKER_DOCKER_IMAGE_DIST_TRUSTY=travisci/ci-connie:packer-1420290255-fafafaf\n\n# matches on `dist: bionic`\nexport TRAVIS_WORKER_DOCKER_IMAGE_DIST_BIONIC=registry.business.com/fancy/ubuntu:bionic\n\n# resolves for `language: ruby`\nexport TRAVIS_WORKER_DOCKER_IMAGE_RUBY=registry.business.com/travisci/ci-ruby:whatever\n\n# resolves for `group: edge` + `language: python`\nexport TRAVIS_WORKER_DOCKER_IMAGE_GROUP_EDGE_PYTHON=travisci/ci-garnet:packer-1530230255-fafafaf\n\n# used when no dist, language, or group matches\nexport TRAVIS_WORKER_DOCKER_IMAGE_DEFAULT=travisci/ci-garnet:packer-1410230255-fafafaf\n```\n\n\n## Development: Running Travis Worker locally\n\nThis section is for anyone wishing to contribute code to Worker. The code\nitself _should_ have godoc-compatible docs (which can be viewed on godoc.org:\n\u003chttps://godoc.org/github.com/travis-ci/worker\u003e), this is mainly a higher-level\noverview of the code.\n\n### Environment\n\nEnsure you've defined the necessary environment variables (see `.example.env`).\n\n### Pull Docker images\n\n```\ndocker pull travisci/ci-amethyst:packer-1504724461\ndocker tag travisci/ci-amethyst:packer-1504724461 travis:default\n```\n\n### Configuration\n\nFor configuration, there are some things like the job-board (`TRAVIS_WORKER_JOB_BOARD_URL`)\nand travis-build (`TRAVIS_WORKER_BUILD_API_URI`) URLs that need to be set. These\ncan be set to the staging values.\n\n```\nexport TRAVIS_WORKER_JOB_BOARD_URL='https://travis-worker:API_KEY@job-board-staging.travis-ci.com'\nexport TRAVIS_WORKER_BUILD_API_URI='https://x:API_KEY@build-staging.travis-ci.org/script'\n```\n\n`TRAVIS_WORKER_BUILD_API_URI` can be found in the env of the job board app, e.g.:\n`heroku config:get JOB_BOARD_BUILD_API_ORG_URL -a job-board-staging`.\n\n#### Images\n\nTODO\n\n#### Configuring the requested provider/backend\n\nEach provider requires its own configuration, which must be provided via\nenvironment variables namespaced by `TRAVIS_WORKER_{PROVIDER}_`.\n\n##### Docker\n\nThe backend should be configured to be Docker, e.g.:\n\n``` bash\nexport TRAVIS_WORKER_PROVIDER_NAME='docker'\nexport TRAVIS_WORKER_DOCKER_ENDPOINT=unix:///var/run/docker.sock        # or \"tcp://localhost:4243\"\nexport TRAVIS_WORKER_DOCKER_PRIVILEGED=\"false\"                          # optional\nexport TRAVIS_WORKER_DOCKER_CERT_PATH=\"/etc/secret-docker-cert-stuff\"   # optional\n```\n\n### Queue configuration\n\n#### File-based queue\n\nFor the queue configuration, there is a file-based queue implementation so you\ndon't have to mess around with RabbitMQ.\n\nYou can generate a payload via the `generate-job-payload.rb` script on travis-scheduler:\n\n`heroku run -a travis-scheduler-staging script/generate-job-payload.rb \u003cjob id\u003e \u003e payload.json`\n\nPlace the file in the `$TRAVIS_WORKER_QUEUE_NAME/10-created.d/` directory, where\nit will be picked up by the worker.\n\nSee `example-payload.json` for an example payload.\n\n#### AMQP-based queue\n\n```\nexport TRAVIS_WORKER_QUEUE_TYPE='amqp'\nexport TRAVIS_WORKER_AMQP_URI='amqp://guest:guest@localhost'\n```\n\nThe web interface is accessible at http://localhost:15672/\n\nTo verify your messages are being published, try:\n\n`rabbitmqadmin get queue=reporting.jobs.builds`\n\nNote: You will first need to install `rabbitmqadmin`. See http://localhost:15672/cli\n\nSee `script/publish-example-payload` for a script to enqueue `example-payload.json`.\n\n### Building and running\n\nRun `make build` after making any changes. `make` also executes the test suite.\n\n0. `make`\n0. `${GOPATH%%:*}/bin/travis-worker`\n\nor in Docker (FIXME):\n\n0. `docker build -t travis-worker .` # or `docker pull travisci/worker`\n0. `docker run --env-file ENV_FILE -ti travis-worker` # or `travisci/worker`\n\n### Testing\n\nRun `make test`. To run backend tests matching `Docker`, for example, run\n`go test -v ./backend -test.run Docker`.\n\n### Verifying and exporting configuration\n\nTo inspect the parsed configuration in a format that can be used as a base\nenvironment variable configuration, use the `--echo-config` flag, which will\nexit immediately after writing to stdout:\n\n``` bash\ntravis-worker --echo-config\n```\n\n\n## Stopping Travis Worker\n\nTravis Worker has two shutdown modes: Graceful and immediate. The graceful\nshutdown will tell the worker to not start any additional jobs but finish the\njobs it is currently running before it shuts down. The immediate shutdown will\nmake the worker stop the jobs it's working on, requeue them, and clean up any\nopen resources (shut down VMs, cleanly close connections, etc.)\n\nTo start a graceful shutdown, send an INT signal to the worker (for example\nusing `kill -INT`). To start an immediate shutdown, send a TERM signal to the\nworker (for example using `kill -TERM`).\n\n## Go dependency management\n\nTravis Worker is built via the standard `go` commands and dependencies managed\nby using Go Modules.\n\n## Release process\n\nSince we want to easily keep track of worker changes, we often associate them with a version number.\nTo find out the current version, check the [changelog](https://github.com/travis-ci/worker/blob/master/CHANGELOG.md) or run `travis-worker --version`.\nWe typically use [semantic versioning](https://semver.org/) to determine how to increase this number.\n\nOnce you've decided what the next version number should be, update the [changelog](https://github.com/travis-ci/worker/blob/master/CHANGELOG.md) making sure you include all relevant changes that happened since the previous version was tagged. You can see these by running `git diff vX.X.X...HEAD`, where `v.X.X.X` is the name of the previous version.\n\nOnce the changelog has been updated and merged to `master`, the merge commit needs to be signed and manually tagged with the version number. To do this, run:\n\n```\ngit tag --sign -a vX.X.X -m \"Worker version vX.X.X\"\ngit push origin vX.X.X\n```\n\nThe Travis build corresponding to this push should build and upload a worker image with the new tag to [Dockerhub](https://hub.docker.com/r/travisci/worker/tags/).\n\nThe next step is to create a new [Github release tag](https://github.com/travis-ci/worker/releases/new) with the appropriate information from the changelog.\n\n## License and Copyright Information\n\nSee [LICENSE file](./LICENSE).\n\n© 2018 Travis CI GmbH\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftravis-ci%2Fworker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftravis-ci%2Fworker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftravis-ci%2Fworker/lists"}