{"id":17675121,"url":"https://github.com/conorgil/buildkite-example-queues","last_synced_at":"2026-02-07T05:00:56.490Z","repository":{"id":141849309,"uuid":"167415825","full_name":"conorgil/buildkite-example-queues","owner":"conorgil","description":"Repo that explores buildkite queues via examples","archived":false,"fork":false,"pushed_at":"2019-01-25T03:44:39.000Z","size":1447,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-07-23T04:44:21.995Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/conorgil.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-01-24T18:26:29.000Z","updated_at":"2019-12-06T07:03:39.000Z","dependencies_parsed_at":null,"dependency_job_id":"a0650e2d-e169-4745-9fde-643bd82b35e4","html_url":"https://github.com/conorgil/buildkite-example-queues","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/conorgil/buildkite-example-queues","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/conorgil%2Fbuildkite-example-queues","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/conorgil%2Fbuildkite-example-queues/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/conorgil%2Fbuildkite-example-queues/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/conorgil%2Fbuildkite-example-queues/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/conorgil","download_url":"https://codeload.github.com/conorgil/buildkite-example-queues/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/conorgil%2Fbuildkite-example-queues/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29186742,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-07T03:35:06.566Z","status":"ssl_error","status_checked_at":"2026-02-07T03:34:57.604Z","response_time":63,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-24T07:10:03.169Z","updated_at":"2026-02-07T05:00:56.410Z","avatar_url":"https://github.com/conorgil.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# buildkite-queues\nThe Buildkite documentation is not perfectly clear on\nthe difference between normal agent `tags` and the special\n`queue` tag. This repo presents a series of examples to\nsee how they operate in practice.\n\n**[UPDATE]:** The main takeaway is that `queue` is a plain old\nregular agent tag, except that agents and jobs that do not \nexplicitly define a `queue` tag are automatically assigned\n`queue=default`.\n\n## Get started\nYou will need the Buildkite agent token from your buildkite\naccount to run these examples.\n\nUseful Buildkite documentation:\n* [Running Buildkite Agent with Docker](https://buildkite.com/docs/agent/v3/docker).\n* [Setting tags on agents](https://buildkite.com/docs/agent/v3/cli-start#setting-tags)\n* [Buildkite Agent Job Queues](https://buildkite.com/docs/agent/v3/queues)\n\n## Background\nWhen starting a buildkite agent, you can assign it zero or\nmore custom `tag`s. The keys and values of these tags can be\nanything you want, but it makes sense to define a structured\nformat for organizational purposes.\n\nThese agent tags are reported to Buildkite and appear in\nthe agent's metadata. Go to the [agents list](https://buildkite.com/organizations/oasislabs/agents)\nin your account and click on a random agent to see the list\nof tags it was assigned when it was started.\n\nThese agent tags can be used to specify certain jobs to run\non agents with matching tags.\n\nFor example, suppose you had a job to build a go binary.\nYou could assign an agent a tag of `go=true` and\nthen specify the job to require an agent with that tag:\n\n```\nsteps:\n  - command: \"script.sh\"\n    agents:\n      go: 'true'\n```\n\nIf an agent does not have the tag `go=true`, then it will\nnever run this job. This is a trivial example and you could\nobviously get fancier and specify the specific version of\ngo running on an agent, etc. The possibilities are endless.\n\n**NOTE:** The main point here is that if a job specifies an\nagent requirement, then an agent without the corresponding\ntag will never run that job.\n\nHOWEVER, of course there is a caveat. The `queue` tag is a special case.\n\nWhether you set it explicitly or not, every single build\nagent is assigned a `queue`. By default, the config file\nfor an agent has `tags=queue=default`, which explicitly\nsets the default queue. However, even if an agent is not\nexplicitly assigned a queue, it will implicitly be added\nto the default queue.\n\nAn agent must match **ALL** of the tags specified by a job\nin order to run that job. If the agent matches all of the tags,\nbut is in the wrong queue, then it won't run the job. If the agent\nis in the correct queue, but does not match all of the other\ntags specified by the job, then it won't run the job.\n\n## Run the examples\nTo see how tags and queues interact with each other,\nI created a series of examples in this repo. The pipeline\nis setup to several jobs, each of which has a different\nsetup of tags and queues (exploring most of the interesting\npermutations).\n\nTo avoid the possibility of committing the buildkite agent token\nto git and also avoid littering my shell history with commands\ncontaining the agent token, I put the agent token into the file \n`~/.buildkite/agent_token` and then use the following \ncommand to run the buildkite agent:\n\n```\ndocker \\\n  run \\\n  -it \\\n  -e BUILDKITE_AGENT_TOKEN=\"`cat ~/.buildkite/agent_token`\" \\\n  -v \"$PWD/config/buildkite-agent.cfg:/buildkite/buildkite-agent.cfg:ro\" \\\n  buildkite/agent\n```\n\nNotice that there are multiple buildkite configuration files in\nthis repo. You can run the buildkite agent with a different config\nfile and then observe which jobs are able to run on that agent.\n\nHere is a summary:\n\n![buildkite-agent-vs-jobs-summary-chart](https://github.com/conorgil/buildkite-example-queues/blob/master/buildkite-agent-vs-jobs-summary-chart.png)\n\nA screenshot of each scenario is included in the `results/` directory.\n\n## API\nNote that the Buildkite Rest API v3 returns tags in a field called\n`meta_data`. I believe legacy versions of the Buildkite API used the\nterm metadata and that was renamed to tags in v3.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconorgil%2Fbuildkite-example-queues","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fconorgil%2Fbuildkite-example-queues","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconorgil%2Fbuildkite-example-queues/lists"}