{"id":20415438,"url":"https://github.com/riot-os/murdock-worker","last_synced_at":"2026-06-05T08:31:14.487Z","repository":{"id":72503790,"uuid":"424925213","full_name":"RIOT-OS/murdock-worker","owner":"RIOT-OS","description":"RIOT Murdock worker Docker Compose file","archived":false,"fork":false,"pushed_at":"2023-03-01T13:32:09.000Z","size":34,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":23,"default_branch":"main","last_synced_at":"2025-01-15T13:44:08.835Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RIOT-OS.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-05T11:31:06.000Z","updated_at":"2023-07-11T22:47:50.000Z","dependencies_parsed_at":"2023-08-22T05:02:40.601Z","dependency_job_id":null,"html_url":"https://github.com/RIOT-OS/murdock-worker","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RIOT-OS%2Fmurdock-worker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RIOT-OS%2Fmurdock-worker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RIOT-OS%2Fmurdock-worker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RIOT-OS%2Fmurdock-worker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RIOT-OS","download_url":"https://codeload.github.com/RIOT-OS/murdock-worker/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241960797,"owners_count":20049340,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T06:16:07.892Z","updated_at":"2025-03-05T03:31:20.214Z","avatar_url":"https://github.com/RIOT-OS.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"## About\n\nThis repo contains a docker-compose.yml that can be used to run a Murdock worker\nfor RIOT.\n\nThe stack will include the following containers:\n\n- one or more container(s) using `riot/murdock-worker` that runs the dwq job runner\n- one ssh_bridge that connects via ssh to the murdock control node and provides\n  access to its disque and redis instances\n- one redis instance acting as ccache storage backend\n- a watchtower instance keeping all containers up-to-date.\n\n## Murdock Worker requirements\n\n- _at least_ four fast cores\n- 2GB RAM per worker + 8GB RAM for ccache\n\n## Prerequisites\n\n- docker-compose\n- git\n- murdock worker ssh key. ping kaspar on Matrix to get it!\n\n## Installation\n\n- Clone this repository to a location of your choice.\n  `/srv/murdock-worker` is what I use.\n- Copy `.env.example` to `.env`\n- Edit `.env`. Change _at least_ the hostname.\n- copy murdock worker ssh key to `ssh/`\n\nBy default, this will connect to \"ci-staging\", which is great for testing.\nOnce everything is working, change .env to connect to \"ci-prod\".\n\n## Starting\n\n`docker-compose up -d --scale worker=N`, with `N` being the number of concurrent\njobs. Each worker will need 2GB RAM (in addition to the 8GB for the shared tmpfs\nfor ccache).\nStart with half the number of *physical* cores.\n\n### Inside an LXC container\n\nRunning docker inside LXC comtainers works very much out of the box. E.g. on Proxmox additionally\nthe fuse and the nesting capabilities need to be enabled and `fuse-overlayfs` needs to be installed.\nThe reason is that native `overlayfs` cannot be used for rootless operation (or when the user\nlaunching the docker containers is only root within an LXC container namespace). Without\n`fuse-overlayfs` the VFS storage driver is used instead, which performs a deep copy of all lower\nlayers into the upper layers. With a large number of layers, this increases storage consumption\ntoo much to be practical.\n\n**Note**: Expect that `fuse-overlayfs` will throttle the performance of your worker quite a bit.\n\n**Also note**: The metrics reported by the vector service will include RAM and CPUs available to the\nbare host (e.g. the Proxmox running the LXC container which runs the docker containers). To fix\nthat, filtered versions of various `/proc/` files can be provided by lxcfs that can be bind-mounted\ninto the docker container. To do so automatically, run\n`docker-compose -f docker-compose.yml -f docker-compose.lxc.yml up -d --scale worker=N` instead of\njust `docker-compose up -d --scale worker=N`.\n\n## Configuration\n\nIf the worker is dedicated to being a Murdock worker, one worker per physical\ncore each running 4 jobs ensures the CPUs keep busy, if enough RAM is available.\nIf RAM is an issue, go down on workers to one per two cores, possible increasing\n`MURDOCK_JOBS` (to e.g., `8`).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Friot-os%2Fmurdock-worker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Friot-os%2Fmurdock-worker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Friot-os%2Fmurdock-worker/lists"}