{"id":15580446,"url":"https://github.com/sbidoul/runboat","last_synced_at":"2025-04-04T20:09:20.001Z","repository":{"id":37941498,"uuid":"417775428","full_name":"sbidoul/runboat","owner":"sbidoul","description":"A simple runbot lookalike on kubernetes. Main goal is replacing the OCA runbot.","archived":false,"fork":false,"pushed_at":"2025-03-24T17:55:48.000Z","size":506,"stargazers_count":126,"open_issues_count":21,"forks_count":56,"subscribers_count":18,"default_branch":"main","last_synced_at":"2025-03-28T19:06:55.434Z","etag":null,"topics":["hacktoberfest","kubernetes","odoo","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sbidoul.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-16T09:09:52.000Z","updated_at":"2025-03-21T08:57:36.000Z","dependencies_parsed_at":"2023-12-25T20:27:08.509Z","dependency_job_id":"4567ca72-8144-4a73-8209-fadf4c08501d","html_url":"https://github.com/sbidoul/runboat","commit_stats":{"total_commits":276,"total_committers":4,"mean_commits":69.0,"dds":0.0905797101449275,"last_synced_commit":"fc6582a34abf435a7207c04aa2439c3dff114040"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbidoul%2Frunboat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbidoul%2Frunboat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbidoul%2Frunboat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbidoul%2Frunboat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sbidoul","download_url":"https://codeload.github.com/sbidoul/runboat/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247242678,"owners_count":20907134,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacktoberfest","kubernetes","odoo","python"],"created_at":"2024-10-02T19:26:07.246Z","updated_at":"2025-04-04T20:09:19.977Z","avatar_url":"https://github.com/sbidoul.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# runboat ☸️\n\nA simple Odoo runbot lookalike on kubernetes. Main goal is replacing the OCA runbot.\n\n[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/sbidoul/runboat/main.svg)](https://results.pre-commit.ci/latest/github/sbidoul/runboat/main)\n\n## Principle of operation\n\nThis program is a Kubernetes operator that manages Odoo instances with pre-installed\naddons. The addons come from commits on branches and pull requests in GitHub\nrepositories. A deployment of a given commit of a given branch or pull request of a\ngiven repository is known as a build.\n\nRunboat has the following main components:\n\n- An in-memory database of deployed builds, with their current status.\n- A REST API to list builds and trigger new deployments as well as start, stop, redeploy\n  or undeploy builds.\n- A GitHub webhook to automatically trigger new builds on pushes to branches and pull\n  requests of supported repositories and branches (configured via regular expressions).\n- A controller that performs the following tasks:\n\n  - monitor deployments in a kubernetes namespaces to maintain the in-memory database;\n  - on new deployments, trigger an initialization job to check out the GitHub repo,\n    install dependencies, create the corresponding postgres database and install the\n    addons in it;\n  - initialization jobs are started concurrently up to a configured limit;\n  - when the initialization job succeeds, preserve the database, filestore, and virtual\n    environment, so everything is ready for an almost instantaneous startup;\n  - when the initialization job fails, flag the deployment as failed;\n  - when there are too many deployments started, stop the oldest started;\n  - when there are too many deployments, delete the oldest created;\n  - when a deployment is deleted, run a cleanup job to drop the database and delete\n    all kubernetes resources associated with the deployment.\n\nThis approach allows the deployment of a very large number of builds which consume no\nmemory nor CPU until they are started. The number of started deployment can also be\nhigh, by reserving limited CPU and memory resources for each, taking advantage of the\nfact that they are typically used infrequently. The number of concurrent initialization\njobs is limited strongly, and they are queued, as these are typically the more\nresource-intensive part of the lifecycle of builds.\n\nAll state is stored in kubernetes resources (labels and annotations on deployments). The\ncontroller can be stopped and restarted without losing state.\n\nIn practice the runboat controller does not know that it is deploying Odoo. All the\nknowledge about *what* is deployed is in the\n[src/runboat/kubefiles](./src/runboat/kubefiles) directory. The kubefiles must implement\na specific contract to be managed by the runboat controller. This contract is described\nin the [Kubernetes resources](#kubernetes-resources) section below.\n\n## Requirements\n\nFor running the builds:\n\n- A namespace in a kubernetes cluster.\n- A wildcard DNS domain that points to the kubernetes ingress.\n- A postgres database, accessible from within the cluster namespace with a user with\n  permissions to create databases.\n\nFor running the controller (runboat itself):\n\n- Python 3.13\n- sqlite3 \u003e= 3.25\n- `kubectl`\n- A `KUBECONFIG` or an in-cluster service account that provides access to the namespace\n  where the builds are deployed, with permissions to create and delete Service, Job,\n  Deployment, PersistentVolumeClaim, Ingress, Secret and ConfigMap resources as well as\n  read and watch Deployments and Jobs.\n- Some sort of reverse proxy to expose the REST API.\n\nThe controller can be run outside the kubernetes cluster or deployed inside it, or even\nin a different cluster.\n\n## Deployment quickstart\n\nA typical deployment looks like this.\n\n![Deployment diagram](./docs/deployment/deployment.png)\n\nThe wiki has an example [docker-compose\nconfiguration](https://github.com/sbidoul/runboat/wiki/example-docker-compose) to get\nyou started with running the runboat controller.\n\nIn that docker compose, you must provide configuration parameters for the postgres\ndatabase and the kubernetes cluster:\n\n- For postgres: host, port, user, password (the postgres user must exist and have\n  permissions to create databases).\n- For kubernetes:\n  - The name of the kubernetes namespace where the builds are deployed.\n  - A KUBECONFIG for a kubernetes account that can create and delete resources in that\n    namespace.\n  - The name of a kubernetes Storage Class that can allocate Persistent Volumes that are\n    reclaimed on delete.\n  - The DNS domain corresponding to a wildcard DNS entry pointing to the kubernetes\n    ingress (so if your wildcard DNS record is `*.runboat-builds.mydomain.com`, you\n    provide `runboat-builds.mydomain.com`).\n\nNote that you can deploy Runboat itself as well Postgres outside or inside the\nkubernetes cluster, or even a different one, depending on your taste.\n\n## Kubernetes resources\n\nAll resources to be deployed in kubernetes for a build are in\n[src/runboat/kubefiles](./src/runboat/kubefiles). They are gathered together from a\n`kustomization.yaml` jinja template that leads to three possible resource groups\ndepending on a `mode` variable in the jinja rendering context:\n\n- `deployment` creates a kubernetes deployment with its associated resources (pvc,\n  service, ingress, ...);\n- `initialization` creates a job that creates the database;\n- `cleanup` creates a job that drops the database;\n\nBesides the three modes, the controller has limited knowledge of what the kubefiles\nactually deploy. It expects the following to hold true:\n\n- the `runboat/build` label is set on all resources, with the unique build name as\n  value;\n- a deployment starts with 0 replicas and is created with a\n  `runboat/init-status=todo` label, as well as a `runboat/cleanup` finalizer;\n- the intialization job and pods have a `runboat/job-kind=initialize` label;\n- the cleanup job and pods have a `runboat/job-kind=cleanup` label.\n- the following annotations are set on deployments:\n\n  - `runboat/repo`: the repository in owner/repo format;\n  - `runboat/target-branch`: the branch or pull request target branch;\n  - `runboat/pr`: the pull request number if this build is for a pull request;\n  - `runboat/git-commit`: the commit sha.\n\n- the home page of a running build is exposed at `http://{build_slug}.{build_domain}`.\n\nDuring the lifecycle of a build, the controller does the following on the deployed\nresources:\n\n- it sets the `runboat/init-status` annotation (`todo`, `started`, `succeeded`,\n  `failed`) on deployments to track the outcome of the initialization jobs;\n- it sets the deployment's `specs.replica` to 1 or 0 to start or stop it;\n- it deletes the deployment when an undeploy is requested (the actual delete occurs\n  later due to the finalizer);\n- it removes the deployment finalizers and deletes resources matching the\n  `runboat/build` label after the cleanup job succeeded.\n\n### Alternative Kubefiles\n\nBy default, Runboat relies on its bundled Kubefiles:\n[src/runboat/kubefiles](./src/runboat/kubefiles)\n\nBut you can define:\n\n- a different default path through environment variable\n  `RUNBOAT_BUILD_DEFAULT_KUBEFILES_PATH`;\n- a different path for a specific repo,\n  by defining the `kubefiles_path` key in `RUNBOAT_REPOS`, e.g.:\n\n```\nRUNBOAT_REPOS=[{\"repo\": \"^oca/.*\", \"branch\": \"^15.0$\", \"builds\": [{\"image\": \"ghcr.io/oca/oca-ci/py3.8-odoo15.0:latest\", \"kubefiles_path\": \"/tmp\"}]}]\n```\n\n## Developing\n\n- setup environment variables (start from `.env.sample`, the meaning of the environment\n  variables is documented in [settings.py](./src/runboat/settings.py))\n- create a virtualenv, make sure to have pip\u003e=21.3.1 and `pip install -c\n  requirements.txt -e .[test]`\n- run with `uvicorn runboat.app:app --log-config=log-config.yaml`\n- api documentation is at `http://localhost:8000/docs`\n- run tests with `pytest` (environment variables used in tests are declared in\n  `.env.test`)\n\n## Running in production\n\n`gunicorn -w 1 -k runboat.uvicorn.RunboatUvicornWorker runboat.app:app`.\n\nOne and only one worker process is allowed at the moment (although nothing really bad\nshould happen if there is more).\n\nGunicorn also necessary so SIGINT/SIGTERM shutdowns after a few seconds. Since we use\n`run_in_executor`, SIGINT/SIGTERM handling does not work very well, and gunicorn makes\nit more robust. See https://bugs.python.org/issue29309. This should resolve itself when\nwe switch to using an async kubernetes client (tracked in\n[#6](https://github.com/sbidoul/runboat/issues/6)).\n\n## Configuration\n\nSee environment variables examples in [Dockerfile](./Dockerfile),\n[.env.sample](./.env.sample) and their documentation in\n[settings.py](./src/runboat/settings.py).\n\n## Credits\n\nAuthored by Stéphane Bidoul (@sbidoul) and\n[contributors](https://github.com/sbidoul/runboat/graphs/contributors) with support of\n[ACSONE](https://acsone.eu).\n\nContributions welcome.\n\nDo not hesitate to reach out for help on how to get started.\n\n## License\n\n`runboat` is distributed under the terms of the\n[MIT](https://spdx.org/licenses/MIT.html) license.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsbidoul%2Frunboat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsbidoul%2Frunboat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsbidoul%2Frunboat/lists"}