{"id":13903239,"url":"https://github.com/miekg/pgo","last_synced_at":"2025-07-19T01:04:20.128Z","repository":{"id":166423105,"uuid":"641832982","full_name":"miekg/pgo","owner":"miekg","description":"container gitops in a simple way","archived":false,"fork":false,"pushed_at":"2024-11-18T19:10:54.000Z","size":271,"stargazers_count":13,"open_issues_count":3,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-03T05:59:10.629Z","etag":null,"topics":["containers","docker-compose","gitlab"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/miekg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-05-17T09:01:43.000Z","updated_at":"2025-02-08T18:21:50.000Z","dependencies_parsed_at":null,"dependency_job_id":"1a18e7b5-4e3b-4012-be7f-7d0ddae17a09","html_url":"https://github.com/miekg/pgo","commit_stats":null,"previous_names":[],"tags_count":57,"template":false,"template_full_name":null,"purl":"pkg:github/miekg/pgo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miekg%2Fpgo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miekg%2Fpgo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miekg%2Fpgo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miekg%2Fpgo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/miekg","download_url":"https://codeload.github.com/miekg/pgo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miekg%2Fpgo/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265866271,"owners_count":23840938,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["containers","docker-compose","gitlab"],"created_at":"2024-08-06T22:01:54.148Z","updated_at":"2025-07-19T01:04:20.088Z","avatar_url":"https://github.com/miekg.png","language":"Go","funding_links":[],"categories":["docker-compose"],"sub_categories":[],"readme":"# PGO\n\nPodman Gitops. This is a subsequent development (or successor?) of\n\u003chttps://github.com/miekg/gitopper\u003e. Where \"gitopper\" integrates with your OS, i.e. use Debian\npackages, \"pgo\" uses a `compose.yaml` as its basis. It runs the compose via `docker compose` (or\ndocker-compose) (podman was dropped, see below, see https://docs.docker.com/engine/install/debian/\nfor docker's installation). It allows for remote interaction via an SSH interface, which `pgoctl`\nmakes easy to use. For this SSH interface no local users need to exist on the target system. The\ncompose is (usually) executed under a particular user name, those users do need an account on the\ntarget system, they do not need shell access, because everything runs through SSH of pgod(8).\n\nThe main idea here is that developers can push stuff easier to production and that you can have some\nof the goodies from Kubernetes, but not the bad stuff like the networking - the big trade-off being\nthat machines are still somewhat special.\n\nCurrent the following compose file variants are supported: \"compose.yaml\", \"compose.yml\",\n\"docker-compose.yml\" and \"docker-compose.yaml\". If you need more flexibility you can point to a\nspecific compose file, with the `compose` key in the config.\n\nEach compose file runs under it's own user-account. That account can then access storage, or\ndatabases it has access to - provisioning that stuff is out-of-scope - assuming your infra can deal\nwith all that. The compose file is parsed and the following settings are *disallowed*:\n\n- privileged=true\n- network_mode=host\n- ipc=host\n- volumes can only reference an absolute path under datadir/\u003cservicename\u003e (see --datadir of\n  pgod(8)), and volumes can only be of type 'bind' or 'volume'\n- each service must have a network that is _not_ in pgo.toml's `networks` for this service, this\n  only enforced if `networks` is set\n\nServers running pgod(8) as still special in some regard, as a developers needs to know which server runs\ntheir compose file. Moving services to a different machine is as easy as starting the compose there,\nbut you need to make sure your infra also updates external records (DNS for example).\n\nThe interface into pgod(8) is via custom SSH implementation that is separate from the machine's SSH.\nThe owner of the Git repo can publish public keys in a `ssh` direcotry and their repo and give\npgoctl(1)-access tp anyone with access the correspondong private key.\n\nNote that pgod runs (as root) under systemd.\n\nA typical config file looks like this:\n\n``` toml\n[[services]]\nname = \"pgo\"\nuser = \"miek\"\nrepository = \"https://github.com/miekg/pgo\"\nregistries = [ \"user:authtoken@registry\" ] # or just authtoken@registry\ncompose = \"compose.yaml\"\nbranch = \"main\"\nenv = [ \"MYENV=bla\", \"OTHERENV=bliep\"]\nurls = { \"pgo.science.ru.nl\" = \"pgo:5007\" }\nnetworks = [ \"reverseproxy\" ]\n# import = \"Caddyfile-import\"\n# reload = \"localhost:caddy//exec caddy reload --config /etc/caddy/Caddyfile --adapter caddyfile\"\n# mount =  \"nfs://server.example.org/share\"\n```\n\nThis file is used by `pgod` and should be updated for each project you want to onboard. To go over\nthis file:\n\n- `name`: this is the name of the service, used to uniquely identify the service across machines.\n- `user`: which user to use to run the docker compose under.\n- `repository` and `branch`: where to find the git repo belonging to this service.\n- `registries`: optional authentication for pulling the docker images from the registry. In\n  \"user:token\" format, is user is omitted, `user` is used. This is a list because there can be more\n  than one private registry. This should match any registries used in the compose file.\n- `compose`: alternate compose file to use.\n- `urls`: what DNS names need to be assigned to this server and to what network and port should they forward.\n- `env`: specify extra environment variables in \"VAR=VALUE\" notation (i.e. secrets).\n- `networks`: which external network can this service use. Empty means all.\n- `import`: create a Caddyfile snippet with reverse proxy statements for all URLs in all services\n  and write this in the directory where the repository is checked out.\n- `reload`: a exec command in pgoctl(1) syntax to reload caddy when a new import file is written.\n- `mount`: specific a NFS volume that will be mounted in `\u003cdatadir\u003e/\u003cname\u003e`, see pgod(8). This NFS mount gets\n  mounted with default options: \"rw,nosuid,hard\".\n\nFor non-root accounts, docker compose will be run with the normal supplementary groups to which the\n*local* docker group has been added. This allows those user to transparently access the docker\nsocket, without going through some addgroup(8) hassle.\n\n### Compose File Extensions\n\nOn every change to the compose file, pgod(8) will down and up your services. If you do not want this\nadded the following your compose file (as a top-level declaration). The default for reload is\n`true`.\n\n~~~ yaml\nx-pgo:\n  reload: false\n~~~\n\n## Requisites\n\nTo use \"pgo\" your project should have:\n\n- A public SSH key (or keys) stored in a `ssh/` directory in your git repo. This keys can **not** have a\n  passphrase protecting them. If there are no keys, or no ssh directory pgoctl(1) will not work.\n- A `compose.yaml` (or any of the variants) in the top-level of your git repo.\n\n## Quick Start\n\nAssuming a working Go compiler you can issue a `make` to compile the binaries. Then:\n\nStart `pgod`: `sudo /cmd/pgod/pgod -c pgo.toml -d /tmp/pgo --debug`. That will output some debug\ndata.\n\nIn other words: it clones the repo, pulls, and starts the containers. It then *tracks*\nupstream and whenever `compose.yaml` changes it will do a `down` and `up`. To force changes\nin that file you can use a `x-gpo-version` in the yaml and change that whenever you want to update\n\"pgo\"\n\nNow with `pgoctl` you can access and control this environment (well not you, because you don't have\nthe private key belonging to the public key that sits in the `ssh/` directory). `pgoctl` want to\nsee `\u003cmachine\u003e:\u003cname\u003e//\u003coperation\u003e` string, i.e. `localhost:pgo//ps` which does a `docker compose\nps` for our stuff:\n\n~~~\n# ask for the status of pgo - denied because the correct key is not found in the repo\n% ./cmd/pgoctl/pgoctl -i ~/id_pgo2 localhost:pgo//ps\nUnauthorized: Key for user \"miek\" does not match any for name pgo\n2023/05/17 20:21:08 [ERROR] Process exited with status 401\n~~~\n\nOnce our committed keys get pulled:\n~~~\n% ./cmd/pgoctl/pgoctl -i ~/id_pgo2 localhost:pgo//ps\nNAME                IMAGE               COMMAND                  SERVICE             CREATED             STATUS              PORTS\npgo-frontend-1      docker.io/busybox   \"/bin/busybox httpd …\"   frontend            9 minutes ago       Up 9 minutes        0.0.0.0:32771-\u003e8080/tcp\n~~~\n\nCurrently implemented are: `up`, `down`, `pull`, `ps`, `logs` and `ping` to see if the\nauthentication works (replies with a \"pong!\" if everything works).\n\n## Integrating with GitLab Environments\n\nIf you want to use pgo with GitLab you needs to setup\n[environments](https://docs.gitlab.com/ee/ci/environments/) that allow you to deploy to\n\"production\", here is an example `.gitlab-ci.yml` that does this:\n\n~~~ yaml\nimage: \"registry.science.ru.nl/cncz/sys/image/cncz-debian-go:latest\"\n\nstages:\n  - deploy\n\ndeploy_production:\n  resource_group: production\n  stage: deploy\n  environment:\n    name: production\n    url: https://example.com\n    on_stop: stop_production\n  script:\n    - pgoctl mymachine:project//pull  # looks for PGOCTL_ID env var, with privkey contents\n    - pgoctl mymachine:project//up\n  when: manual\n\nstop_production:\n  resource_group: production\n  stage: deploy\n  script:\n    - pgoctl mymachine:project//down\n  environment:\n    name: production\n    action: stop\n  when: manual\n~~~\n\nWith 'manual' you can still control when this actually happens.\n\nIf you want to clone a repository that is private, you can create an access token with\n'read_repository' and the \"developer\" role. This can be then used as:\n\n~~~ toml\nrepository = \"https://oauth2:\u003ctoken\u003e@gitlab.science.ru.nl/...\"\n~~~\n\n## Networking and Reverse Proxy\n\nIf services need a network, you'll need to set this up by yourself with Caddy, pgod(8) has support to\nwrite a Caddyfile snippet that routes all URLs to the composer's backends. This does mean the\ncaddy's docker-compose must be setup in such a way that it will read that file *and* configures a\n\"well-known\" network, where other composers can hook into. A setup you can use is having `caddy` as\nthe name for the service *and* the network. This is defined in the\n\u003chttps://github.com/miekg/pgo-caddy\u003e project. Other service need to reference this as an external netwerk.\n\nThe services that are exporting into the caddy snippet only need an \"url\" in their config. So the\n`pgo-caddy` config is a normal pgo service and has this compose config:\n\n~~~ yaml\nservices:\n  caddy:\n    image: docker.io/caddy:2.7-alpine\n    restart: unless-stopped\n    ports:\n      - \"80:80\"\n      - \"443:443\"\n    volumes:\n      - ./caddy/:/etc/caddy/\n    networks:\n      - reverseproxy\n\nnetworks:\n    reverseproxy:\n      name: reverseproxy\n~~~\n\nWhere the network created is called `reverseproxy` and the `Caddyfile-import` is written, by pgod(8), into\nthe directory that gets mounted as a volume inside the caddy container. The `pgo-caddy` git\nrepository has an .gitignore for `caddy/Caddyfile-import`.\n\nOther users of this pgo instance only need to know the network is called `caddy` and commit their\npgo.toml config to make things work, i.e. in those `compose.yaml` they need:\n\n~~~ yaml\nnetworks:\n  reverseproxy:\n    external: true\n    name: reverseproxy\n  myownnetwork:\n~~~\n\nin each service that need a reverse proxy, and then _also_ a `urls` section in `pgo.toml` that ties\neverything together: `urls = { \"example.org\": frontend:8080\" }`. The lone `myownnetwork` is needed\nbecause we want every \"compose\" to be in its own network. This might be enforced in the future.\n\n## pgod\n\nSee the [manual page](./cmd/pgod/pgod.8.md) in [cmd/pgod](./cmd/pgod/).\n\n## pgoctl\n\nSee the [manual page](./cmd/pgoctl/pgoctl.1.md) in [cmd/pgoctl](./cmd/pgoctl). Also details the\n`PGOCTL_ID` mentioned above.\n\n# Podman and Podman-Compose\n\nInitialy pgo was using podman(-compose) to run the images, but this proved to be a challenge.\npodman-compose is a seperate project and has it's own ideas on how to parse a compose.yml file (not\nonly his fault, the format is terrible). But using external network just didn't work, regardless\nwhat syntax was used. Also podman kept complaining about CNI version clashes which were\nundebuggable, so as much as I want to like podman, this is now using docker compose.\n\nAlso in podman 4 the networking moved away from CNI to a new thing written in Rust - which is\ncompletely fine, but does raise the possibility that I can revist networking relatively soon again\nto fix it for podman4.\n\nAlso podman-compose has not seen much releases, so the apt-get install story becomes weaker there as\nwell. Initial experiments with docker made stuff work out of the box.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiekg%2Fpgo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmiekg%2Fpgo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiekg%2Fpgo/lists"}