{"id":19612085,"url":"https://github.com/fnndsc/pman","last_synced_at":"2025-05-07T18:09:21.646Z","repository":{"id":38707220,"uuid":"61242281","full_name":"FNNDSC/pman","owner":"FNNDSC","description":"A process management system written in python","archived":false,"fork":false,"pushed_at":"2024-05-07T19:27:54.000Z","size":1107,"stargazers_count":22,"open_issues_count":27,"forks_count":33,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-05-07T18:08:40.538Z","etag":null,"topics":["docker","flask","flaskrestful","kubernetes","openshift","pipelines","pyhton3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FNNDSC.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-06-15T21:30:56.000Z","updated_at":"2025-03-18T21:49:02.000Z","dependencies_parsed_at":"2024-11-11T10:46:05.374Z","dependency_job_id":"8f1effae-da0e-4570-9f1e-e0431e70dded","html_url":"https://github.com/FNNDSC/pman","commit_stats":null,"previous_names":[],"tags_count":99,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FNNDSC%2Fpman","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FNNDSC%2Fpman/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FNNDSC%2Fpman/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FNNDSC%2Fpman/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FNNDSC","download_url":"https://codeload.github.com/FNNDSC/pman/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252931555,"owners_count":21827112,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","flask","flaskrestful","kubernetes","openshift","pipelines","pyhton3"],"created_at":"2024-11-11T10:45:48.983Z","updated_at":"2025-05-07T18:09:21.584Z","avatar_url":"https://github.com/FNNDSC.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ![ChRIS logo](https://github.com/FNNDSC/ChRIS_ultron_backEnd/blob/master/docs/assets/logo_chris.png) pman\n\n[![Version](https://img.shields.io/docker/v/fnndsc/pman?sort=semver)](https://hub.docker.com/r/fnndsc/pman)\n[![MIT License](https://img.shields.io/github/license/fnndsc/pman)](LICENSE)\n[![ci](https://github.com/FNNDSC/pman/actions/workflows/ci.yml/badge.svg)](https://github.com/FNNDSC/pman/actions/workflows/ci.yml)\n\n_pman_, which once stood for **p**rocess **man**ager,\nis a [Flask](https://flask-restful.readthedocs.io/) application\nproviding an API for creating jobs with various schedulers e.g.\nKubernetes, Podman, Docker Swarm, and SLURM.\nIt basically translates its own JSON interface to requests for\nthe various supported backends.\n\n_pman_ is tightly-coupled to\n[_pfcon_](https://github.com/FNNDSC/pfcon). _pman_ and _pfcon_\nare typically deployed as a pair, providing the _pfcon_ service.\n\n## Running `pman`\n\nThe easiest way to see it in action is to run\n[_miniChRIS-docker_](https://github.com/FNNDSC/miniChRIS-docker).\nThe instructions that follow are for _pman_ hackers and developers.\n\n## Development\n\nThis section describes how to set up a local instance of *pman* for development.\n\n### Using Docker Compose\n\nThese instructions run _pman_ inside a container using Docker and Docker Swarm for scheduling jobs.\nHot-reloading of changes to the code is enabled.\n\n```shell\ndocker swarm init --advertise-addr 127.0.0.1\ndocker compose up -d\n```\n\n### Using Docker Swarm\n\nTo run a full test using `docker stack deploy`,\nrun the test harness `test_swarm.sh`.\n\n```shell\n./test_swarm.sh\n```\n\n### Using Podman for Development\n\n_pman_ must be able to schedule containers via Podman by communicating to the Podman socket.\n\n```shell\nsystemctl --user start podman.service\nexport DOCKER_HOST=\"$(podman info --format '{{ .Host.RemoteSocket.Path }}')\"\n```\n\n#### Install _pman_ using Python\n\n```shell\npython -m venv venv\nsource venv/bin/activate\npip install -r requirements/local.txt\npip install -e .\n```\n\n#### Run _pman_ using Python in Development Mode\n\n```shell\npython -m pman\n```\n\n### Using Kubernetes via Kind\n\nhttps://github.com/FNNDSC/pman/wiki/Development-Environment:-Kubernetes\n\n## Configuration\n\n_pman_ is configured by environment variables.\nRefer to the source code in [pman/config.py](pman/config.py)\nfor exactly how it works.\n\n### How Storage Works\n\n_pman_ relies on _pfcon_ to manage data in a directory known as \"storeBase.\"\nThe \"storeBase\" is a storage space visible to every node in your cluster.\n\nFor single-machine deployments using Docker and Podman, the best solution\nis to use a local volume mounted by _pfcon_ at `/var/local/storeBase`.\n_pman_ should be configured with `STORAGE_TYPE=docker_local_volume` `VOLUME_NAME=...`.\n\nOn Kubernetes, a single PersistentVolumeClaim should be used. It is mounted\nby _pfcon_ at `/var/local/storeBase`.\n_pman_ should be configured with `STORAGE_TYPE=kubernetes_pvc` `VOLUME_NAME=...`.\n\nSLURM has no concept of volumes, though SLURM clusters typically use a NFS share\nmounted to the same path on every node. _pman_ should be configured with\n`STORAGE_TYPE=host` `STOREBASE=...`, specify the share mount point as `STOREBASE`.\n\n#### `swarm` v.s. `docker`\n\nOriginally, _pman_ interfaced with the Docker Swarm API for the sake of supporting multi-node clusters.\nHowever, more often than not, _pman_ is run on a single-machine. Such is the case for developer\nenvironments, \"host\" compute resources for our single-machine production deployments of CUBE,\nand production deployments of _CUBE_ on our Power9 supercomputers. Swarm mode is mostly an annoyance\nand its multi-node ability is poorly tested. Furthermore, multi-node functionality is\nbetter provided by `CONTAINER_ENV=kubernetes`.\n\nIn _pman_ v4.1, `CONTAINER_ENV=docker` was introduced as a new feature and the default configuration.\nIn this mode, _pman_ uses the Docker Engine API instead of the Swarm API, which is much more convenient\nfor single-machine use cases.\n\n### Podman Support\n\n**`CONTAINER_ENV=docker` is compatible with Podman.**\n\nPodman version 3 or 4 are known to work.\n\n#### Rootless Podman\n\nConfigure the user to be able to set resource limits.\n\nhttps://github.com/containers/podman/blob/main/troubleshooting.md#symptom-23\n\n### Environment Variables\n\n| Environment Variable     | Description                                                                                                                     |\n|--------------------------|---------------------------------------------------------------------------------------------------------------------------------|\n| `SECRET_KEY`             | [Flask secret key][flask docs]                                                                                                  |\n| `CONTAINER_ENV`          | one of: \"swarm\", \"kubernetes\", \"cromwell\", \"docker\"                                                                             |\n| `STORAGE_TYPE`           | one of: \"host\", \"docker_local_volume\", \"kubernetes_pvc\"                                                                         |\n| `STOREBASE`              | where job data is stored, valid when `STORAGE_TYPE=host`, conflicts with `VOLUME_NAME`                                          |\n| `VOLUME_NAME`            | name of data volume, valid when `STORAGE_TYPE=docker_local_volume` or `STORAGE_TYPE=kubernetes_pvc`                             |\n| `PFCON_SELECTOR`         | label on the pfcon container, may be specified for pman to self-discover `VOLUME_NAME` (default: `org.chrisproject.role=pfcon`) |\n| `CONTAINER_USER`         | Set job container user in the form `UID:GID`, may be a range for random values                                                  | \n| `ENABLE_HOME_WORKAROUND` | If set to \"yes\" then set job environment variable `HOME=/tmp`                                                                   |\n| `SHM_SIZE`               | Size of `/dev/shm` in mebibytes. (Supported only in Docker, Podman, and Kubernetes.)                                            |\n| `JOB_LABELS`             | CSV list of key=value pairs, labels to apply to container jobs                                                                  |\n| `JOB_LOGS_TAIL`          | (int) maximum size of job logs                                                                                                  |\n| `IGNORE_LIMITS`          | If set to \"yes\" then do not set resource limits on container jobs (for making things work without effort)                       |\n| `REMOVE_JOBS`            | If set to \"no\" then pman will not delete jobs (for debugging)                                                                   |\n\n[flask docs]: https://flask.palletsprojects.com/en/2.1.x/config/#SECRET_KEY\n\n### `STOREAGE_TYPE=host`\n\nWhen `STORAGE_TYPE=host`, then specify `STOREBASE` as a mount point path on the host(s).\n\n### `STOREAGE_TYPE=docker_local_volume`\n\nFor single-machine instances, use a Docker/Podman local volume as the \"storeBase.\"\nThe volume should exist prior to the start of _pman_. It can be identified one of\ntwo ways:\n\n- Manually, by passing the volume name to the variable `VOLUME_NAME`\n- Automatically: _pman_ inspects a container with the label `org.chrisproject.role=pfcon`\n  and selects the mountpoint of the bind to `/var/local/storeBase`\n\n#### `STORAGE_TYPE=kubernetes_pvc`\n\nWhen `STORAGE_TYPE=kubernetes_pvc`, then `VOLUME_NAME` must be the name of a\nPersistentVolumeClaim configured as ReadWriteMany.\n\nIn cases where the volume is only writable to a specific UNIX user,\nsuch as a NFS-backed volume, `CONTAINER_USER` can be used as a workaround.\n\n### Kubernetes-Specific Options\n\nApplicable when `CONTAINER_ENV=kubernetes`\n\n| Environment Variable      | Description                                     |\n|---------------------------|-------------------------------------------------|\n| `JOB_NAMESPACE`           | Kubernetes namespace for created jobs           |\n| `NODE_SELECTOR`           | Pod `nodeSelector`                              |\n\n### SLURM-Specific Options\n\nApplicable when `CONTAINER_ENV=cromwell`\n\n| Environment Variable | Description                                          |\n|----------------------|------------------------------------------------------|\n| `CROMWELL_URL`       | Cromwell URL                                         |\n| `TIMELIMIT_MINUTES`  | SLURM job time limit                                 |\n\nFor how it works, see https://github.com/FNNDSC/pman/wiki/Cromwell\n\n### Container User Security\n\nSetting an arbitrary container user, e.g. with `CONTAINER_USER=123456:123456`,\nincreases security but will cause (unsafely written) _ChRIS_ plugins to fail.\nIn some cases, `ENABLE_HOME_WORKAROUND=yes` can get the plugin to work\nwithout having to change its code.\n\nIt is possible to use a random container user with `CONTAINER_USER=1000000000-2147483647:1000000000-2147483647`\nhowever considering that *pfcon*'s UID never changes, this will cause everything to break.\n\n## Missing Features\n\n_pman_'s configuration has gotten messy over the years because it attempts to provide an interface\nacross vastly different systems. Some mixing-and-matching of options are unsupported:\n\n- `IGNORE_LIMITS=yes` only works with `CONTAINER_ENV=docker` (or podman).\n- `JOB_LABELS=...` only works with `CONTAINER_ENV=docker` (or podman) and `CONTAINER_ENV=kubernetes`.\n- `CONTAINER_USER` does not work with `CONTAINER_ENV=cromwell`\n- `CONTAINER_ENV=cromwell` does not forward environment variables.\n- `STORAGE_TYPE=host` is not supported for Kubernetes\n\n## TODO\n\n- [ ] Dev environment and testing for Kubernetes and SLURM.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffnndsc%2Fpman","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffnndsc%2Fpman","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffnndsc%2Fpman/lists"}