{"id":19138525,"url":"https://github.com/rsgalloway/distman","last_synced_at":"2026-01-06T06:26:45.164Z","repository":{"id":261727368,"uuid":"879471053","full_name":"rsgalloway/distman","owner":"rsgalloway","description":"super simple file distribution system","archived":false,"fork":false,"pushed_at":"2025-05-06T14:02:46.000Z","size":133,"stargazers_count":1,"open_issues_count":13,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-05-06T14:54:41.873Z","etag":null,"topics":["dist","distribution","file-distribution","package-manager","poetry","poetry-python","setuptools","vfx-pipeline"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rsgalloway.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-10-28T01:03:04.000Z","updated_at":"2025-05-06T14:02:50.000Z","dependencies_parsed_at":"2024-12-09T15:24:00.826Z","dependency_job_id":"a5137121-de6f-4f01-ba64-dd532dfe1b04","html_url":"https://github.com/rsgalloway/distman","commit_stats":null,"previous_names":["rsgalloway/distman"],"tags_count":22,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsgalloway%2Fdistman","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsgalloway%2Fdistman/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsgalloway%2Fdistman/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsgalloway%2Fdistman/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rsgalloway","download_url":"https://codeload.github.com/rsgalloway/distman/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252711055,"owners_count":21792249,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dist","distribution","file-distribution","package-manager","poetry","poetry-python","setuptools","vfx-pipeline"],"created_at":"2024-11-09T06:43:06.052Z","updated_at":"2026-01-06T06:26:45.159Z","avatar_url":"https://github.com/rsgalloway.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"distman\n=======\n\nSimple software distribution for complex pipelines,\n\n`distman` is a config-driven deployment tool that performs safe, versioned rollouts\nof software, scripts, and configuration files to predefined locations on the\nfilesystem. It’s ideal for environments where deterministic deployments,\nenvironment-aware transforms, and clean rollback are more important than packaging\nor pip installation.\n\nWhen used alongside [envstack](https://github.com/rsgalloway/envstack), distman\ncan act as a drop-in alternative to virtualenv or Python packaging, enabling you\nto:\n\n- Install Python tools and libraries via source-controlled builds\n- Swap or rollback versions using atomic symlinks\n- Configure and isolate runtime environments using .env files\n- Avoid the overhead of pip/venv/pyproject setups for internal tools or scripts\n\n## Installation\n\nThe easiest way to install:\n\n```bash\n$ pip install -U distman\n```\n\n## Quickstart\n\n`distman` looks for a dist file called `dist.json` at the root of a directory or\ngit repo. The dist file defines the file distrubution instructions.\n\nThe basic format of the `dist.json` file is:\n\n```json\n{\n    \"author\": \"\u003cemail\u003e\",\n    \"targets\": {\n        \"\u003ctarget\u003e\": {\n            \"source\": \"\u003csource-path\u003e\",\n            \"destination\": \"\u003ctarget-path\u003e\"\n        },\n    }\n}\n```\n\nwhere `\u003csource-path\u003e` is the relative path to the source file, directory or\nlink, and `\u003ctarget-path\u003e` is the target destination path, and `\u003ctarget\u003e` is a\nnamed target label to use when running `distman` commands. You can define as\nmany targets as you need.\n\nSee the `dist.json` file in this repo for an example.\n\nTarget paths can include environment variables, such as those defined in the\n`distman.env` envstack file, where variables in paths are defined with curly\nbrackets only, e.g.:\n\n```bash\n\"{DEPLOY_ROOT}/lib/python/distman\"\n```\n\nWhen files are distributed (or disted), they are copied to a `versions` folder\nand a symlink is created to the version. When a new version is disted, the\nversion number is incremented and the link is updated.\n\n#### Wildcards\n\nYou can use shell-style wildcards (e.g., *) in the \"source\" field of a target\ndefinition to match multiple files or directories. This is useful when you want\nto distribute a group of files without listing each one individually.\n\nWhen using wildcards, you must also use numeric substitution variables (%1, %2,\netc.) in the \"destination\" path. These correspond to the wildcard matches in\norder of appearance.\n\n```json\n\"targets\": {\n  \"build\": {\n    \"source\": \"build/*.py\",\n    \"destination\": \"{DEPLOY_ROOT}/lib/python/%1\"\n  }\n}\n```\n\nIn this example:\n\n- `build/*.py` expands to all `.py` files in the `build/` folder.\n- Each matched file is symlinked to `{DEPLOY_ROOT}/lib/python/filename.py`.\n\n\u003e Wildcards are expanded at runtime using Python's glob and fnmatch mechanisms.\nMatching results are processed and symlinked individually.\n\n#### Transforms\n\n`distman` supports a customizable pipeline of transforms that operate on each\ntarget before final distribution. These transforms can run:\n\n- Custom Python functions (func)\n- Shell commands (script)\n\nPipeline steps can be defined globally at the top of a dist.json file, or\nper-target:\n\n```json\n\"pipeline\": {\n    \"formatting\": {\n        \"script\": [\"black --check {input}\"]\n    },\n    \"replace_tokens\": {\n        \"func\": \"distman.transform.replace_tokens\",\n        \"options\": {\n            \"tokens\": { \"__VERSION__\": \"1.2.3\" }\n        }\n    }\n}\n```\n\n## Usage\n\nTo dist files defined in a `dist.json` file (remove -d when ready):\n\n```bash\n$ dist -d\n```\n\nThis will dist files to the `${DEPLOY_ROOT}` folder defined in the provided\n`distman.env` [envstack](https://github.com/rsgalloway/envstack) file and might\nlook something like this when disted:\n\n```\n${DEPLOY_ROOT}\n├── bin\n│   ├── distman -\u003e versions/distman.0.c73fe42\n│   └── versions\n│       └── distman.0.c73fe42\n└── lib\n    └── python\n        ├── distman -\u003e versions/distman.0.c73fe42\n        └── versions\n            └── distman.0.c73fe42\n                ├── cli.py\n                ├── config.py\n                ├── dist.py\n                ├── __init__.py\n                ├── logger.py\n                ├── source.py\n                └── util.py\n```\n\nTo override the deployment folder, update the `distman.env` environment stack\nfile then re-dist:\n\n```bash\n$ dist [-d]\n```\n\nBy default, `distman` dists to a prod folder under `${DEPLOY_ROOT}`. This can be\nchanged at any time using `${ENV}` or updating or modifying the `distman.env`\nenvstack file:\n\n```bash\n$ ENV=dev dist [-d]\n```\n\nThis will change `prod` to `dev` in the target deplyment path. This is useful\nfor deploying files or code to different development environments.\n\n## Dist Info\n\nWhen disting files, `distman` will create hidden dist info files that meta data\nabout the source files. For example, if the source file is called `foobar.py`\nthen the dist info file that will be created will be called `.foobar.py.dist`.\nThe dist info files will be created at the deployment root.\n\n## Caching\n\nIn high-latency environments (for example, remote users accessing a shared\ndeployment root over VPN, SMB, or NFS), loading tools and libraries directly\nfrom `${DEPLOY_ROOT}` can introduce noticeable startup delays.\n\nTo address this, distman provides an optional local cache mechanism that\nmirrors the currently active versions of deployed files to a local filesystem\nlocation.\n\n### Cache command\n\nTo check or update the cache:\n\n```bash\n$ distman cache [OPTIONS]\n```\n\n## Config\n\nMost configuration is done in the `distman.env`\n[envstack](https://github.com/rsgalloway/envstack) file.\n\nDefault config settings are in the config.py module. The following environment\nvariables are supported:\n\n| Variable        | Description |\n|-----------------|-------------|\n| $DEPLOY_ROOT    | file deployment root directory |\n| $CACHE_ROOT     | local cache root directory |\n| $CACHE_TTL      | time-to-live (seconds) for cache checks |\n| $ENV            | target environment (e.g. prod or dev) |\n| $IGNORE_MISSING | ignore missing source paths in targets |\n| $LOG_DIR        | directory to write log files |\n| $LOG_LEVEL      | logging level to use (DEBUG, INFO, etc) |\n| $MAX_VERSIONS   | max number of versions when checking for changes |\n| $ROOT           | dist root directory |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frsgalloway%2Fdistman","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frsgalloway%2Fdistman","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frsgalloway%2Fdistman/lists"}