{"id":50868092,"url":"https://github.com/nopoz/portrieve","last_synced_at":"2026-06-15T03:07:19.758Z","repository":{"id":360132946,"uuid":"1248832910","full_name":"nopoz/portrieve","owner":"nopoz","description":"Back up, restore, and migrate Portainer stacks as plain Docker Compose files.","archived":false,"fork":false,"pushed_at":"2026-06-11T06:19:36.000Z","size":1128,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-11T08:12:31.174Z","etag":null,"topics":["backup","bash","cli","devops","docker","docker-compose","migration","portainer","restore","self-hosted"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nopoz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-25T05:05:32.000Z","updated_at":"2026-06-11T06:19:32.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/nopoz/portrieve","commit_stats":null,"previous_names":["nopoz/stackport","nopoz/portrieve"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/nopoz/portrieve","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nopoz%2Fportrieve","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nopoz%2Fportrieve/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nopoz%2Fportrieve/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nopoz%2Fportrieve/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nopoz","download_url":"https://codeload.github.com/nopoz/portrieve/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nopoz%2Fportrieve/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34345664,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-15T02:00:07.085Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","bash","cli","devops","docker","docker-compose","migration","portainer","restore","self-hosted"],"created_at":"2026-06-15T03:07:19.227Z","updated_at":"2026-06-15T03:07:19.752Z","avatar_url":"https://github.com/nopoz.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Portrieve\n\n[![Release](https://img.shields.io/github/v/release/nopoz/portrieve?sort=semver)](https://github.com/nopoz/portrieve/releases)\n[![License](https://img.shields.io/github/license/nopoz/portrieve)](LICENSE)\n[![CI](https://github.com/nopoz/portrieve/actions/workflows/ci.yml/badge.svg)](https://github.com/nopoz/portrieve/actions/workflows/ci.yml)\n[![Publish](https://github.com/nopoz/portrieve/actions/workflows/publish.yml/badge.svg)](https://github.com/nopoz/portrieve/actions/workflows/publish.yml)\n[![ghcr.io](https://img.shields.io/badge/ghcr.io-nopoz%2Fportrieve-blue?logo=github)](https://github.com/nopoz/portrieve/pkgs/container/portrieve)\n\n**Back up, restore, and migrate [Portainer](https://www.portainer.io/) stacks as\nplain Docker Compose files.** Portrieve exports every stack from a Portainer\ninstance to version-controllable files, then redeploys them, on the same server\nor a different one, through the Portainer API. It is the round trip Portainer\nitself does not offer: backups you can read and commit, plus a one-command restore\nor migration.\n\n- **Export** writes a synced, Docker-native backup of every stack across every\n  environment: `docker-compose.yml`, `.env`, stack metadata, and per-endpoint\n  network info.\n- **Import** redeploys those backups (or any compose/`.env` files you supply),\n  creating new stacks and recreating the external networks they depend on,\n  including subnet/gateway so stacks that pin static IPs keep working.\n\nPortrieve runs as a container or as a single Bash script. Its commands:\n\n```bash\nportrieve export       # back up every stack\nportrieve import       # restore or migrate stacks\nportrieve test         # check config and connectivity\nportrieve endpoints    # list environments (IDs, names)\nportrieve stacks       # list stacks\n```\n\n![Portrieve demo](demo.gif)\n\n## Contents\n\n- [Features](#features)\n- [Why Portrieve](#why-portrieve)\n- [Prerequisites](#prerequisites)\n- [Run with Docker](#run-with-docker)\n- [Run as a script](#run-as-a-script)\n- [Discovery commands](#discovery-commands)\n- [Export](#export)\n- [Installing yq](#installing-yq)\n- [Import](#import)\n- [Security notes](#security-notes)\n- [Troubleshooting](#troubleshooting)\n- [Disclaimer](#disclaimer)\n- [More of my projects](#more-of-my-projects)\n- [Support](#support)\n- [License](#license)\n\n## Features\n\n- Discovers and processes all Portainer endpoints (environments) automatically\n- Exports each stack's `docker-compose.yml`, `.env`, and raw stack metadata\n- Exports per-endpoint Docker network info (`networks.json`)\n- Sync-style backups: prunes configs for stacks/endpoints deleted in Portainer\n- Imports stacks back into Portainer (standalone Compose and Swarm)\n- Migrates stacks between endpoints/hosts with `--endpoint` (id or name)\n- Recreates `external: true` networks before deploying, carrying over driver,\n  labels, options and IPAM (subnet/gateway)\n- Safe by default: existing stacks are skipped on import unless `--update`\n- `--dry-run` previews every import action without making changes\n- Discovery commands (`test`, `endpoints`, `stacks`) to inspect a Portainer\n  instance and find the values you need for import\n- Runs as a script or as a container, with optional cron-scheduled backups\n- Colorized output, run logging, and automatic retry/backoff on API requests\n\n## Why Portrieve\n\nMost Portainer backup utilities only cover the *backup* half, and Portainer's own\nAPI backup is an all-or-nothing database archive. Portrieve is built around the\n**round trip** and per-stack granularity:\n\n| Capability | Typical backup tools | Native `/api/backup` | This tool |\n|---|:--:|:--:|:--:|\n| Per-stack `docker-compose.yml` | some | no (DB blob) | yes |\n| Per-stack `.env` | rarely | no | yes |\n| Docker network info | no | no | yes |\n| Granular per-stack restore via API | no | no (full DB only) | yes |\n| Recreate external networks (incl. IPAM) | no | n/a | yes |\n| Migrate a stack to another endpoint/host | no | no | yes |\n| Plain files you can commit to git | partial | no | yes |\n\nIf you only need scheduled backups, mature containerized tools already do that\nwell. Portrieve's niche is **getting stacks back in** selectively, on any\nendpoint, with their environment variables and external networks intact, which\nmakes it as much a migration and disaster-recovery tool as a backup one.\n\n## Prerequisites\n\nYou need a Portainer API access token and a reachable Portainer URL, then either:\n\n- **Docker** (simplest): all dependencies are baked into the image, so there is\n  nothing else to install. See [Run with Docker](#run-with-docker).\n- **A shell**, to run the script directly. Requires Bash, `curl`, and `jq`, plus\n  the optional [`yq`](https://github.com/mikefarah/yq) for `import`'s external\n  network detection. Without `yq`, network recreation is skipped (with a warning)\n  and everything else works. See [Installing yq](#installing-yq).\n\n## Run with Docker\n\nEvery Portainer user already runs Docker, so the container is the easiest way to\nuse Portrieve: no Bash, `jq`, or `yq` to install, and it works the same on Linux,\nmacOS, Windows, and NAS systems. Credentials are passed as environment variables\n(`PORTAINER_URL`, `PORTAINER_API_KEY`), and backups land in `/backup`.\n\nPull the published image:\n\n```bash\ndocker pull ghcr.io/nopoz/portrieve:latest\n```\n\nOr build it locally:\n\n```bash\ndocker build -t portrieve .\n```\n\nOne-shot commands (any subcommand works: `export`, `import`, `test`, `endpoints`,\n`stacks`):\n\n```bash\n# Check connectivity\ndocker run --rm \\\n  -e PORTAINER_URL=\"https://portainer.example.com:9443/api\" \\\n  -e PORTAINER_API_KEY=\"ptr_xxxxx\" \\\n  ghcr.io/nopoz/portrieve:latest test\n\n# Export every stack to a host directory\ndocker run --rm \\\n  -e PORTAINER_URL=\"https://portainer.example.com:9443/api\" \\\n  -e PORTAINER_API_KEY=\"ptr_xxxxx\" \\\n  -v \"$PWD/portainer_backups:/backup\" \\\n  ghcr.io/nopoz/portrieve:latest export\n```\n\n### Scheduled backups\n\nSet `SCHEDULE` to a cron expression and the container runs the command on that\nschedule, staying up between runs. This is the recommended way to keep unattended,\nversion-controllable backups.\n\nThe included [`docker-compose.yml`](docker-compose.yml) is a ready-made scheduled\nbackup service. Put your credentials in a local `.env` file (git-ignored), then:\n\n```bash\ndocker compose up -d        # daily export at 03:00 by default\ndocker compose logs -f      # watch runs\n```\n\n| Variable | Purpose |\n|----------|---------|\n| `PORTAINER_URL` | Portainer API URL (include the `/api` suffix) |\n| `PORTAINER_API_KEY` | Portainer API access token |\n| `PORTAINER_INSECURE` | Set `true` to skip TLS verification for a self-signed or private cert (default off) |\n| `SCHEDULE` | Cron expression; when set, the command runs on this schedule |\n| `TZ` | Timezone the schedule is interpreted in (default UTC) |\n| `RUN_ON_START` | Also run once at container start (default `true`) |\n| `PORTAINER_BACKUP_DIR` | Backup directory inside the container (default `/backup`) |\n| `PUID` / `PGID` | Own the exported files as this user/group; the container reconciles the backup tree and drops privileges. Unset runs as root |\n| `PORTAINER_BACKUP_UMASK` | Umask for exported files (default `077`, owner-only; e.g. `027` for group read) |\n\nBackups contain secrets (`.env` values, stack metadata), so exports are written\nowner-only by default. To own them as a specific user, set `PUID`/`PGID`; the\ncontainer reconciles the existing tree and drops privileges before running.\n\nQuote the numeric values in YAML: `PORTAINER_BACKUP_UMASK: \"077\"`, `PUID: \"1000\"`.\nUnquoted, YAML reads a leading-zero number like `077` as octal, so the container\nreceives `63` and derives a world-readable mode. Portrieve guards against this\n(it rejects a non-octal umask and falls back to `077`), but quoting is correct.\n\n## Run as a script\n\nPrefer to run it directly on a host? Portrieve is a single Bash script\n(`portrieve.sh`).\n\n1. Clone the repository and make the script executable:\n   ```bash\n   git clone https://github.com/nopoz/portrieve.git\n   cd portrieve\n   chmod +x portrieve.sh\n   ```\n\n2. Provide credentials as environment variables:\n   ```bash\n   export PORTAINER_URL=\"http://YOUR_PORTAINER_IP:9000/api\"\n   export PORTAINER_API_KEY=\"YOUR_API_KEY\"\n   ```\n   or in a config file (the variables above override it):\n   ```bash\n   cp .portainer_config.sample .portainer_config\n   # then edit api_key and portainer_url\n   ```\n   The URL **must** include the `/api` suffix. Generate an API key in Portainer\n   under **My account → Access tokens**, and keep credentials out of version\n   control (`.portainer_config` is already git-ignored).\n\n3. Run any command:\n   ```bash\n   # Back up every stack into ./portainer_backups\n   ./portrieve.sh export\n\n   # Preview a full restore (no changes made)\n   ./portrieve.sh import --source portainer_backups --dry-run\n\n   # Restore for real (existing stacks are skipped, not overwritten)\n   ./portrieve.sh import --source portainer_backups\n   ```\n\n## Discovery commands\n\nRead-only helpers for inspecting a Portainer instance, handy for confirming your\nsetup works and for finding the endpoint ID/name to pass to `import --endpoint`.\nAll three print a human-readable table by default, or raw JSON with `--json`.\n\n```bash\n# Verify config, connectivity and API key; report how much is visible\n./portrieve.sh test\n\n# List environments: use an ID or NAME here for import's --endpoint\n./portrieve.sh endpoints\n\n# List all stacks, or just those on one endpoint\n./portrieve.sh stacks\n./portrieve.sh stacks --endpoint nas        # by name\n./portrieve.sh stacks --endpoint 1 --json   # by id, as JSON\n```\n\nExample:\n\n```\n$ ./portrieve.sh endpoints\nID    NAME                         TYPE          STATUS\n1     nas                          Docker        up\n16    htpc                         Docker-Agent  up\n20    desktop                      Docker-Agent  up\n\n$ ./portrieve.sh test\n[INFO] Testing connection to http://192.168.1.2:9000/api\n[SUCCESS] Connected and authenticated (HTTP 200)\n[INFO] Endpoints visible: 3\n[INFO] Stacks visible: 101\n```\n\n`test` exits non-zero on failure and reports the cause (e.g. authentication\nfailed, host unreachable), making it suitable for scripts and health checks.\n\n## Export\n\n```bash\n./portrieve.sh export\n```\n\n| Option           | Description                                      |\n|------------------|--------------------------------------------------|\n| `--config FILE`  | Config file path (default `.portainer_config`)   |\n| `--out DIR`      | Output directory (default `portainer_backups`)   |\n\nThe export tree is **synced** on every run: directories for stacks or endpoints\nthat no longer exist in Portainer are removed from the backup, so the directory\nalways mirrors live state.\n\n### Output structure\n\n```\nportainer_backups/\n├── export.log\n└── endpoint-name-{endpoint_id}/\n    ├── networks.json\n    └── stack-name/\n        ├── docker-compose.yml\n        ├── .env\n        └── stack_metadata.json\n```\n\n- `export.log`: detailed log of the most recent export run\n- `networks.json`: user-defined Docker networks for the endpoint (name, driver,\n  labels, IPAM), used by `import` to recreate external networks. Predefined\n  `bridge`/`host`/`none` networks are excluded. Endpoints whose Docker API is\n  unreachable are skipped here without failing the run.\n- Per stack: the compose file, its environment variables (if any), and the raw\n  stack metadata returned by Portainer\n\n## Installing yq\n\n\u003e **Important:** there are two unrelated tools named `yq`. This script requires\n\u003e **[mikefarah/yq](https://github.com/mikefarah/yq) v4+** (written in Go). The\n\u003e Python `yq` (kislyuk, a `jq` wrapper) uses different syntax and is **not\n\u003e compatible**; the script detects it at runtime and warns. On Debian/Ubuntu,\n\u003e `apt install yq` typically installs the wrong (Python) one, so prefer the\n\u003e options below.\n\n| Platform | Command |\n|----------|---------|\n| macOS / Linux (Homebrew) | `brew install yq` |\n| Linux (Snap) | `sudo snap install yq` |\n| Alpine | `apk add yq` |\n| Arch | `pacman -S go-yq` |\n| Windows | `choco install yq` · `winget install yq` · `scoop install yq` |\n\n**Debian / Ubuntu** (where `apt` gives the wrong `yq`): use Snap if available,\notherwise grab the static binary (no prerequisites, works in containers/WSL):\n\n```bash\nsudo curl -sSL -o /usr/local/bin/yq \\\n  \"https://github.com/mikefarah/yq/releases/latest/download/yq_linux_$(dpkg --print-architecture)\"\nsudo chmod +x /usr/local/bin/yq\nyq --version   # should print: yq (https://github.com/mikefarah/yq/) version v4.x\n```\n\n## Import\n\nImport can read this tool's own backups, a single stack directory, or an\narbitrary compose file you provide.\n\n```bash\n# Import every stack from a backup tree into their original endpoints\n./portrieve.sh import --source portainer_backups\n\n# Preview first, without making changes\n./portrieve.sh import --source portainer_backups --dry-run\n\n# Import one backed-up stack into a specific endpoint\n./portrieve.sh import --stack portainer_backups/nas-1/grafana --endpoint 2\n\n# Import an arbitrary compose file as a new stack\n./portrieve.sh import --compose ./my-compose.yml --name myapp --endpoint 1\n\n# Overwrite stacks that already exist (otherwise they are skipped)\n./portrieve.sh import --source portainer_backups --update\n```\n\n| Option                | Description                                                        |\n|-----------------------|--------------------------------------------------------------------|\n| `--config FILE`       | Config file path (default `.portainer_config`)                     |\n| `--source DIR`        | Import all stacks under a backup tree (default `portainer_backups`)|\n| `--stack DIR`         | Import a single stack directory                                    |\n| `--compose FILE`      | Import a single compose file (requires `--name`)                   |\n| `--name NAME`         | Stack name (used with `--compose`)                                 |\n| `--endpoint ID\\|NAME` | Target endpoint override (else uses metadata `EndpointId`)         |\n| `--update`            | Update stacks that already exist (default: skip them)              |\n| `--prune`             | With `--update`, prune services removed from the compose file      |\n| `--dry-run`           | Show planned actions without calling any write endpoints           |\n\nThe three source modes are mutually exclusive and take precedence in this order:\n`--compose` → `--stack` → `--source`.\n\n`--stack` and `--source` recognize any standard compose filename (`compose.yaml`,\n`compose.yml`, `docker-compose.yaml`, `docker-compose.yml`); if a directory holds\nmore than one, Docker's precedence order picks the winner.\n\n### Importing your own existing compose files\n\nYou do not need a portrieve backup to import. Point `--compose` at any single\nfile, or arrange a folder so each stack lives in its own subdirectory (the\nsubdirectory name becomes the stack name) and bulk-import with `--source`:\n\n```text\nmystacks/\n├── grafana/compose.yaml\n├── immich/docker-compose.yml\n└── nginx/\n    ├── compose.yaml\n    └── .env\n```\n\n```bash\n# Bulk-import a folder of existing stacks onto endpoint 1\n./portrieve.sh import --source mystacks --endpoint 1\n\n# Or one file at a time (any filename), naming each stack explicitly\n./portrieve.sh import --compose ./grafana.yaml --name grafana --endpoint 1\n```\n\nA sibling `.env` next to a compose file is picked up automatically. Re-run with\n`--update` to reconcile changes; existing stacks are skipped otherwise.\n\n### How import resolves things\n\n- **Endpoint:** `--endpoint` (id or name) wins; otherwise the `EndpointId` from a\n  backed-up `stack_metadata.json`. The resolved id is validated against the live\n  endpoint list before anything is deployed.\n- **Environment variables:** a stack's `.env` is parsed back into Portainer\n  environment pairs. Blank lines and `#` comments are ignored; each remaining line\n  is split on its first `=`, so values may themselves contain `=`.\n- **Type:** Swarm vs standalone is read from `stack_metadata.json` (`Type` 1 =\n  Swarm, 2 = Compose); explicit `--compose` imports default to standalone Compose.\n  For Swarm, the target cluster's `SwarmID` is detected from the endpoint on a\n  best-effort basis.\n- **Networks:** networks declared *inside* the compose file are created by\n  Portainer automatically. Networks marked `external: true` are recreated first,\n  carrying over the matching `networks.json` entry's driver, labels, options and\n  IPAM (subnet/gateway) when one is found, so stacks that pin static IPs\n  (`ipv4_address`) keep working. With no saved entry a plain `bridge` network is\n  created. This step requires a compatible [`yq`](#installing-yq). Importing a\n  stack folder in isolation (away from its endpoint's `networks.json`) falls back\n  to `bridge` for external networks.\n- **Conflicts:** a stack whose name already exists on the target endpoint is\n  skipped with a warning, unless `--update` is given (then it is updated via PUT).\n  Import is therefore safe to re-run.\n\n### Cross-host migration caveats\n\nWhen importing a stack onto a *different* host than it came from, a few compose\ndetails are host-specific and may need attention:\n\n- **Host-pinned ports.** A binding like `192.168.1.2:8191:8191` ties the port to one\n  host's IP and will not bind on another host (\"cannot assign requested address\").\n  Drop the IP prefix (`8191:8191`) or change it before importing.\n- **Bind-mount paths.** Host paths in `volumes:` (for example `/volume2/docker/...`)\n  must exist on the target, or Docker will create them empty and the service\n  starts with no data. Provision the paths first when migrating stateful stacks.\n- **Subnet conflicts.** A recreated external network keeps its original subnet. If\n  that subnet overlaps a network already on the target host, creation fails; pick a\n  free subnet or remove the fixed `IPAM` config before importing.\n\n## Security notes\n\n- `.portainer_config` holds your API key; keep it out of version control (it is\n  git-ignored here).\n- Exported `.env` files and metadata may contain secrets. The backup directory is\n  git-ignored and is **not** encrypted by this tool, so protect it accordingly.\n- Prefer HTTPS for `portainer_url` in production.\n\n## Troubleshooting\n\n1. **Cannot connect to Portainer**: verify `portainer_url` (including the `/api`\n   suffix) and that the API key is valid with sufficient permissions.\n2. **Missing dependencies**: install `curl` and `jq` (e.g. `apt install curl jq`).\n   For full import network support, install `yq` (see [Installing yq](#installing-yq)).\n3. **External networks not recreated on import**: install a compatible `yq`. If\n   import warns *\"Found an incompatible 'yq'\"*, you have the Python `yq` instead\n   of [mikefarah/yq](https://github.com/mikefarah/yq) v4+; reinstall via the\n   methods in [Installing yq](#installing-yq).\n4. **Permission denied**: run `chmod +x portrieve.sh` and ensure you have write\n   access to the backup directory.\n\n## Disclaimer\n\nProvided as-is, without warranty. Import creates and updates real stacks on your\nPortainer instance; always run with `--dry-run` first to review what will happen,\nand verify you have backups before using `--update` against live environments.\n\n## More of my projects\n\nOther open-source tools I maintain that you might find useful:\n\n- [**Hosaka**](https://github.com/nopoz/hosaka) - Docker image update monitor with notifications and one-click updates.\n- [**pfSense DNSCrypt Proxy**](https://github.com/nopoz/pfsense-dnscrypt-proxy) - a pfSense package for DNSCrypt Proxy: encrypted DNS with full GUI support.\n\n## Support\n\nIf Portrieve is useful to you, please [star it on GitHub](https://github.com/nopoz/portrieve) to support the project.\n\n## License\n\nReleased under the [MIT License](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnopoz%2Fportrieve","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnopoz%2Fportrieve","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnopoz%2Fportrieve/lists"}