{"id":28480486,"url":"https://github.com/hashmap-kz/kubectl-syncpod","last_synced_at":"2026-04-04T10:14:13.779Z","repository":{"id":295760941,"uuid":"990720656","full_name":"hashmap-kz/kubectl-syncpod","owner":"hashmap-kz","description":"🌀 High-Speed File Transfer to and from Kubernetes PVCs","archived":false,"fork":false,"pushed_at":"2025-06-23T10:41:19.000Z","size":161,"stargazers_count":5,"open_issues_count":3,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-06-23T11:39:19.706Z","etag":null,"topics":["golang","kubectl","kubectl-plugin","kubectl-plugins","kubernetes"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hashmap-kz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-26T14:28:11.000Z","updated_at":"2025-06-23T10:41:16.000Z","dependencies_parsed_at":"2025-06-06T09:28:59.628Z","dependency_job_id":null,"html_url":"https://github.com/hashmap-kz/kubectl-syncpod","commit_stats":null,"previous_names":["hashmap-kz/kubectl-syncpod"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/hashmap-kz/kubectl-syncpod","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hashmap-kz%2Fkubectl-syncpod","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hashmap-kz%2Fkubectl-syncpod/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hashmap-kz%2Fkubectl-syncpod/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hashmap-kz%2Fkubectl-syncpod/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hashmap-kz","download_url":"https://codeload.github.com/hashmap-kz/kubectl-syncpod/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hashmap-kz%2Fkubectl-syncpod/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261474475,"owners_count":23164035,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["golang","kubectl","kubectl-plugin","kubectl-plugins","kubernetes"],"created_at":"2025-06-07T19:06:33.893Z","updated_at":"2026-04-04T10:14:13.766Z","avatar_url":"https://github.com/hashmap-kz.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kubectl-syncpod\n\n_High-Speed File Transfer to and from Kubernetes PVCs_\n\n[![License](https://img.shields.io/github/license/hashmap-kz/kubectl-syncpod)](https://github.com/hashmap-kz/kubectl-syncpod/blob/master/LICENSE)\n[![Go Report Card](https://goreportcard.com/badge/github.com/hashmap-kz/kubectl-syncpod)](https://goreportcard.com/report/github.com/hashmap-kz/kubectl-syncpod)\n[![Workflow Status](https://img.shields.io/github/actions/workflow/status/hashmap-kz/kubectl-syncpod/ci.yml?branch=master)](https://github.com/hashmap-kz/kubectl-syncpod/actions/workflows/ci.yml?query=branch:master)\n[![GitHub Issues](https://img.shields.io/github/issues/hashmap-kz/kubectl-syncpod)](https://github.com/hashmap-kz/kubectl-syncpod/issues)\n[![Go Version](https://img.shields.io/github/go-mod/go-version/hashmap-kz/kubectl-syncpod)](https://github.com/hashmap-kz/kubectl-syncpod/blob/master/go.mod#L3)\n[![Latest Release](https://img.shields.io/github/v/release/hashmap-kz/kubectl-syncpod)](https://github.com/hashmap-kz/kubectl-syncpod/releases/latest)\n\n## Table of Contents\n\n- [About](#about)\n    - [Features](#features)\n    - [Typical Use Cases](#typical-use-cases)\n- [Example CLI Usage](#example-cli-usage)\n    - [Upload local directory to PVC](#upload-local-directory-to-pvc)\n    - [Download directory from PVC to local machine](#download-directory-from-pvc-to-local-machine)\n- [Installation](#installation)\n    - [Using krew](#using-krew-coming-soon)\n    - [Homebrew installation](#homebrew-installation)\n    - [Manual Installation](#manual-installation)\n        - [Example installation script](#example-installation-script-for-unix-based-os-_requirements-tar-curl-jq_)\n    - [Package-Based installation](#package-based-installation-suitable-in-cicd)\n        - [Debian](#debian)\n        - [Alpine Linux](#alpine-linux)\n- [Execution Flow](#execution-flow)\n- [Comparison Table](#comparison-table)\n- [License](#license)\n\n---\n\n## About\n\n- While `kubectl cp` and `kubectl exec` can both be used to copy files, performance degrades significantly when the\n  target size is large (e.g., ~100Gi). In such cases, execution becomes drastically slow.\n\n- Additionally, both approaches have limitations: they either require extra tools (such as `tar`) to be installed in the\n  container, or they cannot be used at all with minimal base images like `distroless` or `scratch`.\n\n- Most importantly, these methods do not support concurrent read/write operations - a critical limitation when\n  performance and throughput matter.\n\n### Features\n\n- Upload local files or directories to a pod's mounted volume\n- Download pod volume files back to local machine\n- Safe overwrite protection\n- Auto-rename existing remote directories (`pgdata-new-original-YYYY-MM-DD-HHMMSS`)\n- Concurrent file transfer with worker pool\n- Preserves directory structure\n- Optional automatic `chown` of uploaded files inside the pod (via Kubernetes exec API)\n- Fully based on **SFTP** + **Kubernetes Exec API** — no side effects on other pod processes\n\n### Typical Use Cases\n\nYou’re running PostgreSQL as a StatefulSet, and you need to restore a database from a basebackup and a WAL archive.\nIf the volume is hostPath-based, this is relatively straightforward - you simply copy the required files onto the target\nnode.\nBut when using CSI-backed volumes (e.g., via a cloud provider), where the PVC is mounted as a block device, the\nsituation becomes more complex. In such cases, conventional tools fall short.\n\nAlso - you may want to scale your StatefulSet to zero and back up the PVC contents safely and efficiently - for local\ntesting, migration, or recovery.\n\nBasic Scenarios:\n\n- Download backup from PVC for verification / restore\n- Sync files between PVCs and local environment\n- Testing PVC mount behavior\n- CI/CD pipelines to prepare volume data\n\n---\n\n## Example CLI Usage\n\n**[`^        back to top        ^`](#table-of-contents)**\n\n### Upload local directory to PVC:\n\n```bash\nkubectl-syncpod upload \\\n  --namespace pgrwl-test \\\n  --pvc postgres-data \\\n  --mount-path=/var/lib/postgresql/data \\\n  --src=backups \\\n  --dst=pgdata-new \\\n  --allow-overwrite \\\n  --owner=\"999:999\"\n```\n\nBehavior:\n\n- If `/var/lib/postgresql/data/pgdata-new` exists -\u003e it is renamed (safe overwrite)\n- Contents of `backups/` are uploaded into `/var/lib/postgresql/data/pgdata-new/`\n- File ownership is set to `999:999` inside the helper pod\n\n### Download directory from PVC to local machine:\n\n```bash\nkubectl-syncpod download \\\n  --namespace pgrwl-test \\\n  --pvc postgres-data \\\n  --mount-path=/var/lib/postgresql/data \\\n  --src=pgdata-new \\\n  --dst=backups-copy\n```\n\nBehavior:\n\n- Contents of `/var/lib/postgresql/data/pgdata-new/` are downloaded\n- Files are written to `./backups-copy/`\n- Directory structure is preserved\n\n## Installation\n\n**[`^        back to top        ^`](#table-of-contents)**\n\n### Using `krew` (coming soon)\n\n**Coming soon, [PR](https://github.com/kubernetes-sigs/krew-index/pull/5547) is on review**\n\n1. Install the [Krew](https://krew.sigs.k8s.io/docs/user-guide/setup/) plugin manager if you haven’t already.\n2. Run the following command:\n\n```bash\nkubectl krew install syncpod\n```\n\n### Homebrew installation\n\n```bash\nbrew tap hashmap-kz/homebrew-tap\nbrew install kubectl-syncpod\n```\n\n### Manual Installation\n\n1. Download the latest binary for your platform from\n   the [Releases page](https://github.com/hashmap-kz/kubectl-syncpod/releases).\n2. Place the binary in your system's `PATH` (e.g., `/usr/local/bin`).\n\n#### Example installation script for Unix-Based OS _(requirements: tar, curl, jq)_\n\n```bash\n(\nset -euo pipefail\n\nOS=\"$(uname | tr '[:upper:]' '[:lower:]')\"\nARCH=\"$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\\(arm\\)\\(64\\)\\?.*/\\1\\2/' -e 's/aarch64$/arm64/')\"\nTAG=\"$(curl -s https://api.github.com/repos/hashmap-kz/kubectl-syncpod/releases/latest | jq -r .tag_name)\"\n\ncurl -L \"https://github.com/hashmap-kz/kubectl-syncpod/releases/download/${TAG}/kubectl-syncpod_${TAG}_${OS}_${ARCH}.tar.gz\" |\ntar -xzf - -C /usr/local/bin \u0026\u0026 \\\nchmod +x /usr/local/bin/kubectl-syncpod\n)\n```\n\n### Package-Based installation (suitable in CI/CD)\n\n#### Debian\n\n```\nsudo apt update -y \u0026\u0026 sudo apt install -y curl\ncurl -LO https://github.com/hashmap-kz/kubectl-syncpod/releases/latest/download/kubectl-syncpod_linux_amd64.deb\nsudo dpkg -i kubectl-syncpod_linux_amd64.deb\n```\n\n#### Alpine Linux\n\n```\napk update \u0026\u0026 apk add --no-cache bash curl\ncurl -LO https://github.com/hashmap-kz/kubectl-syncpod/releases/latest/download/kubectl-syncpod_linux_amd64.apk\napk add kubectl-syncpod_linux_amd64.apk --allow-untrusted\n```\n\n---\n\n## Execution Flow\n\n**[`^        back to top        ^`](#table-of-contents)**\n\n`kubectl-syncpod` spins up a **temporary helper pod** that:\n\n- Mounts your target PVC\n- Runs an `sshd` server with an in-memory public key\n- Listens on a randomized NodePort\n- Accepts connections only via a secure, ephemeral SSH private key (never written to disk)\n\nThe CLI then:\n\n- Uses an in-memory SFTP client to **recursively transfer files**\n- Skips files that are **already present and match by SHA-256**\n- Cleans up the helper pod and service automatically\n\n![kubectl-syncpod](docs/assets/flow-v1.svg)\n\n---\n\n## Comparison Table\n\n**[`^        back to top        ^`](#table-of-contents)**\n\n| Feature                                   | `kubectl cp`                    | `kubectl exec`          | `kubectl-syncpod` (SFTP mode)         |\n|-------------------------------------------|---------------------------------|-------------------------|---------------------------------------|\n| Uses sidecar or helper pod                | ❌                               | ❌                       | ✅                                     |\n| Works with PVCs                           | ⚠️ Only if mounted in container | ⚠️ Manual path required | ✅ Helper pod mounts PVC               |\n| Requires tools in container (`tar`, `sh`) | ✅                               | ✅                       | ❌ (uses `sshd` in helper pod)         |\n| Supports `readOnlyRootFilesystem` pods    | ❌                               | ❌                       | ✅                                     |\n| Works on `distroless`/`scratch` images    | ❌                               | ❌                       | ✅                                     |\n| Affects main application container        | ✅                               | ✅                       | ❌                                     |\n| Requires container to run as root         | Often yes                       | Often yes               | ❌ or configurable via helper pod spec |\n| Safe for production workloads             | ⚠️ Risky                        | ⚠️ Risky                | ✅ (safe for read)                     |\n| Auto-cleans after sync                    | ❌                               | ❌                       | ✅                                     |\n| Supports concurrent transfers             | ❌                               | ❌                       | ✅ (parallel SFTP workers)             |\n| Performance on large file trees           | 🐢 Slow                         | 🐢 Slow                 | 🚀 Fast (streaming + concurrency)     |\n\n---\n\n## **License**\n\nThis project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhashmap-kz%2Fkubectl-syncpod","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhashmap-kz%2Fkubectl-syncpod","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhashmap-kz%2Fkubectl-syncpod/lists"}