{"id":27404762,"url":"https://github.com/nuvo/skbn","last_synced_at":"2025-04-14T05:47:14.537Z","repository":{"id":49835625,"uuid":"146213160","full_name":"nuvo/skbn","owner":"nuvo","description":"Copy files and directories between Kubernetes and cloud storage","archived":false,"fork":false,"pushed_at":"2024-06-12T13:35:52.000Z","size":113,"stargazers_count":75,"open_issues_count":20,"forks_count":75,"subscribers_count":10,"default_branch":"master","last_synced_at":"2024-12-30T12:45:38.981Z","etag":null,"topics":["abs","aws","azure","azure-blob-storage","go","golang","kubernetes","minio","s3"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nuvo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-08-26T20:14:41.000Z","updated_at":"2024-12-02T08:04:43.000Z","dependencies_parsed_at":"2024-06-18T17:06:07.516Z","dependency_job_id":"13997d7f-6c04-4a84-927a-72cc9c9b0b0d","html_url":"https://github.com/nuvo/skbn","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nuvo%2Fskbn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nuvo%2Fskbn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nuvo%2Fskbn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nuvo%2Fskbn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nuvo","download_url":"https://codeload.github.com/nuvo/skbn/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248830418,"owners_count":21168272,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["abs","aws","azure","azure-blob-storage","go","golang","kubernetes","minio","s3"],"created_at":"2025-04-14T05:47:09.527Z","updated_at":"2025-04-14T05:47:14.511Z","avatar_url":"https://github.com/nuvo.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Release](https://img.shields.io/github/release/nuvo/skbn.svg)](https://github.com/nuvo/skbn/releases)\n[![Travis branch](https://img.shields.io/travis/nuvo/skbn/master.svg)](https://travis-ci.org/nuvo/skbn)\n[![Docker Pulls](https://img.shields.io/docker/pulls/nuvo/skbn.svg)](https://hub.docker.com/r/nuvo/skbn/)\n[![Go Report Card](https://goreportcard.com/badge/github.com/nuvo/skbn)](https://goreportcard.com/report/github.com/nuvo/skbn)\n[![license](https://img.shields.io/github/license/nuvo/skbn.svg)](https://github.com/nuvo/skbn/blob/master/LICENSE)\n\n# Skbn\n\nSkbn is a tool for copying files and directories between Kubernetes and cloud storage providers. It is named after the 1981 video game [Sokoban](https://en.wikipedia.org/wiki/Sokoban).\nSkbn uses an in-memory buffer for the copy process, to avoid excessive memory consumption.\nSkbn currently supports the following providers:\n\n* AWS S3\n* Minio S3\n* Azure Blob Storage\n\n## Install\n\n### Prerequisites\n\n1. git\n2. [dep](https://github.com/golang/dep)\n\n### From a release\n\nDownload the latest release from the [Releases page](https://github.com/nuvo/skbn/releases) or use it with a [Docker image](https://hub.docker.com/r/nuvo/skbn)\n\n### From source\n\n```\nmkdir -p $GOPATH/src/github.com/nuvo \u0026\u0026 cd $_\ngit clone https://github.com/nuvo/skbn.git \u0026\u0026 cd skbn\nmake\n```\n\n## Usage\n\n### Copy files from Kubernetes to S3\n\n```\nskbn cp \\\n    --src k8s://\u003cnamespace\u003e/\u003cpodName\u003e/\u003ccontainerName\u003e/\u003cpath\u003e \\\n    --dst s3://\u003cbucket\u003e/\u003cpath\u003e\n```\n\n### Copy files from S3 to Kubernetes\n\n```\nskbn cp \\\n    --src s3://\u003cbucket\u003e/\u003cpath\u003e \\\n    --dst k8s://\u003cnamespace\u003e/\u003cpodName\u003e/\u003ccontainerName\u003e/\u003cpath\u003e\n```\n\n### Copy files from Kubernetes to Azure Blob Storage\n\n```\nskbn cp \\\n    --src k8s://\u003cnamespace\u003e/\u003cpodName\u003e/\u003ccontainerName\u003e/\u003cpath\u003e \\\n    --dst abs://\u003caccount\u003e/\u003ccontainer\u003e/\u003cpath\u003e\n```\n\n### Copy files from Azure Blob Storage to Kubernetes\n\n```\nskbn cp \\\n    --src abs://\u003caccount\u003e/\u003ccontainer\u003e/\u003cpath\u003e \\\n    --dst k8s://\u003cnamespace\u003e/\u003cpodName\u003e/\u003ccontainerName\u003e/\u003cpath\u003e\n```\n\n## Advanced usage\n\n### Copy files from source to destination in parallel\n\n```\nskbn cp \\\n    --src ... \\\n    --dst ... \\\n    --parallel \u003cn\u003e\n```\n* `n` is the number of files to be copied in parallel (for full parallelism use 0)\n\n### Set in-memory buffer size\n\nSkbn copies files using an in-memory buffer. To control the buffer size:\n\n```\nskbn cp \\\n    --src ... \\\n    --dst ... \\\n    --buffer-size \u003cf\u003e\n```\n* `f` is the in-memory buffer size (in MB) to use for files copy. This flag should be used with caution when used in conjunction with `--parallel`\n* The default value for `buffer-size` is 6.75 MB, and was decided based on benchmark\n\n### Minio S3 support\n\nSkbn supports file copy from and to a Minio S3 endpoint. To let skbn know how your minio is configured, you can set the following environment variables:\n\n```\nAWS_ACCESS_KEY_ID=\u003cyour username\u003e\nAWS_SECRET_ACCESS_KEY=\u003cyour password\u003e\nAWS_S3_ENDPOINT=http(s)://\u003chost\u003e:\u003cport\u003e\nAWS_S3_NO_SSL=true # disables SSL\nAWS_S3_FORCE_PATH_STYLE=true # enforce path style bucket access\n```\n\n## Added bonus section\n\n### Copy files from S3 to Azure Blob Storage\n\n```\nskbn cp \\\n    --src s3://\u003cbucket\u003e/\u003cpath\u003e \\\n    --dst abs://\u003caccount\u003e/\u003ccontainer\u003e/\u003cpath\u003e\n```\n\n### Copy files from Azure Blob Storage to S3\n\n```\nskbn cp \\\n    --src abs://\u003caccount\u003e/\u003ccontainer\u003e/\u003cpath\u003e \\\n    --dst s3://\u003cbucket\u003e/\u003cpath\u003e\n```\n\n### Copy files from Kubernetes to Kubernetes\n\n```\nskbn cp \\\n    --src k8s://\u003cnamespace\u003e/\u003cpodName\u003e/\u003ccontainerName\u003e/\u003cpath\u003e \\\n    --dst k8s://\u003cnamespace\u003e/\u003cpodName\u003e/\u003ccontainerName\u003e/\u003cpath\u003e\n```\n\n### Copy files from S3 to S3\n\n```\nskbn cp \\\n    --src s3://\u003cbucket\u003e/\u003cpath\u003e \\\n    --dst s3://\u003cbucket\u003e/\u003cpath\u003e\n```\n\n### Copy files from Azure Blob Storage to Azure Blob Storage\n\n```\nskbn cp \\\n    --src abs://\u003caccount\u003e/\u003ccontainer\u003e/\u003cpath\u003e \\\n    --dst abs://\u003caccount\u003e/\u003ccontainer\u003e/\u003cpath\u003e\n```\n\n## Credentials\n\n\n### Kubernetes\n\nSkbn tries to get credentials in the following order:\n1. if `KUBECONFIG` environment variable is set - skbn will use the current context from that config file\n2. if `~/.kube/config` exists - skbn will use the current context from that config file with an [out-of-cluster client configuration](https://github.com/kubernetes/client-go/tree/master/examples/out-of-cluster-client-configuration)\n3. if `~/.kube/config` does not exist - skbn will assume it is working from inside a pod and will use an [in-cluster client configuration](https://github.com/kubernetes/client-go/tree/master/examples/in-cluster-client-configuration)\n\n\n### AWS\n\nSkbn uses the default AWS [credentials chain](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html).\nIn addition, the `AWS_REGION` environment variable should be set (default is `eu-central-1`).\n\n### Azure Blob Storage\n\nSkbn uses `AZURE_STORAGE_ACCOUNT` and `AZURE_STORAGE_ACCESS_KEY` environment variables for authentication.\n\n## Examples\n\n1. [In-cluster example](/examples/in-cluster)\n2. [Code example](/examples/code)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnuvo%2Fskbn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnuvo%2Fskbn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnuvo%2Fskbn/lists"}