{"id":18676659,"url":"https://github.com/mikeghen/kubernetes-gcs-sftp","last_synced_at":"2025-10-12T20:11:37.230Z","repository":{"id":81764425,"uuid":"116197839","full_name":"mikeghen/kubernetes-gcs-sftp","owner":"mikeghen","description":"SFTP Server using Google Cloud Storage","archived":false,"fork":false,"pushed_at":"2019-08-14T16:56:17.000Z","size":4,"stargazers_count":62,"open_issues_count":5,"forks_count":16,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-25T21:51:19.532Z","etag":null,"topics":["docker","google-cloud-storage","kubernetes","sftp"],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mikeghen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-01-04T01:06:40.000Z","updated_at":"2023-11-14T06:51:15.000Z","dependencies_parsed_at":"2023-03-12T14:00:09.040Z","dependency_job_id":null,"html_url":"https://github.com/mikeghen/kubernetes-gcs-sftp","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mikeghen%2Fkubernetes-gcs-sftp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mikeghen%2Fkubernetes-gcs-sftp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mikeghen%2Fkubernetes-gcs-sftp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mikeghen%2Fkubernetes-gcs-sftp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mikeghen","download_url":"https://codeload.github.com/mikeghen/kubernetes-gcs-sftp/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248505921,"owners_count":21115354,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","google-cloud-storage","kubernetes","sftp"],"created_at":"2024-11-07T09:30:21.265Z","updated_at":"2025-10-12T20:11:32.187Z","avatar_url":"https://github.com/mikeghen.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GCS SFTP Server\nSFTP Server designed to store data in Google Cloud Storage (GCS) Buckets\n\nThis is based upon [atmoz/sftp](https://github.com/atmoz/sftp) project.\n\n# Dockerfile\nWe need to setup an image (based on atomz/sftp) so that we can mount to Google Cloud Storage. That means just installing [gcsfuse](https://github.com/GoogleCloudPlatform/gcsfuse/tree/master/docs).\n\nFind and build your own image using the `Dockerfile` provided.\n\n\n# Mounting Buckets\nWe use gcsfuse `--uid`, `--gid`, and `--only-dir` arguments to mount each SFTP users home directory to a single bucket. Inside the bucket, we create a directory for each user manually. (Not sure if using `--only-dir` will work unless the directory already exists)\n\nSample Bucket Directory Structure:\n```\nbucket-name\n  - /user1\n  - /user2\n```\n\nThe mounting is done in `etc/sftp.d/mount_user_directories.sh`. When deploying to Kubernetes, this script gets executed as a `postStart` command.\n\n## Access Control for GCS Bucket\nWe just need to ensure your GKE cluster is created with the OAuth scope https://www.googleapis.com/auth/devstorage.read_write, and everything else will be handled automatically. Alternatively, we can mount a file in Service Account JSON key.\n\n# Setup Instructions\n## Dependancies\nFor testing, you will need to have Minikube and Docker installed.\n\nFor deployment, you will need to have the gcloud SDK.\n\n## Configuration\nYou can configure SFTP user accounts by adjusting what's in `etc/sftp/users.conf` and `etc/sftp.d/mount_user_directories.sh`.\n\nWhen adding a new user, add a new line into `etc/sftp/users.conf`:\n```\nusername:password:uid:gid:directory\n```\nWhere `uid` is a number (e.g. 1003) and `gid` is a number (e.g. 1003).\nAnd then add a new line into `etc/sftp.d/mount_user_directories.sh` to monunt their `directory` to a GCS bucket:\n```\nrunuser -l partner1 -c \\\n'export GOOGLE_APPLICATION_CREDENTIALS=/credentials/gcloud-key.json \u0026\u0026 \\\ngcsfuse -o nonempty --only-dir username bucket /home/username/ftp'\n```\nThis command will mount the bucket as the given user. It also does some environment variable trickery.\n\n:warning: User passwords are committed to this repo as a demo. Not the best to commit them in practice.\n\n## Production Deployment\nTo deploy to GKE follow these steps:\n\n### To Do\n- [ ] Push docker image to dockerhub\n- [ ] Document production deployment instructions\n\n## Development Setup for Testing\nFollow these steps to run this locally with `minikube`.\n\n### 1. Start minikube:\n```\nminikube start\n```\n\n### 2. Tell minikube to use local docker images:\n```\neval $(minikube docker-env)\n```\n\n### 3. Build a local image from the `Dockerfile`:\n```\ndocker build --rm -t mikeghen/kube-sftp .\n```\n\n### 4. Setup Secrets and Config Mappings\nYou'll need to adjust files in `etc` so that it reflects the SFTP users you're planning to use. You'll also need a Service Account as well.\n\nThen, you can run these commands to put these files on the cluster as secrets:\n```\nkubectl create secret generic users --from-file=users.conf=./etc/sftp/users.conf\nkubectl create secret generic sftp-gcloud-key --from-file=gcloud-key.json=./secrets/gcloud-key.json\nkubectl create configmap gcs-mounts --from-file=gcs-mounts.sh=./etc/sftp.d/gcs-mounts.sh\n```\n* **users** - Code for maintaining users credentials for SFTP access\n* **sftp-cloud-key** - JSON Key for GCS Service Account\n* **gcs-mounts** - Code for mounting GCS bucket\n\n### 5. Deploy the SFTP server to Kubernetes:\n```\nkubectl apply -f sftp.yaml\n```\n### 6. Get the test IP and port:\n```\nminikube service sftp --url\n```\nThis will give you the IP and NodePort port.\n\n:information_source: We use NodePort 30022 for SFTP.\n\n### 7. Confirm you can SFTP using the usernames and password you setup in `etc/sftp*` with `sftp` utility:\n```\n$ sftp -P 30022 username@192.168.99.100\nusername@192.168.99.100's password:\nsftp\u003e pwd\n/directory\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmikeghen%2Fkubernetes-gcs-sftp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmikeghen%2Fkubernetes-gcs-sftp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmikeghen%2Fkubernetes-gcs-sftp/lists"}