{"id":13560853,"url":"https://github.com/stv0g/gose","last_synced_at":"2025-05-08T23:44:06.448Z","repository":{"id":37988850,"uuid":"467490754","full_name":"stv0g/gose","owner":"stv0g","description":"A tera-scale file uploader","archived":false,"fork":false,"pushed_at":"2025-04-28T05:38:20.000Z","size":892,"stargazers_count":238,"open_issues_count":18,"forks_count":8,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-05-08T23:43:57.508Z","etag":null,"topics":["file-sharing","file-upload","golang","s3","self-hosted"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stv0g.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-03-08T11:50:49.000Z","updated_at":"2025-05-07T15:13:14.000Z","dependencies_parsed_at":"2024-01-07T14:10:15.226Z","dependency_job_id":"246b2ffb-b488-467d-89bd-2b6e68430e34","html_url":"https://github.com/stv0g/gose","commit_stats":null,"previous_names":[],"tags_count":20,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stv0g%2Fgose","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stv0g%2Fgose/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stv0g%2Fgose/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stv0g%2Fgose/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stv0g","download_url":"https://codeload.github.com/stv0g/gose/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253166474,"owners_count":21864467,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["file-sharing","file-upload","golang","s3","self-hosted"],"created_at":"2024-08-01T13:00:50.126Z","updated_at":"2025-05-08T23:44:06.437Z","avatar_url":"https://github.com/stv0g.png","language":"Go","funding_links":[],"categories":["Go","Software"],"sub_categories":["File Transfer - Single-click \u0026 Drag-n-drop Upload"],"readme":"\u003c!--\nSPDX-FileCopyrightText: 2023 Steffen Vogel \u003cpost@steffenvogel.de\u003e\nSPDX-License-Identifier: Apache-2.0\n--\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003cimg style=\"width: 30%; margin: 4em 0\" src=\"frontend/img/gose-logo.svg\" alt=\"GoSƐ logo\" /\u003e\n    \u003ch1 align=\"center\"\u003eGoSƐ: A terascale file-uploader\u003c/h1\u003e\n\u003c/p\u003e\n\n\u003c!-- [![Codacy coverage](https://img.shields.io/codacy/coverage/27eec133fcfd4459885d78f52d03daa8?style=flat-square)](https://app.codacy.com/gh/stv0g/gose/) --\u003e\n\u003c!-- [![GitHub Workflow Status](https://img.shields.io/github/workflow/status/stv0g/wice/build?style=flat-square)](https://github.com/stv0g/wice/actions) --\u003e\n[![goreportcard](https://goreportcard.com/badge/github.com/stv0g/gose?style=flat-square)](https://goreportcard.com/report/github.com/stv0g/gose/)\n[![Codacy grade](https://img.shields.io/codacy/grade/27eec133fcfd4459885d78f52d03daa8?style=flat-square)](https://app.codacy.com/gh/stv0g/gose/)\n[![License](https://img.shields.io/github/license/stv0g/gose?style=flat-square)](https://github.com/stv0g/gose/blob/main/LICENSE)\n![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/stv0g/gose?style=flat-square)\n[![Go Reference](https://pkg.go.dev/badge/github.com/stv0g/gose.svg)](https://pkg.go.dev/github.com/stv0g/gose)\n\nGoSƐ is a modern and scalable file-uploader focusing on scalability and simplicity. It is a little hobby project I’ve been working on over the last weekends.\n\nThe only requirement for GoSƐ is a S3 storage backend which allows to it to scale horizontally without the need for additional databases or caches. Uploaded files a divided into equally sized chunks which are hashed with a MD5 digest in the browser for upload. This allows GoSƐ to skip chunks which already exist. Seamless resumption of interrupted uploads and storage savings are the consequence.\n\nAnd either way both upload and downloads are always directed directly at the S3 server so GoSƐ only sees a few small HTTP requests instead of the bulk of the data. Behind the scenes, GoSƐ uses many of the more advanced S3 features like [Multi-part Uploads](https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html) and [Pre-signed Requests](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html) to make this happen.\n\nUsers have a few options to select between multiple pre-configured S3 buckets/servers or enable browser \u0026 mail notifications about completed uploads. A customisable retention / expiration time for each upload is also selectable by the user and implemented by [S3 life-cycle policies](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html). Optionally, users can also opt-in to use an external service to shorten the URL of the uploaded file.\n\nCurrently a single concurrent upload of a single file is supported. Users can observe the progress via a table of details statistics, a progress-bar and a chart showing the current transfer speed.\n\nGoSƐ aims at keeping its deployment simple and by bundling both front- \u0026 backend components in a single binary or Docker image. GoSƐ has been tested with AWS S3, Ceph’s RadosGW and Minio. Pre-built binaries and Docker images of GoSƐ are available for all major operating systems and architectures at the [release page](https://github.com/stv0g/gose/releases).\n\nGoSƐ is open-source software licensed under the Apache 2.0 license.\n\nCheck our my [blog article](https://noteblok.net/2022/04/03/gos%c9%9b-a-terascale-file-uploader/) for more background info.\n\n## Features\n\n-   De-duplication of uploaded files based on their content-hash\n    -   Uploads of existing files will complete in no-time without re-upload\n-   S3 Multi-part uploads\n    -   Resumption of interrupted uploads\n-   Drag \u0026 Drop of files\n-   Browser notifications about failed \u0026 completed uploads\n-   User-provided object expiration/retention time\n-   Copy URL of uploaded file to clip-board\n-   Detailed transfer statistics and progress-bar / chart\n-   Installation via single binary or container\n    -   JS/HTML/CSS Frontend is bundled into binary\n-   Scalable to multiple replicas\n    -   All state is kept in the S3 storage backend\n    -   No other database or cache is required\n-   Direct up \u0026 download to Amazon S3 via presigned URLs\n    -   GoSƐ deployment does not see an significant traffic\n-   UTF-8 filenames\n-   Multiple user-selectable buckets / servers\n-   Optional link shortening via an external service\n-   Optional notification about new uploads via [shoutrrr](https://containrrr.dev/shoutrrr/v0.5/)\n    -   Mail notifications to user-provided recipient\n-   Cross-platform support:\n    -   Operating systems: Windows, macOS, Linux, BSD\n    -   Architectures: arm64, amd64, armv7, i386\n\n## Roadmap\n\nCheckout the [Github issue tracker](https://github.com/stv0g/gose/issues?q=is%3Aissue+is%3Aopen+label%3Aenhancement).\n\n## [Demo](https://gose.0l.de) (click for Live-Demo)\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://gose.0l.de\"\u003e\n        \u003cimg style=\"max-width: 400px\" src=\"https://user-images.githubusercontent.com/285829/161408820-f0f304ff-2066-462a-98d6-d59a79889d83.gif\" alt=\"GoSƐ demo screencast\" /\u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n\n## Installation\n\n### Pre-compiled binaries from GitHub releases\n\nTake the download link for your OS/Arch from the [Releases Page](https://github.com/stv0g/gose/releases/) and run:\n\n```bash\nsudo wget https://github.com/stv0g/gose/releases/download/v0.0.2/gose_0.0.2_linux_amd64 -O /usr/local/bin/gose\nchmod +x /usr/local/bin/gose\n```\n\n### Kubernetes / Kustomize\n\n1.  Copy default configuration file: `cp config.yaml kustomize/config.yaml`\n2.  Adjust config: `nano kustomize/config.yaml`\n3.  Apply configuration: `kubectl apply -k kustomize`\n\n### Docker\n\nVia environment variables in `.env` file:\n\n```bash\ndocker run --env-file=.env --publish=8080:8080 ghcr.io/stv0g/gose\n```\n\nor via a configuration file:\n\n```bash\ndocker run -v$(pwd)/config.yaml:/config.yaml --publish=8080:8080 ghcr.io/stv0g/gose -config /config.yaml\n```\n\n#### Docker Compose\n\nWe ship a `compose.yaml` file to get you started.\nPlease adjust the environment variables in it and then run:\n\n```bash\ndocker compose up -d\n```\n\n#### Nix\n\nGoSƐ is available in [Nixpkgs](https://github.com/NixOS/nixpgks) as well as a flake:\n\n```bash\nnix profile install nixpkgs#gose\n\n# Or via our flake\nnix profile install github:stv0g/gose#gose\n```\n\n## Configuration\n\nGoSƐ can be configured via a configuration file and/or environment variables\n\n### File\n\nFor reference have a look at the [example configuration file](config.yaml).\n\n### Environment variables\n\nAll settings from the configuration file can also be set via environment variables:\n\n| Variable                               | Default Value                                                             | Description                           |\n| :--                                    | :--                                                                       | :--                                   |\n| `GOSE_LISTEN`                          | `\":8080\"`                                                                 | Listen address and port of Gose       |\n| `GOSE_BASE_URL`                        | `\"http://localhost:8080\"`                                                 | Base URL at which GoSƐ is accessible  |\n| `GOSE_STATIC`                          | `\"./dist\"`                                                                | Directory of frontend assets (pre-compiled binaries of GoSƐ come with assets embedded into binary.) |\n| `GOSE_BUCKET`                          | `gose-uploads`                                                            | Name of S3 bucket                     |\n| `GOSE_ENDPOINT`                        | (without `http(s)://` prefix, but with port number)                       | Hostname:Port of S3 server            |\n| `GOSE_REGION`                          | `us-east-1`                                                               | Region of S3 server                   |\n| `GOSE_PATH_STYLE`                      | `false`                                                                   | Prepend bucket name to path           |\n| `GOSE_NO_SSL`                          | `false`                                                                   | Disable SSL encryption for S3         |\n| `GOSE_ACCESS_KEY`                      |                                                                           | S3 Access Key                         |\n| `GOSE_SECRET_KEY`                      |                                                                           | S3 Secret Key                         |\n| `GOSE_SETUP_BUCKET`                    | `true`                                                                    | Create S3 bucket if do not exists     |\n| `GOSE_SETUP_CORS`                      | `true` (if supported by S3 implementation)                                | Setup S3 bucket CORS rules            |\n| `GOSE_SETUP_LIFECYCLE`                 | `true` (if supported by S3 implementation)                                | Setup S3 bucket lifecycle rules       |\n| `GOSE_SETUP_ABORT_INCOMPLETE_UPLOADS`  | `31`                                                                      | Number of days after which incomplete uploads are cleaned-up (set to 0 to disable) |\n| `GOSE_MAX_UPLOAD_SIZE`                 | `1TB`                                                                     | Maximum upload size                   |\n| `GOSE_PART_SIZE`                       | `16MB`                                                                    | Part-size for multi-part uploads      |\n| `AWS_ACCESS_KEY_ID`                    |                                                                           | alias for `GOSE_ACCESS_KEY`           |\n| `AWS_SECRET_ACCESS_KEY`                |                                                                           | alias for `GOSE_SECRET_KEY`           |\n\nConfiguration of link shortener and notifiers must be done via a [configuration file](#file).\n\n## Author\n\nGoSƐ has been written by [Steffen Vogel](mailto:post@steffenvogel.de).\n\n## License\n\nGoSƐ is licensed under the [Apache 2.0 license](./LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstv0g%2Fgose","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstv0g%2Fgose","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstv0g%2Fgose/lists"}