{"id":13587752,"url":"https://github.com/anthonybudd/S4","last_synced_at":"2025-04-07T22:31:34.400Z","repository":{"id":130731745,"uuid":"301392514","full_name":"anthonybudd/S4","owner":"anthonybudd","description":"S4 is 100% S3 compatible storage, accessed through Tor and distributed using IPFS.","archived":false,"fork":false,"pushed_at":"2021-03-03T20:08:07.000Z","size":2169,"stargazers_count":93,"open_issues_count":2,"forks_count":9,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-02-14T21:56:22.917Z","etag":null,"topics":["data","docker","ipfs","object-storage","s3","s4","storage"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/anthonybudd.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-10-05T11:57:42.000Z","updated_at":"2024-08-01T16:33:43.805Z","dependencies_parsed_at":null,"dependency_job_id":"b6218221-fb65-4d0c-b350-315e44f17e34","html_url":"https://github.com/anthonybudd/S4","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anthonybudd%2FS4","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anthonybudd%2FS4/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anthonybudd%2FS4/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anthonybudd%2FS4/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/anthonybudd","download_url":"https://codeload.github.com/anthonybudd/S4/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247740846,"owners_count":20988280,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data","docker","ipfs","object-storage","s3","s4","storage"],"created_at":"2024-08-01T15:06:20.811Z","updated_at":"2025-04-07T22:31:32.489Z","avatar_url":"https://github.com/anthonybudd.png","language":"Shell","readme":"\u003cp align=\"center\"\u003e\n\u003cimg width=\"192\" src=\"https://raw.githubusercontent.com/anthonybudd/s4/master/docs/img/s4-logo-dark.svg\" alt=\"S4\"\u003e\n\u003c/p\u003e\n\n\n# S4\nS4 is 100% compatible AWS S3 storage, accessed through Tor and distributed using IPFS.\n\nTor acts as a DNS and *hides* the physical location of the S4 server. [IPFS](https://github.com/ipfs/ipfs) acts as a CDN and will make your data permanently accessible and is impossible to take offline once it has been published. A [sidecar docker container](https://github.com/anthonybudd/s4-client) is provided to seamlessly proxy requests from your existing S3 code over Tor to S4.\n\n\n\u003cp align=\"center\"\u003e\n\u003ca href=\"https://youtu.be/RMNjpAmCvcQ\"\u003e\n\u003cimg width=\"450\" src=\"https://raw.githubusercontent.com/anthonybudd/s4/master/docs/img/s4-youtube-embed.png\" alt=\"S4 Video Tutorial\"\u003e\n\u003c/a\u003e\n\u003c/p\u003e\n\n## Create S4 server\nYou can set-up your own local S4 server with just 4 commands.\n```sh\n$ git clone git@github.com:anthonybudd/S4.git \u0026\u0026 cd S4/\n\n$ cp .env.example .env\n\n$ docker run -it --rm -v $(pwd)/data/tor:/web anthonybudd/s4-tor-proxy generate ^s4\n\n$ docker-compose up\n```\n## Node.js AWS-SDK Example\nOnce you have created your local S4 server, you can test it using the included [example](https://github.com/anthonybudd/S4/blob/master/example/src/index.js). This demo shows the current Node.js AWS-SDK working with the [sidecar docker container](https://github.com/anthonybudd/s4-client) to proxy the AWS S3 requests over Tor back to your local S4 server.\n\n```sh\n$ cd example/\n\n$ nano .env\n\n$ docker-compose up -d\n\n$ docker exec -ti my-app node src/index.js\n```\n\n## Architecture\n\u003cp  align=\"center\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/anthonybudd/s4/master/docs/img/s4-diagram.png\"  alt=\"S4\"\u003e\n\u003c/p\u003e\n\nTor does not just provide anonymity, Tor acts as a DNS. S4 uses Tor hidden services, data is uploaded into an S4 bucket over Tor. This also means you can create a free .onion domain name without needing any 3rd-parties. A [sidecar container](https://github.com/anthonybudd/s4-client) is provided to seamlessly proxy requests from your existing S3 code over Tor to S4.\n\nIPFS acts as a CDN. Each S4 bucket is published to IPFS under a separate key allowing you to address your files using any existing IPFS HTTP gateway like this [https://ipfs.io/ipns/[BUCKET-KEY]/image.jpg](http://ipfs.io/ipns/QmTvcZ719MEiYjhn2RxwtiYGkJr6ea4xnSGqdxj5vRii8k/flag.jpg). This link will never change or go offline, the more people who request it, the faster the data will be returned to the user.\n\n\n## Build a Node\n\u003cp  align=\"center\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/anthonybudd/s4/master/docs/img/s4-node.png\" width=\"300\" alt=\"S4 node\"\u003e\n\u003c/p\u003e\nS4 is designed to democratise object-storage technology so anyone can have S3-like data storage without needing to depend on giant cloud tech providers. It was specifically built to run on a Raspberry Pi in production. A full blog post has been provided with in-depth instructions on how to build your own S4 node. A basic overview is provided below.\n\n- Install Ubuntu 20.04\n\n-  `$ git clone git@github.com:anthonybudd/S4.git`\n\n-  `$ cd S4`\n\n-  `$ ./install` \n\n-  `$ sudo mount /dev/XXX /mnt/ssd`\n\n-  `$ sed -i \"\" \"s#S4_ROOT=.*#S4_ROOT=/mnt/ssd#\" .env`\n\n-  `$ sed -i \"\" \"s#ACCESS_KEY=.*#ACCESS_KEY=$(openssl rand -hex 10)#\" .env`\n\n-  `$ sed -i \"\" \"s#SECRET_KEY=.*#SECRET_KEY=$(openssl rand -hex 20)#\" .env`\n\n-  `$ sudo reboot`\n\n-  `$ docker run -it --rm -v /mnt/ssd/tor:/web anthonybudd/s4-tor-proxy:arm64v8 generate ^s4`\n\n-  `$ docker-compose up`\n\n","funding_links":[],"categories":["Shell","Distributed computing projects"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanthonybudd%2FS4","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanthonybudd%2FS4","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanthonybudd%2FS4/lists"}