{"id":37211125,"url":"https://github.com/numbersprotocol/go-ds-s3","last_synced_at":"2026-01-15T00:02:43.580Z","repository":{"id":57653675,"uuid":"452977491","full_name":"numbersprotocol/go-ds-s3","owner":"numbersprotocol","description":"An s3 datastore implementation","archived":false,"fork":true,"pushed_at":"2022-01-28T07:55:39.000Z","size":1092,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-06-20T10:15:30.058Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"ipfs/go-ds-s3","license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/numbersprotocol.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-01-28T07:30:58.000Z","updated_at":"2022-01-24T14:49:47.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/numbersprotocol/go-ds-s3","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/numbersprotocol/go-ds-s3","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/numbersprotocol%2Fgo-ds-s3","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/numbersprotocol%2Fgo-ds-s3/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/numbersprotocol%2Fgo-ds-s3/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/numbersprotocol%2Fgo-ds-s3/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/numbersprotocol","download_url":"https://codeload.github.com/numbersprotocol/go-ds-s3/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/numbersprotocol%2Fgo-ds-s3/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28439606,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T22:37:52.437Z","status":"ssl_error","status_checked_at":"2026-01-14T22:37:31.496Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-15T00:02:42.763Z","updated_at":"2026-01-15T00:02:43.570Z","avatar_url":"https://github.com/numbersprotocol.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# S3 Datastore Implementation\n\nThis is an implementation of the datastore interface backed by amazon s3.\n\n**NOTE:** Plugins only work on Linux and MacOS at the moment. You can track the progress of this issue here: https://github.com/golang/go/issues/19282\n\n## Building and Installing\n\nYou must build the plugin with the *exact* version of go used to build the go-ipfs binary you will use it with. You can find the go version for go-ipfs builds from dist.ipfs.io in the build-info file, e.g. https://dist.ipfs.io/go-ipfs/v0.4.22/build-info or by running `ipfs version --all`.\n\nIn addition to needing the exact version of go, you need to build the correct version of this plugin.\n\n* To build against a released version of go-ipfs, checkout the `release/v$VERSION` branch and build.\n* To build against a custom (local) build of go-ipfs, run `make IPFS_VERSION=/path/to/go-ipfs/source`.\n\nYou can then install it into your local IPFS repo by running `make install`.\n\n## Bundling\n\nAs go plugins can be finicky to correctly compile and install, you may want to consider bundling this plugin and re-building go-ipfs. If you do it this way, you won't need to install the `.so` file in your local repo and you won't need to worry about getting all the versions to match up.\n\n```bash\n# We use go modules for everything.\n\u003e export GO111MODULE=on\n\n# Clone go-ipfs.\n\u003e git clone https://github.com/ipfs/go-ipfs\n\u003e cd go-ipfs\n\n# Pull in the datastore plugin (you can specify a version other than latest if you'd like).\n\u003e go get github.com/ipfs/go-ds-s3/plugin@latest\n\n# Add the plugin to the preload list.\n\u003e echo \"s3ds github.com/ipfs/go-ds-s3/plugin 0\" \u003e\u003e plugin/loader/preload_list\n\n# ( this first pass will fail ) Try to build go-ipfs with the plugin\n\u003e make build\n\n# Update the deptree\n\u003e go mod tidy\n\n# Now rebuild go-ipfs with the plugin\n\u003e make build\n\n# (Optionally) install go-ipfs\n\u003e make install\n```\n\n## Detailed Installation\n\nFor a brand new ipfs instance (no data stored yet):\n\n1. Copy s3plugin.so $IPFS_DIR/plugins/go-ds-s3.so (or run `make install` if you are installing locally).\n2. Run `ipfs init`.\n3. Edit $IPFS_DIR/config to include s3 details (see Configuration below).\n4. Overwrite `$IPFS_DIR/datastore_spec` as specified below (*Don't do this on an instance with existing data - it will be lost*).\n\n### Configuration\n\nThe config file should include the following:\n```json\n{\n  \"Datastore\": {\n  ...\n\n    \"Spec\": {\n      \"mounts\": [\n        {\n          \"child\": {\n            \"type\": \"s3ds\",\n            \"region\": \"us-east-1\",\n            \"bucket\": \"$bucketname\",\n            \"rootDirectory\": \"$bucketsubdirectory\",\n            \"accessKey\": \"\",\n            \"secretKey\": \"\"\n          },\n          \"mountpoint\": \"/blocks\",\n          \"prefix\": \"s3.datastore\",\n          \"type\": \"measure\"\n        },\n```\n\nIf the access and secret key are blank they will be loaded from the usual ~/.aws/.\nIf you are on another S3 compatible provider, e.g. Linode, then your config should be:\n\n```json\n{\n  \"Datastore\": {\n  ...\n\n    \"Spec\": {\n      \"mounts\": [\n        {\n          \"child\": {\n            \"type\": \"s3ds\",\n            \"region\": \"us-east-1\",\n            \"bucket\": \"$bucketname\",\n            \"rootDirectory\": \"$bucketsubdirectory\",\n            \"regionEndpoint\": \"us-east-1.linodeobjects.com\",\n            \"accessKey\": \"\",\n            \"secretKey\": \"\"\n          },\n          \"mountpoint\": \"/blocks\",\n          \"prefix\": \"s3.datastore\",\n          \"type\": \"measure\"\n        },\n```\n\nIf you are configuring a brand new ipfs instance without any data, you can overwrite the datastore_spec file with:\n\n```\n{\"mounts\":[{\"bucket\":\"$bucketname\",\"mountpoint\":\"/blocks\",\"region\":\"us-east-1\",\"rootDirectory\":\"$bucketsubdirectory\"},{\"mountpoint\":\"/\",\"path\":\"datastore\",\"type\":\"levelds\"}],\"type\":\"mount\"}\n```\n\nOtherwise, you need to do a datastore migration.\n\n## Contribute\n\nFeel free to join in. All welcome. Open an [issue](https://github.com/ipfs/go-ipfs-example-plugin/issues)!\n\nThis repository falls under the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).\n\n### Want to hack on IPFS?\n\n[![](https://cdn.rawgit.com/jbenet/contribute-ipfs-gif/master/img/contribute.gif)](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md)\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnumbersprotocol%2Fgo-ds-s3","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnumbersprotocol%2Fgo-ds-s3","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnumbersprotocol%2Fgo-ds-s3/lists"}