{"id":13929092,"url":"https://github.com/loune/s3leveldown","last_synced_at":"2026-02-12T15:35:36.699Z","repository":{"id":17003879,"uuid":"80989346","full_name":"loune/s3leveldown","owner":"loune","description":"An implementation of LevelDOWN that uses Amazon S3. Turn your S3 bucket into a DB","archived":false,"fork":false,"pushed_at":"2025-10-15T08:46:36.000Z","size":290,"stargazers_count":62,"open_issues_count":2,"forks_count":12,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-10-15T23:37:37.014Z","etag":null,"topics":["leveldown","levelup","s3"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/loune.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-02-05T11:48:08.000Z","updated_at":"2025-10-15T08:46:37.000Z","dependencies_parsed_at":"2024-01-14T18:09:17.819Z","dependency_job_id":null,"html_url":"https://github.com/loune/s3leveldown","commit_stats":{"total_commits":27,"total_committers":6,"mean_commits":4.5,"dds":0.7037037037037037,"last_synced_commit":"edf855b06ea9b6c8f7eade2144ddc0b46f8756f4"},"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/loune/s3leveldown","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loune%2Fs3leveldown","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loune%2Fs3leveldown/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loune%2Fs3leveldown/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loune%2Fs3leveldown/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/loune","download_url":"https://codeload.github.com/loune/s3leveldown/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loune%2Fs3leveldown/sbom","scorecard":{"id":599793,"data":{"date":"2025-08-11","repo":{"name":"github.com/loune/s3leveldown","commit":"d8f23c2a111efd4f9463646c696b6f76126987b1"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.3,"checks":[{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/nodejs-test.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/nodejs-test.yml:22: update your workflow using https://app.stepsecurity.io/secureworkflow/loune/s3leveldown/nodejs-test.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/nodejs-test.yml:24: update your workflow using https://app.stepsecurity.io/secureworkflow/loune/s3leveldown/nodejs-test.yml/master?enable=pin","Info:   0 out of   2 GitHub-owned GitHubAction dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Code-Review","score":0,"reason":"Found 0/22 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 8 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":9,"reason":"1 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-v6h2-p8h4-qcjw"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-21T00:07:43.013Z","repository_id":17003879,"created_at":"2025-08-21T00:07:43.013Z","updated_at":"2025-08-21T00:07:43.013Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29370548,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-12T08:51:36.827Z","status":"ssl_error","status_checked_at":"2026-02-12T08:51:26.849Z","response_time":55,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["leveldown","levelup","s3"],"created_at":"2024-08-07T18:02:06.592Z","updated_at":"2026-02-12T15:35:36.670Z","avatar_url":"https://github.com/loune.png","language":"JavaScript","funding_links":[],"categories":["others","JavaScript"],"sub_categories":[],"readme":"# S3LevelDown\n\nAn [abstract-leveldown](https://github.com/Level/abstract-leveldown) compliant implementation of [LevelDOWN](https://github.com/Level/leveldown) that uses [Amazon S3](https://aws.amazon.com/s3/) as a backing store. S3 is actually a giant key-value store on the cloud, even though it is marketed as a file store. Use this database with the [LevelUP](https://github.com/Level/levelup/) API.\n\nTo use this optimally, please read \"Performance considerations\" and \"Warning about concurrency\" sections below.\n\nYou could also use this as an alternative API to read/write S3. The API simpler to use when compared to the AWS SDK!\n\n## Installation\n\nInstall `s3leveldown` and peer dependencies `levelup` and `@aws-sdk/client-s3` with `yarn` or `npm`.\n\n```bash\n$ npm install s3leveldown @aws-sdk/client-s3 levelup\n```\n\n## Documentation\n\nSee the [LevelUP API](https://github.com/Level/levelup#api) for high level usage.\n\n### `new S3LevelDown(location [, s3])`\n\nConstructor of `s3leveldown` backing store. Use with `levelup`.\n\nArguments:\n* `location` name of the S3 bucket with optional sub-folder. Example `mybucket` or `mybucket/folder`.\n* `s3` Optional `S3` client from `aws-sdk`. A default client will be used if not specified.\n\n## Example\n\nPlease refer to the [AWS SDK docs to set up your API credentials](http://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/setting-credentials-node.html) before using.\n\n### Using Promises\n\n```js\nconst levelup = require('levelup');\nconst S3LevelDown = require('s3leveldown');\n\n(async () =\u003e {\n  // create DB\n  const db = levelup(new S3LevelDown('mybucket'));\n\n  // put items\n  await db.batch()\n    .put('name', 'Pikachu')\n    .put('dob', 'February 27, 1996')\n    .put('occupation', 'Pokemon')\n    .write();\n  \n  // read items\n  await db.createReadStream()\n    .on('data', data =\u003e { console.log('data', `${data.key.toString()}=${data.value.toString()}`); })\n    .on('close', () =\u003e { console.log('done!') });\n})();\n```\n\n### Using Callbacks\n\n```js\nconst levelup = require('levelup');\nconst S3LevelDown = require('s3leveldown');\n\nconst db = levelup(new S3LevelDown('my_bucket'));\n\ndb.batch()\n  .put('name', 'Pikachu')\n  .put('dob', 'February 27, 1996')\n  .put('occupation', 'Pokemon')\n  .write(function () { \n    db.readStream()\n      .on('data', console.log)\n      .on('close', function () { console.log('Pika pi!') })\n  });\n```\n\n### Example with min.io\n\nYou could also use s3leveldown with S3 compatible servers such as [MinIO](https://github.com/minio/minio).\n\n```js\nconst levelup = require('levelup');\nconst S3LevelDown = require('s3leveldown');\nconst AWS = require('aws-sdk');\n\nconst s3 = new AWS.S3({\n  apiVersion: '2006-03-01',\n  accessKeyId: 'YOUR-ACCESSKEYID',\n  secretAccessKey: 'YOUR-SECRETACCESSKEY',\n  endpoint: 'http://127.0.0.1:9000',\n  s3ForcePathStyle: true,\n  signatureVersion: 'v4'\n});\n\nconst db = levelup(new S3LevelDown('my_bucket', s3));\n```\n\n### [Example with PouchDB](./examples/pouchdb)\n\n## Sub folders\n\nYou can create your Level DB in a sub-folder in your S3 bucket, just use `my_bucket/sub_folder` when passing the location.\n\n## Performance considerations\n\nThere are a few performance caveats due to the limited API provided by the AWS S3 API:\n\n* When iterating, getting values is expensive. A seperate S3 API call is made to get the value of each key. If you don't need the value, pass `{ values: false }` in the options. Each S3 API call can return 1000 keys, so if there are 3000 results, 3 calls are made to list the keys, and if getting values as well, another 3000 API calls are made.\n\n* Avoid iterating large datasets when passing `{ reverse: true }`. Since the S3 API call do not allow retrieving keys in reverse order, the entire result set needs to be stored in memory and reversed. If your database is large ( \u003e5k keys ), be sure to provide start (`gt`, `gte`) and end (`lt`, `lte`), or the entire database will need to be fetched.\n\n* By default when iterating, 1000 keys will be returned. If you only want 10 keys for example, set `{ limit: 10 }` and the S3 API call will only request 10 keys. Note that if you have `{ reverse: true }`, this optimisation does not apply as we need to fetch everything from start to end and reverse it in memory. To override the default number of keys to return in a single API call,  you can set the ` s3ListObjectMaxKeys` option when creating the iterator. The maximum accepted by the S3 API is 1000.\n\n* Specify the AWS region of the bucket to improve performance, by calling `AWS.config.update({ region: 'ap-southeast-2' });` replace `ap-southeast-2` with your region.\n\n## Warning about concurrency\n\nIndividual operations (`put` `get` `del`) are atomic as guaranteed by S3, but the implementation of `batch` is not atomic. Two concurrent batch calls will have their operations interwoven. Don't use any plugins which require this to be atomic or you will end up with your database corrupted! However, if you can guarantee that only one process will write the S3 bucket at a time, then this should not be an issue. Ideally, you want to avoid race conditions where two processes are writing to the same key at the same time. In those cases the last write wins.\n\nIterator snapshots are not supported. When iterating through a list of keys and values, you may get the changes, similar to dirty reads.\n\n## Tests and debug\n\nS3LevelDown uses [debug](https://github.com/visionmedia/debug). To see debug message set the environment variable `DEBUG=S3LevelDown`.\n\nTo run the test suite, you need to set a S3 bucket to the environment variable `S3_TEST_BUCKET`. Also be sure to [set your AWS credentials](http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html)\n\n```bash\n$ S3_TEST_BUCKET=my-test-bucket npm run test\n```\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Floune%2Fs3leveldown","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Floune%2Fs3leveldown","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Floune%2Fs3leveldown/lists"}