{"id":19926612,"url":"https://github.com/microlinkhq/keyv-s3","last_synced_at":"2025-05-03T08:32:24.265Z","repository":{"id":37929246,"uuid":"287557919","full_name":"microlinkhq/keyv-s3","owner":"microlinkhq","description":"Amazon S3 storage adapter for Keyv.","archived":false,"fork":false,"pushed_at":"2024-05-07T15:10:22.000Z","size":154,"stargazers_count":9,"open_issues_count":0,"forks_count":5,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-10-30T08:03:47.639Z","etag":null,"topics":["keyv","s3","s3-bucket"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/microlinkhq.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-08-14T14:56:41.000Z","updated_at":"2024-08-02T17:11:08.000Z","dependencies_parsed_at":"2022-08-25T22:22:54.531Z","dependency_job_id":"42948143-90a8-44e5-8006-e82503654430","html_url":"https://github.com/microlinkhq/keyv-s3","commit_stats":{"total_commits":131,"total_committers":4,"mean_commits":32.75,"dds":0.05343511450381677,"last_synced_commit":"cffb83afe10199f948dcea992a943d6b35f6c689"},"previous_names":[],"tags_count":53,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microlinkhq%2Fkeyv-s3","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microlinkhq%2Fkeyv-s3/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microlinkhq%2Fkeyv-s3/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microlinkhq%2Fkeyv-s3/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/microlinkhq","download_url":"https://codeload.github.com/microlinkhq/keyv-s3/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224358040,"owners_count":17298038,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["keyv","s3","s3-bucket"],"created_at":"2024-11-12T22:29:54.439Z","updated_at":"2025-05-03T08:32:24.257Z","avatar_url":"https://github.com/microlinkhq.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/microlinkhq/cdn/raw/master/dist/logo/banner.png#gh-light-mode-only\" alt=\"microlink logo\"\u003e\n  \u003cimg src=\"https://github.com/microlinkhq/cdn/raw/master/dist/logo/banner-dark.png#gh-dark-mode-only\" alt=\"microlink logo\"\u003e\n  \u003cbr\u003e\n  \u003cbr\u003e\n\u003c/div\u003e\n\n![Last version](https://img.shields.io/github/tag/microlinkhq/keyv-s3.svg?style=flat-square)\n[![Coverage Status](https://img.shields.io/coveralls/microlinkhq/keyv-s3.svg?style=flat-square)](https://coveralls.io/github/microlinkhq/keyv-s3)\n[![NPM Status](https://img.shields.io/npm/dm/keyv-s3.svg?style=flat-square)](https://www.npmjs.org/package/keyv-s3)\n\n\u003e [Amazon S3](https://aws.amazon.com/s3) storage adapter for [Keyv](https://github.com/lukechilds/keyv).\n\n## Motivation\n\n[Microlink API](https://microlink.io/api) has a built-in cache layer for speed up consecutive API calls, caching the response based in a configurable [ttl](https://microlink.io/docs/api/parameters/ttl). You can read more about how it works in our [blog post](https://microlink.io/blog/edge-cdn/).\n\nUntil now, we delegate into [Redis](https://github.com/lukechilds/keyv-redis) for serving the cache layer because it's fast; However, it's also expensive when you have a certain size.\n\nThe service is serving [more than 8 millions API calls per month](http://analytics.microlink.io), where every request is creating a cache entry; That means running out of space could be done relatively fast.\n\nWe wanted a cache layer that met the following premises:\n\n- Infinite space.\n- Failover tolerant.\n- Cheap at scale.\n\nKnowing what we wanted, and keeping in mind [Microlink API](https://microlink.io/api) response payload is JSON, we found [Amazon S3](https://aws.amazon.com/s3) bucket excellent storage.\n\n## Install\n\n```bash\n$ npm install @aws-sdk/client-s3 keyv-s3 --save\n```\n\n## Usage\n\nFirst, you need to create a S3 bucket to be used as you cache layer\n\n![](https://i.imgur.com/YkslBcd.png)\n\nYou need to setup a properly bucket name to be possible access via HTTP.\n\nFor doing that, we setup a CNAME over the bucket name, in our case we use [CloudFlare](https://www.cloudflare.com).\n\n![](https://i.imgur.com/GpCxX1G.png)\n\nThat's a necessary step because querying via S3 API is slower than query directly to the bucket.\n\nAlso, CloudFlare gives us an extra perfomance boost since it's on top of the Bucket domain associated, caching successive calls and save your S3 quota.\n\nIn case you are using CloudFlare as well, we have the following page rule associated:\n\n![](https://i.imgur.com/dANGotv.png)\n\nNow you can connect with your S3 caching layer from code:\n\n```js\nconst KeyvS3 = require('keyv-s3')\n\nconst keyvS3 = new KeyvS3({\n  region: 'us-east-1',\n  namespace: 'c.microlink.io',\n  accessKeyId: process.env.S3_ACCESS_KEY_ID,\n  secretAccessKey: process.env.S3_SECRET_ACCESS_KEY\n})\n```\n\nYou can interact with `keyvS3` using [keyv instance methods](https://keyv.js.org).\n\n## Backblaze B2 support\n\nFor using [Backblaze B2](https://www.backblaze.com/b2/cloud-storage.html) as provider, you have to provide `s3client` with Blackbaze B2 endpoint.\n\nAlso, since Backblaze B2 buckets name doesn't allow a dot, you should to specify the `hostname` to be used for mapping the bucket:\n\n```js\nconst keyvS3 = new KeyvS3({\n  region: 'us-east-1',\n  namespace: 'c-microlink-io',\n  hostname: 'c.microlink.io',\n  s3client: new S3Client({\n    region: process.env.B2_REGION,\n    endpoint: `https://s3.${process.env.B2_REGION}.backblazeb2.com`,\n    credentials: {\n      accessKeyId: process.env.B2_ACCESS_KEY_ID,\n      secretAccessKey: process.env.B2_SECRET_ACCESS_KEY\n    }\n  })\n})\n```\n\n## License\n\n**keyv-s3** © [microlink.io](https://microlink.io), released under the [MIT](https://github.com/microlinkhq/keyv-s3/blob/master/LICENSE.md) License.\u003cbr\u003e\nAuthored and maintained by [Kiko Beats](https://kikobeats.com) with help from [contributors](https://github.com/microlinkhq/keyv-s3/contributors).\n\n\u003e [microlink.io](https://microlink.io) · GitHub [microlink.io](https://github.com/microlinkhq) · X [@microlinkhq](https://x.com/microlinkhq)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmicrolinkhq%2Fkeyv-s3","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmicrolinkhq%2Fkeyv-s3","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmicrolinkhq%2Fkeyv-s3/lists"}