Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/microlinkhq/keyv-s3
Amazon S3 storage adapter for Keyv.
https://github.com/microlinkhq/keyv-s3
keyv s3 s3-bucket
Last synced: about 2 months ago
JSON representation
Amazon S3 storage adapter for Keyv.
- Host: GitHub
- URL: https://github.com/microlinkhq/keyv-s3
- Owner: microlinkhq
- License: mit
- Created: 2020-08-14T14:56:41.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2024-05-07T15:10:22.000Z (8 months ago)
- Last Synced: 2024-10-30T08:03:47.639Z (2 months ago)
- Topics: keyv, s3, s3-bucket
- Language: JavaScript
- Homepage:
- Size: 150 KB
- Stars: 9
- Watchers: 1
- Forks: 5
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE.md
Awesome Lists containing this project
README
[](https://coveralls.io/github/microlinkhq/keyv-s3)
[](https://www.npmjs.org/package/keyv-s3)
> [Amazon S3](https://aws.amazon.com/s3) storage adapter for [Keyv](https://github.com/lukechilds/keyv).
## Motivation
[Microlink API](https://microlink.io/api) has a built-in cache layer for speed up consecutive API calls, caching the response based in a configurable [ttl](https://microlink.io/docs/api/parameters/ttl). You can read more about how it works in our [blog post](https://microlink.io/blog/edge-cdn/).
Until now, we delegate into [Redis](https://github.com/lukechilds/keyv-redis) for serving the cache layer because it's fast; However, it's also expensive when you have a certain size.
The service is serving [more than 8 millions API calls per month](http://analytics.microlink.io), where every request is creating a cache entry; That means running out of space could be done relatively fast.
We wanted a cache layer that met the following premises:
- Infinite space.
- Failover tolerant.
- Cheap at scale.
Knowing what we wanted, and keeping in mind [Microlink API](https://microlink.io/api) response payload is JSON, we found [Amazon S3](https://aws.amazon.com/s3) bucket excellent storage.
## Install
```bash
$ npm install @aws-sdk/client-s3 keyv-s3 --save
```
## Usage
First, you need to create a S3 bucket to be used as you cache layer
You need to setup a properly bucket name to be possible access via HTTP.
For doing that, we setup a CNAME over the bucket name, in our case we use [CloudFlare](https://www.cloudflare.com).
That's a necessary step because querying via S3 API is slower than query directly to the bucket.
Also, CloudFlare gives us an extra perfomance boost since it's on top of the Bucket domain associated, caching successive calls and save your S3 quota.
In case you are using CloudFlare as well, we have the following page rule associated:
Now you can connect with your S3 caching layer from code:
```js
const KeyvS3 = require('keyv-s3')
const keyvS3 = new KeyvS3({
region: 'us-east-1',
namespace: 'c.microlink.io',
accessKeyId: process.env.S3_ACCESS_KEY_ID,
secretAccessKey: process.env.S3_SECRET_ACCESS_KEY
})
```
You can interact with `keyvS3` using [keyv instance methods](https://keyv.js.org).
## Backblaze B2 support
For using [Backblaze B2](https://www.backblaze.com/b2/cloud-storage.html) as provider, you have to provide `s3client` with Blackbaze B2 endpoint.
Also, since Backblaze B2 buckets name doesn't allow a dot, you should to specify the `hostname` to be used for mapping the bucket:
```js
const keyvS3 = new KeyvS3({
region: 'us-east-1',
namespace: 'c-microlink-io',
hostname: 'c.microlink.io',
s3client: new S3Client({
region: process.env.B2_REGION,
endpoint: `https://s3.${process.env.B2_REGION}.backblazeb2.com`,
credentials: {
accessKeyId: process.env.B2_ACCESS_KEY_ID,
secretAccessKey: process.env.B2_SECRET_ACCESS_KEY
}
})
})
```
## License
**keyv-s3** © [microlink.io](https://microlink.io), released under the [MIT](https://github.com/microlinkhq/keyv-s3/blob/master/LICENSE.md) License.
Authored and maintained by [Kiko Beats](https://kikobeats.com) with help from [contributors](https://github.com/microlinkhq/keyv-s3/contributors).
> [microlink.io](https://microlink.io) · GitHub [microlink.io](https://github.com/microlinkhq) · Twitter [@microlinkhq](https://twitter.com/microlinkhq)