{"id":20766998,"url":"https://github.com/binded/cloud-cache","last_synced_at":"2025-05-11T08:34:08.396Z","repository":{"id":50702391,"uuid":"67250419","full_name":"binded/cloud-cache","owner":"binded","description":"Node.js persistent caching module with pluggable backing store (local file system, Amazon S3, Google Drive / Cloud, PostgreSQL...) and streaming API","archived":false,"fork":false,"pushed_at":"2018-11-29T14:13:06.000Z","size":3401,"stargazers_count":70,"open_issues_count":2,"forks_count":7,"subscribers_count":11,"default_branch":"master","last_synced_at":"2024-11-07T05:11:43.590Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/binded.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-09-02T19:47:26.000Z","updated_at":"2024-01-04T16:07:18.000Z","dependencies_parsed_at":"2022-09-16T15:10:32.146Z","dependency_job_id":null,"html_url":"https://github.com/binded/cloud-cache","commit_stats":null,"previous_names":["blockai/cloud-cache"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binded%2Fcloud-cache","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binded%2Fcloud-cache/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binded%2Fcloud-cache/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binded%2Fcloud-cache/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/binded","download_url":"https://codeload.github.com/binded/cloud-cache/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225030735,"owners_count":17409940,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-17T11:27:10.786Z","updated_at":"2024-11-17T11:27:11.407Z","avatar_url":"https://github.com/binded.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# cloud-cache\n\n[![Build Status](https://travis-ci.org/blockai/cloud-cache.svg?branch=master)](https://travis-ci.org/blockai/cloud-cache)\n\nNode.js caching library with pluggable backing store via\n[abstract-blob-store](https://github.com/maxogden/abstract-blob-store).\n[Streaming support](#stream-api) makes it particularly useful for\ncaching larger values like resized/cropped images or transcoded videos.\n\n[![blob-store-compatible](https://raw.githubusercontent.com/maxogden/abstract-blob-store/master/badge.png)](https://github.com/maxogden/abstract-blob-store)\n\n\u003c!-- START doctoc generated TOC please keep comment here to allow auto update --\u003e\n\u003c!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --\u003e\n**Table of Contents**\n\n- [Features](#features)\n- [Install](#install)\n- [Usage](#usage)\n  - [Setting up the client](#setting-up-the-client)\n  - [Promise API](#promise-api)\n  - [Stream API](#stream-api)\n    - [Error Handling](#error-handling)\n  - [Errors](#errors)\n- [How it works](#how-it-works)\n- [Partial Writes / Durability](#partial-writes--durability)\n\n\u003c!-- END doctoc generated TOC please keep comment here to allow auto update --\u003e\n\n## Features\n\n- [Promise](#promise-api) and [Stream](#stream-api) based APIs\n- Supported backing stores:\n  - [AWS S3](https://github.com/jb55/s3-blob-store)\n  - [Google Cloud Storage](https://github.com/maxogden/google-cloud-storage)\n  - [Azure Storage](https://github.com/svnlto/azure-blob-store)\n  - [LevelDB](https://github.com/diasdavid/level-blob-store)\n  - [PostgreSQL](https://github.com/finnp/postgres-blob-store)\n  - [Local file system](https://github.com/mafintosh/fs-blob-store)\n  - [IPFS](https://github.com/ipfs/ipfs-blob-store)\n  - [etc.](https://github.com/maxogden/abstract-blob-store)\n- Supported data types:\n  - Buffer\n  - JSON types\n    - Number\n    - String\n    - Boolean\n    - Array\n    - Object\n\n## Install\n\n```bash\nnpm install --save cloud-cache\n```\n\nRequires Node v6+\n\n## Usage\n\nSee [./test](./test) directory for usage examples.\n\n### Setting up the client\n\n```javascript\nimport cloudCache from 'cloud-cache'\nconst cache = cloudCache(blobStore [, opts])\n```\n\n* `blobStore`: **blobStore** [abstract-blob-store](https://www.npmjs.com/package/abstract-blob-store) instance\n* `opts.keyPrefix`: **String** `cloudcache/` global key prefix that will be automatically prepended to all keys\n\n### Promise API\n\nAll methods return [promises](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise).\n\n**cache.get(key)** Get a key.\n\n* `key`: **String** the key to get\n\nThrows a `KeyNotExistsError` error if the key doesn't exist or value has\nexpired.\n\n```javascript\ncache.get('key')\n```\n\n**cache.set(key, value [, opts])** Stores a new value in the cache.\n\n* `key`: **String** the key to set\n* `value`: **Mixed** Buffer or any JSON compatible value.\n* `opts.ttl`: **Number**, `Infinity` Time to live: how long the data needs to be stored measured in `seconds`\n\n```javascript\ncache.set('foo', 'bar', { ttl: 60 * 60 })\n```\n\n**cache.del(key)** Delete a key.\n\n* `key`: **String** the key to delete\n\n```javascript\ncache.del('key')\n```\n\n**cache.getOrSet(key, getValueFn [, opts])** Returns cached value, storing and returning value on cache misses.\n\n* `key`: **String** the key to get\n* `getValueFn`: **Function** function to evaluate on cache misses\n* `opts`: **Object** same as `cache.set`\n* `opts.refresh`: **Boolean** `false` forces a cache miss\n\nThe arguments are the same as **cache.set**, except that `value` must be\na **function** or a **promise returning function** that evaluates / resolves\nto a valid **cache.set** value. The function will only be evaluated on cache misses.\n\n```javascript\ncache.getOrSet('google.com', () =\u003e (\n  fetch('http://google.com/').then(body =\u003e body.text())\n))\n```\n\n### Stream API\n\n**cache.getStream(key)**\n\n* `key`: **String** the key to read\n\nReturns a Readable Stream.\n\nEmits a `KeyNotExistsError` error if the key doesn't exist or value has\nexpired.\n\nAlias: `cache.gets(key)`\n\n```javascript\ncache.getStream('olalonde/avatar.png').pipe(req)\n```\n\n**cache.setStream(key [, opts])**\n\n* `key`: **String** the key to set\n* `opts`: **Object** same as `cache.set`\n\nReturns a Writable Stream.\n\nAlias: `cache.sets(key)`\n\n```javascript\nresizeImage('/tmp/avatar.png').pipe(cache.setStream('olalonde/avatar.png'))\n```\n\n**cache.getOrSetStream(key, getStreamFn [, opts])**\n\n* `key`: **String** the key to get\n* `getStreamFn`: **Function** Read Stream returning function that will\n    be called on cache misses.\n* `opts`: **Object** same as `cache.getOrSet`\n\nReturns a Readable Stream.\n\nImportant:\n\n- The stream returned by `getStreamFn` might not be cached if the\n    stream returned by `cache.getOrSetStream` not fully consumed (e.g. by piping it).\n- A `finish` event is fired to indicate that the stream was completely\n    saved to the cache.\n\n```javascript\ncache.getOrSetStream('olalonde/avatar.png', () =\u003e resizeImage('/tmp/avatar.png')).pipe(req)\n```\n\n#### Error Handling\n\nThe streams returned by cache may emit `error` events. We recommend\nusing [pipe() from the mississippi module](https://github.com/maxogden/mississippi#pipe)\nto avoid unhandled errors and make sure the cache stream closes properly\nif the destination has an error.\n\ne.g.:\n\n```javascript\nimport { pipe } from 'mississippi'\n// ...\npipe(cache.getOrSetStream('key', getReadStream), req, (err) =\u003e {\n  if (err) return next(err)\n})\n```\n\n### Errors\n\n- `CloudCacheError` this base class is inherited by the errors below\n- `KeyNotExistsError` thrown/emitted when trying to get a non existent / expired key. Exposes a `key` property\n\nThe error classes can be accessed through import or as a property on the cache object,\ne.g.:\n\n```javascript\nimport { CloudCacheError, KeyNotExistsError } from 'cloud-cache'\n// ...\ncache.CloudCacheError === CloudCacheError // true\ncache.KeyNotExistsError === KeyNotExistsError // true\nKeyNotExistsError instanceof CloudCacheError // true\n```\n\n## How it works\n\nCloud-cache encodes each cached value as a file stored on a storage\nprovider (S3, file system, etc.). The files start with a small JSON\nheader which contains metadata (e.g. `creation time, ttl, data type,\netc.`), followed by a newline character (`\\n`)  and finally, the actual\ncached value. Values are encoded as JSON, except for buffers or streams\nwhich are stored as raw bytes.\n\nThis means that cached values aren't very useful to applications which\nare unaware of the header.\n\nIf you are caching transformed images to S3 for example, you couldn't\nreference the S3 URL directly from an HTML image tag for example\n(because the browser wouldn't know it needs to ignore everything before\nthe first newline character).\n\nYou could however serve the images from a Node.js HTTP server and use\nthe stream API to stream the image from S3 (e.g.\n`cache.gets('olalonde/avatar.png').pipe(res)`).\n\nCloud-cache evicts expired values on read which means that expired\nvalues will remain stored as long as they are not read.\n\n## Partial Writes / Durability\n\nCloud-cache does not guarantee that **set** operations will be atomic\nand instead delegates that responsibility to the underlying store\nimplementation. If the underlying store doesn't guarantee atomic writes,\npartial writes can happen (e.g. if the process crashes in the middle of\na write). For example, `fs-blob-store` will\n[happily](https://github.com/mafintosh/fs-blob-store/pull/6) write half\nof a stream to the file system. `s3-blob-store`, on the other hand, will\nonly write a stream which has been fully consumed.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbinded%2Fcloud-cache","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbinded%2Fcloud-cache","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbinded%2Fcloud-cache/lists"}