{"id":13681529,"url":"https://github.com/PatrickJS/redis-dataloader","last_synced_at":"2025-04-30T03:31:46.065Z","repository":{"id":40243235,"uuid":"75498879","full_name":"PatrickJS/redis-dataloader","owner":"PatrickJS","description":"Batching and Caching layer using Redis as the Caching layer","archived":false,"fork":false,"pushed_at":"2022-12-30T21:02:56.000Z","size":196,"stargazers_count":89,"open_issues_count":8,"forks_count":20,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-11-07T04:11:51.518Z","etag":null,"topics":["batching","dataloader","datastore","facebook-dataloader","promise","redis-cache","redis-dataloader"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PatrickJS.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-12-03T20:36:17.000Z","updated_at":"2024-03-06T23:16:55.000Z","dependencies_parsed_at":"2023-01-31T16:45:30.771Z","dependency_job_id":null,"html_url":"https://github.com/PatrickJS/redis-dataloader","commit_stats":null,"previous_names":["dubfriend/redis-dataloader"],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PatrickJS%2Fredis-dataloader","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PatrickJS%2Fredis-dataloader/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PatrickJS%2Fredis-dataloader/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PatrickJS%2Fredis-dataloader/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PatrickJS","download_url":"https://codeload.github.com/PatrickJS/redis-dataloader/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223459729,"owners_count":17148713,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["batching","dataloader","datastore","facebook-dataloader","promise","redis-cache","redis-dataloader"],"created_at":"2024-08-02T13:01:31.951Z","updated_at":"2024-11-12T00:32:24.142Z","avatar_url":"https://github.com/PatrickJS.png","language":"JavaScript","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"readme":"# Redis Dataloader\n\nBatching and Caching layer using Redis as the Caching layer.\nRedis Dataloader wraps [Facebook Dataloader](https://github.com/facebook/dataloader),\nadding Redis as a caching layer.\n\n`npm install redis-dataloader`  \nor  \n`yarn add redis-dataloader`\n\n## Example\n\n```javascript\nconst redisClient = require('redis').createClient();\n// the \"ioredis\" module is also supported\n// const Redis = require('ioredis');\n// const redisClient = new Redis();\nconst DataLoader = require('dataloader');\nconst RedisDataLoader = require('redis-dataloader')({ redis: redisClient });\n\nconst loader = new RedisDataLoader(\n    // set a prefix for the keys stored in redis. This way you can avoid key\n    // collisions for different data-sets in your redis instance.\n    'redis-key-prefix',\n    // create a regular dataloader. This should always be set with caching disabled.\n    new DataLoader(myBatchLoadFunction, { cache: false }),\n    // The options here are the same as the regular dataloader options, with\n    // the additional option \"expire\"\n    {\n        // caching here is a local in memory cache. Caching is always done\n        // to redis.\n        cache: true,\n        // if set redis keys will be set to expire after this many seconds\n        // this may be useful as a fallback for a redis cache.\n        expire: 60,\n        // can include a custom serialization and deserialization for\n        // storage in redis.\n        serialize: date =\u003e date.getTime(),\n        deserialize: timestamp =\u003e new Date(timestamp),\n        // Set this to true to return Buffer objects to the deserialize function\n        // when using the ioredis driver.\n        buffer: false\n    }\n);\n\n// load an individual item by its key\nloader.load(5).then(resp =\u003e console.log(resp));\n\n//clear an individiaul item from the local and redis cache.\nloader.clear(5).then(() =\u003e {})\n```\n\n## API Documentation\n\nIn general, RedisDataLoader has the same API as the Facebook Dataloader Api,\nwith a few differences. Read through the [Facebook Dataloader documentation](https://github.com/facebook/dataloader) and then note the differences mentioned here.\n\n- `clear` returns a promise (waits until redis succeeds at deleting the key). Facebook Dataloader's `clear` method is synchronous.\n- `clearAll` is not available (redis does not have an efficient way to do this?)\n- `prime` will always overwrite the cache. Facebook Dataloader will only write to\nits cache if a value is not already present. Prime is asyncronous and returns a Promise.\n- dataloader results must be either `null` or a JSON object.\n- two functions: `clearLocal(key)` and `clearAllLocal()` allow you to clear the local cache only.\n\n### Instantiation\n\n#### Dependency inject a Redis Connection\n\n```javascript\nconst redis = require('redis').createClient();\nconst RedisDataLoader = require('redis-dataloader')({ redis: redis });\n```\n\n#### Create a new Dataloader.\n\nEach Dataloader holds its own local in memory cache (Same as Facebook Dataloader),\nand additionally caches to your Redis instance.\n\n```javascript\nconst loader = new RedisDataLoader('\u003credis key prefix\u003e', '\u003cFacebook Dataloader\u003e', '\u003cOptions\u003e');\n```\n\n##### Redis Key Prefix\n\nSpecify a Prefix that will be appended to each key when storing in Redis.\n\nSo for example if your prefix is \"bar\" and you call `loader.load('foo')`, this key\nwill be stored in Redis as **bar:foo**\n\n##### Facebook Dataloader\n\nA regular Facebook Dataloader is passed in as the second parameter. It will be\nused to fetch data from your underlying datastore (mongo, sql, whatever).\nIt is very important to **disable the cache** on this dataloader. Redis dataloader\nwill already do local in memory caching (unless you disable it).\n\n##### Options\n\nAll the options available to Facebook Dataloader can be passed in here. An\nadditional option called **expire** is also available, and will set a ttl in seconds\non all keys set in redis if this option is passed.\n\nThe `cacheKeyFn` will default to serialize objects and arrays using [json-stable-stringify](https://github.com/substack/json-stable-stringify) and allow all other values to pass through unchanged.\n\n`buffer` will pass a Buffer object to the deserialize function rather than a string.\nIf not using the ioredis driver this will throw an error at instation.\n\n### Caching\n\nThe purpose of Redis Dataloader is to provide a caching layer in redis on top\nof the Facebook Dataloader. Facebook's Dataloader provides a local in memory cache.\nThis may be ok for short lived per-request caches, but may not be sufficient if\nyou need a long lived cache and/or you have multiple webservers that need to share\ndata.\n\nRedis Dataloader will additionally use the same local cache that Facebook Dataloader\nprovides. It will first check the local cache, then check the redis cache, before\nfinally checking your underlying datastore. This pattern may be desirable if for\nexample you create a new DataLoader for each request. If your dataloader is long-lived\nyou may want to disable to the local cache, and just rely on the redis cache instead\n\n```javascript\nconst loader = new RedisDataLoader('prefix', new DataLoader(), { cache: false });\n```\n\n## Development\n\n1. Install Dependencies `npm install`\n1. Start Redis `docker-compose stop \u0026\u0026 docker-compose rm \u0026\u0026 docker-compose build \u0026\u0026 docker-compose up -d`\n1. Run Tests `npm test`\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPatrickJS%2Fredis-dataloader","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FPatrickJS%2Fredis-dataloader","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPatrickJS%2Fredis-dataloader/lists"}