{"id":15561267,"url":"https://github.com/bobbui/smart-redis","last_synced_at":"2025-04-09T16:51:50.155Z","repository":{"id":41797177,"uuid":"168261275","full_name":"bobbui/smart-redis","owner":"bobbui","description":"Redis cache which automatically build and refresh itself.","archived":false,"fork":false,"pushed_at":"2022-04-28T19:27:05.000Z","size":139,"stargazers_count":10,"open_issues_count":7,"forks_count":1,"subscribers_count":0,"default_branch":"master","last_synced_at":"2024-05-01T18:25:01.933Z","etag":null,"topics":["cache","caching","caching-library","redis","redis-cache","redis-client","redis-module"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bobbui.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-01-30T01:53:30.000Z","updated_at":"2021-12-25T23:16:09.000Z","dependencies_parsed_at":"2022-08-11T17:50:46.216Z","dependency_job_id":null,"html_url":"https://github.com/bobbui/smart-redis","commit_stats":null,"previous_names":["thangbn/smart-redis"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bobbui%2Fsmart-redis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bobbui%2Fsmart-redis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bobbui%2Fsmart-redis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bobbui%2Fsmart-redis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bobbui","download_url":"https://codeload.github.com/bobbui/smart-redis/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248073091,"owners_count":21043364,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cache","caching","caching-library","redis","redis-cache","redis-client","redis-module"],"created_at":"2024-10-02T16:06:52.725Z","updated_at":"2025-04-09T16:51:50.112Z","avatar_url":"https://github.com/bobbui.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# smart-redis\n\nRedis cache which automatically build and refresh itself.\n\n## Motivation\nThere is data which take really long time to fetch from sources (e.g: reporting query, analytics query) but user need fast response time and result to be always be available. The solution is to create a cache that can build and refresh itself in the background.\n\nIf this sounds like your problem then it is exactly what you're looking for.\n\n## Features\n1. Auto build and update cache on specified interval.\n2. Zero downtime cache refresh.\n3. Support multiple application instances.\n\nThis library has been used in production for few months, but use at your own risk.\n\n## Usage\n\n### Install\n```bash\nnpm install smart-redis --save\n```\nor\n```bash\nyarn add smart-redis\n```\n### init\n\n```javascript\nconst CacheBuilder   = require('smart-redis').CacheBuilder;\nlet cacheBuilder = await CacheBuilder.build({\n    redisOption: {\n        host: 'localhost',\n        port: 6379,\n        password: ''\n    },\n    masterCacheKey: 'sample-key'\n});\n```\n\n### cache register\n```javascript\nfunction sampleDataFetchingFn() {\n    return [{\n        'id': \"id1\",\n        'value': 'value1'\n    },\n        {\n            'id': \"id2\",\n            'value': 'value2'\n        },\n    ]\n}\ncacheBuilder.register({\n    dataFetchingFn: sampleDataFetchingFn,  // \n    cacheKeys: ['sample-key-list', 'sample-key-hash'],\n    redisDataTypes: ['list', 'hash'],\n    refreshInterval: 100,\n    idProp: 'id',\n    name: 'sampleCacheName'\n});\n```\ncache should be built on the redis:\n```bash\n127.0.0.1:6379\u003e keys *\n1) \"CACHE_MASTER_KEYsample-key\"\n2) \"sample-key-hash\"\n3) \"sample-key-list\"\n127.0.0.1:6379\u003e lrange sample-key-list 0 1\n1) \"{\\\"id\\\":\\\"id1\\\",\\\"value\\\":\\\"value1\\\"}\"\n2) \"{\\\"id\\\":\\\"id2\\\",\\\"value\\\":\\\"value2\\\"}\"\n127.0.0.1:6379\u003e hgetall sample-key-hash\n1) \"id2\"\n2) \"{\\\"id\\\":\\\"id2\\\",\\\"value\\\":\\\"value2\\\"}\"\n3) \"id1\"\n4) \"{\\\"id\\\":\\\"id1\\\",\\\"value\\\":\\\"value1\\\"}\"\n127.0.0.1:6379\u003e\n```\n\nsee **lib/sample.js** for full example.\n\n## API references\n\n1. **CacheBuilder.build(options?): Promise\u003cCacheBuilder\u003e**\nentry point function to create CacheBuilder instance.\noptions parameter:\n- redisOption: option to initialize node_redis, see here for full detail: http://redis.js.org/#api-rediscreateclient\n- masterCacheKey: optional prefix for master key, if dont have multiple CacheBuilder instance, you can safely ignore it.\n- logging: optional logging option for debugging purpose\n- logging.enable: whether enable debug logging, default is false\n- logging.logger: customized logger accept log4j and winston instance, default is built-in log4js logger\n- masterAliveCheck: whether to perform master alive check by ping master hostname, only work when application instance running on different machine that can be pinged, do not work inside Kubernetes.\n\n2. **CacheBuilder.register(options): string**\nfunction to register a new cache\noptions parameter:\n- name: cache builder instance name, if not pass an unique name will be generated, should contains only alphanumeric, _ or -\n- dataFetchingFn: function to fetch data. 'list' data type expects an array, 'hash' data type expects an array of object.\n- cacheKeys: array of key strings,\n- redisDataTypes: array of data type corresponding to the aforementioned array of keys, only ['hash', 'list', 'string'] are supported\n- refreshInterval: interval to refresh,\n- idProp: property name to extract key use for hash data structure  \nReturns: unique cache name will be auto generated if not passed\n\n## How it work?\n\nsmart-redis natively support multi application instances deployment\n\n### Cache building and refreshing\n0. Only master instance can build and refresh cache.\n1. Application can register a cache by provide a function to fetch the data, a cache key and a refresh interval.\n2. Cache will be built immediately upon registered and after each time specified refresh interval elapsed.\n    1. **smart-redis** using the function to fetch the data, returned data is stored into cache\n    2. **smart-redis** stored fetched data into cache corresponding to the specified data type\n    3. go to sleep\n\n### Master election\nMaster instance is the application instance that responsible for cache building and refreshing. Only one application instance can become master.  \nWhen an application with **smart-redis** start, it will try to become a master instance\n1. If it becomes master: there will be a master key stored in redis pointing to this instance. cache will be built and refreshed from this instance\n    - If this instance is terminated (by exit, SIGINT, or SIGTERM signal), it will release it master key.\n2. If it couldn't become master:\n    - it will go to sleep and periodically check for master aliveness (if enabled):\n        - If master is alive, go to sleep\n        - If master is not alive, try to become master.\n    - it also listen to master key deletion event. If master key got deleted, all non-master instances will try to become master.\n\n## TODO\n- better connection refused error handling\n- add support for custom parameter to the dataFetchingFn function.\n- add support for **set** data type.\n- encapsulate private function and constructor\n- un-register, update existing cache instance\n- add unit test\n- bug fix: disconnect will render master key monitoring disabled\n- add ES linting\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbobbui%2Fsmart-redis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbobbui%2Fsmart-redis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbobbui%2Fsmart-redis/lists"}