Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/sagiegurari/multiple-redis
Run redis commands against multiple redis instances.
https://github.com/sagiegurari/multiple-redis
database nodejs pubsub redis redis-client
Last synced: 8 days ago
JSON representation
Run redis commands against multiple redis instances.
- Host: GitHub
- URL: https://github.com/sagiegurari/multiple-redis
- Owner: sagiegurari
- License: apache-2.0
- Created: 2015-09-02T21:15:29.000Z (about 9 years ago)
- Default Branch: master
- Last Pushed: 2023-01-19T17:37:45.000Z (almost 2 years ago)
- Last Synced: 2024-10-15T04:26:49.389Z (21 days ago)
- Topics: database, nodejs, pubsub, redis, redis-client
- Language: JavaScript
- Size: 191 KB
- Stars: 13
- Watchers: 7
- Forks: 2
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- Contributing: .github/CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
# multiple-redis
[](https://www.npmjs.org/package/multiple-redis) [](https://github.com/sagiegurari/multiple-redis/actions) [](https://coveralls.io/r/sagiegurari/multiple-redis) [](https://snyk.io/test/github/sagiegurari/multiple-redis) [](http://inch-ci.org/github/sagiegurari/multiple-redis) [](https://github.com/sagiegurari/multiple-redis/blob/master/LICENSE) [](https://www.npmjs.org/package/multiple-redis)
> Run redis commands against multiple redis instances.
* [Overview](#overview)
* [Why?](#why)
* [How This Library Works](#howlibworks)
* [Simple Scenario](#scenario)
* [Usage](#usage)
* [Debug](#debug)
* [Installation](#installation)
* [Limitations](#limitations)
* [API Documentation](docs/api.md)
* [Contributing](.github/CONTRIBUTING.md)
* [Release History](#history)
* [License](#license)
## Overview
This library enables to submit redis commands to multiple redis instances.
The client interface is the same as the 'redis' node package at: [node redis](https://github.com/NodeRedis/node_redis)
However, every command actually invokes multiple redis backends.
### Why
Generally in production you would like a failover capability for your redis server.
Working with only one redis instance can cause your entire production system to fail in case redis goes down for any reason.
For example, when using redis as an express session store and redis is down, users HTTP requests will be rejected.
So how can we setup high availability?
You could setup redis master and redis slave. That would have those two redis servers sync data between them and you can read it from both.
But what if the master redis goes down? If you didn't setup the slave as writable, you are in a possible problem until the master is up.
If you did, again there is an issue when master goes back up.
Not to mention you have to setup a deployment installation process to set one redis for master and one for slave (well... at least one).
So one solution is to be able to change the master in runtime in such a scenario, so now you have a choice to deploy sentinal.
But that is even more work on the production team and what if sentinal goes down? Who monitors the monitor?
So another choice is to let the app code detect that the master is down and let it send the commands to select a new master from the existing slaves.
But you have multiple instances of your app, and they can all try to select a new master and cause issues for your redis servers.
So existing solutions don't work? Of course they do. But they are a pain to configure, manage and monitor in production.
This is where this library comes in.
What if you had multiple standalone redis servers that didn't know of each other?
Your app ensures they are sync or at least it knows to pick up the data from the correct place.
If one goes down, no problem. You are running with the redis servers that are still up and they too hold the data you need.
**Important to explain, this does not mean that I do not support the Redis master/slave + sentinal solution** but sometimes using multiple
independent Redis servers serves as a better solution, which is much more simple to deploy and manage in production.
### How This Library Works
This library basically does 2 main things.
* Get commands - When a get (which does not modify data) command is called, the redis client will go redis by redis in a sequence until it finds a redis which provides data for the get command.
Any error, or any redis which is unable to provide the data is ignored.
Once a specific redis provides the data, the next redis servers are skipped and the command callback is invoked with that data.
* Set/Other commands - When a non 'get' command is called, all redis servers are invoked in parallel with the same command to ensure that all redis servers are updated with latest data.
If at least redis server was able to process the command, the original command callback will receive a valid response.
This means that at least once redis server needs to work good for the main client to notify the calling code that everything works ok.
To ensure you don't get outdated data, the redis servers should work without persistence to disk.
So when a redis server goes back up, it is empty. But you still get the data from the other redis that was up and holds that data.
When that key gets updated with new data, both redis servers are now holding the latest version.
A side affect of this solution is that every publish event would be received multiple times by the subscribers, so you need to code accordingly and prevent any possible issues from handling duplicate published messages.
### Simple Scenario
Let us take the express redis session store as an example.
Since this library provides the same redis interface as the common redis client, you can provide this library to the redis session store.
When a new session is created, it will be created in all redis servers (in this example, lets assume we have 2).
In case the first redis server suddenly fails, the session store is still able to fetch and update the session data from the second redis server.
When the first redis server comes back up, the session is still available to the session store from the second redis server and
any session modification (due to some HTTP request) will cause both redis servers to now hold the latest express session data.
It is by no means, a perfect solution, but it does has its advantages.
First and foremost, its simple deployment requirements.
## Usage
In order to use this library, you need to either create the redis clients or provide the redis connection data as follows:
```js
//create multiple redis clients
var redis = require('redis');
var client1 = redis.createClient(...);
var client2 = redis.createClient(...);
//create the wrapper client
var MultipleRedis = require('multiple-redis');
var multiClient = MultipleRedis.createClient([client1, client2]);
multiClient.once('all-ready', function onReady() {
//run any command on the multi client instead of the original clients
multiClient.set('string key', 'string val', callback);
});
```
Or
```js
//create the wrapper client with connection info
var MultipleRedis = require('multiple-redis');
var multiClient = MultipleRedis.createClient([{
host: 'host1',
port: 6379
}, {
host: 'host2',
port: 6379
}], options);
multiClient.once('all-ready', function onReady() {
//run any command on the multi client instead of the original clients
multiClient.set('string key', 'string val', callback);
});
```
The rest of the API is the same as defined in the redis node library: [node redis](https://github.com/NodeRedis/node_redis#api)
## Debug
In order to turn on debug messages, use the standard nodejs NODE_DEBUG environment variable.
````ini
NODE_DEBUG=multiple-redis
````
## Installation
In order to use this library, just run the following npm install command:
```sh
npm install --save multiple-redis
```
## Limitations
Not all of the original redis client attributes are available (for example: redis.debug_mode).
Also Multi is currently not supported.
## API Documentation
See full docs at: [API Docs](docs/api.md)
## Contributing
See [contributing guide](.github/CONTRIBUTING.md)
| Date | Version | Description |
| ----------- | ------- | ----------- |
| 2020-05-12 | v2.0.0 | Migrate to github actions and upgrade minimal node version |
| 2020-02-10 | v1.3.0 | Upgrade to redis 3 |
| 2019-06-24 | v1.2.1 | Added 'forceParallel' option to issue GET in parallel #5 |
| 2017-07-26 | v1.0.61 | Added all-ready event and allConnected attribute |
| 2017-01-18 | v1.0.29 | setnx type commands now run in sequence and not in parallel |
| 2015-10-22 | v0.0.16 | Timeout child commands (see childCommandTimeout option) |
| 2015-09-23 | v0.0.7 | Upgrade to redis 2.0 |
| 2015-09-03 | v0.0.3 | Added support for connected and server_info attributes. |
| 2015-09-03 | v0.0.2 | Initial release. |
## License
Developed by Sagie Gur-Ari and licensed under the Apache 2 open source license.