{"id":20875017,"url":"https://github.com/servicestack/redis-config","last_synced_at":"2025-05-12T15:31:03.293Z","repository":{"id":36530049,"uuid":"40835821","full_name":"ServiceStack/redis-config","owner":"ServiceStack","description":"Configuration, scripts and docs for setting up redis servers and sentinels in popular configurations","archived":false,"fork":false,"pushed_at":"2017-02-16T02:44:19.000Z","size":6143,"stargazers_count":102,"open_issues_count":1,"forks_count":37,"subscribers_count":21,"default_branch":"master","last_synced_at":"2024-08-18T00:56:01.138Z","etag":null,"topics":["linux","osx","redis","redis-sentinel","sentinel-configuration","windows"],"latest_commit_sha":null,"homepage":"","language":"Batchfile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ServiceStack.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-08-16T19:32:16.000Z","updated_at":"2024-05-08T19:12:14.000Z","dependencies_parsed_at":"2022-09-22T16:42:43.821Z","dependency_job_id":null,"html_url":"https://github.com/ServiceStack/redis-config","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ServiceStack%2Fredis-config","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ServiceStack%2Fredis-config/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ServiceStack%2Fredis-config/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ServiceStack%2Fredis-config/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ServiceStack","download_url":"https://codeload.github.com/ServiceStack/redis-config/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225140104,"owners_count":17427090,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["linux","osx","redis","redis-sentinel","sentinel-configuration","windows"],"created_at":"2024-11-18T06:41:07.179Z","updated_at":"2024-11-18T06:41:07.836Z","avatar_url":"https://github.com/ServiceStack.png","language":"Batchfile","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Instant Redis Sentinel Setup\n\n![Instant Redis Sentinel Setup](https://raw.githubusercontent.com/ServiceStack/Assets/master/img/redis/instant-sentinel-setup.png)\n\nThis repository contains the necessary configuration and redis binaries to quickly setup the [Basic Sentinel configuration](http://redis.io/topics/sentinel#example-2-basic-setup-with-three-boxes) on Linux, OSX or Windows servers. The default configuration supports spawning multiple redis processes which can conveniently all be run on a single server using the included start-all/stop-all scripts (ideal for development environments mimicking their production Sentinel environments). \n\n## Usage \n\nTo run the included Sentinel configuration, clone this repo on the server you want to run it on:\n\n    git clone https://github.com/ServiceStack/redis-config.git\n    \nThen run the scripts for the target Operating System. This repository includes the [latest stable](http://redis.io/download) pre-built binaries for OSX and [MSOpen Tech's latest builds](https://github.com/ServiceStack/redis-windows#running-microsofts-native-port-of-redis) for Windows port of Redis in the `/bin` folder. Due to Linux binaries being less portable, the Linux bash scripts assumes an existing install of redis is available in your $PATH.\n\n### Windows\n\nStart 1x Master, 2x Slaves and 3x Sentinel redis-servers:\n\n    cd redis-config\\sentinel3\\windows\n    start-all.cmd\n\nShutdown started instances:\n\n    stop-all.cmd\n\n### OSX\n\nStart 1x Master, 2x Slaves and 3x Sentinel redis-servers:\n\n    cd redis-config\\sentinel3\\osx\n    ./start-all.sh\n\nShutdown started instances:\n\n    ./stop-all.sh\n\n### Linux\n\nStart 1x Master, 2x Slaves and 3x Sentinel redis-servers:\n\n    cd redis-config\\sentinel3\\linux\n    ./start-all.sh\n\nShutdown started instances:\n\n    ./stop-all.sh\n\nThe bash scripts for OSX and Linux require execute permission which can be enabled with:\n\n    chmod a=rx start-all.sh\n    chmod a=rx stop-all.sh\n\n### Checking the running instances\n\nAfter spawning multiple redis-server instances the `start-all` script will pause for a key press. Wait a few seconds before hitting return until you see **+slave** and **+sentinel** log entries in the Sentinel's console output. These entries show the sentinels successfully auto-detecting and registering the different redis instances. \n\nHitting return will ask the first 2 Sentinel servers for info of the current master and slaves. If everything's configured and working correctly it will show info on each server.\n\n### Reset to Original Configuration\n\nTo capture the active roles of the different redis instances, redis will rewrite the redis.conf and sentinel.conf files. You can reset it back to the original configuration by discarding all changes in your cloned git repository which git lets us do with:\n\n    $ git reset --hard head\n\n## Sentinel Configuration\n\nThe goal of this project is to specify the minimal amount of info required to create a working Sentinel configuration. Anything not specified falls back to use the original **redis.conf** defaults shipped with the latest stable distribution of Redis. \n\nThe configurations are logically split into separate `/server-{port}` directories to match the redis and sentinel instances to run on each server. They use layered config so they're easily customizable and can be started independently with the included per-server start/stop scripts.\n\nWe'll take a close look at how one of the slaves are configured to see how it fits together. For this example we'll walk through the configurations for OSX located at [/sentine3/osx](https://github.com/ServiceStack/redis-config/tree/master/sentinel3/osx):\n\n```\n/osx\n  /server-6380\n  /server-6381     #Config for Slave node\n    redis.conf     #Config for Slave instance on port 6381\n    sentinel.conf  #Config for Sentinel instance on port 26381\n  /server-6382\n\nredis.conf         #Default redis.conf shipped in latest Redis Stable\n```\n\nFrom the directory structure above we see there's a separate configuration for the master (6380) and its slaves (6381,6382) instances. We've started from port 6380 so this could also be run along-side an existing redis intance on 6379 if needed.\n\n#### [/server-6381/redis.conf](https://github.com/ServiceStack/redis-config/blob/master/sentinel3/osx/server-6381/redis.conf):\n\nThe configuration for this slave is contained in `redis.conf` which is just:\n\n```\n# Relative to ./sentinel3/osx\ninclude redis.conf\n\nport 6381\ndir ./server-6381/state\nslaveof 127.0.0.1 6380\n```\n\nThe `include` directive lets us easily inherit the default `/redis.conf` allowing us to only provide non-default configuration specific to this slave. In this case it will run on port **6381**, persist any RDB snapshots in **./server-6381/state** directory and run as a replicated slave to the master instance running on **127.0.0.1 6380**:\n\n#### [/server-6381/sentinel.conf](https://github.com/ServiceStack/redis-config/blob/master/sentinel3/osx/server-6381/sentinel.conf):\n\nThe configuration for the Sentinel indicates it should run on port **26381** and monitor the master at **127.0.0.1 6380** that belongs to the **mymaster** group and requires a quorum of **2** sentinels to reach consensus before any action can be taken:\n\n```\n# Relative to ./sentinel3/osx\n\nport 26381\ndir ./server-6381/state\nsentinel monitor mymaster 127.0.0.1 6380 2\n```\n\n### Localhost vs Network IP's\n\nThis configuration assumes all redis instances are running locally on **127.0.0.1**. If you're instead running it on a remote server that you want anyone in your network to be able to access, you'll need to either change the IP Address in the `*.conf` to use the servers Network IP or alternatively you can leave the defaults but you'll need to map any loopback IP's to its Network IP in the Redis Client library. \n\nThis can be configured in the C# [ServiceStack.Redis](https://github.com/ServiceStack/ServiceStack.Redis) `RedisSentinel` client using the `IpAddressMap` Dictionary. \n\nE.g. if the remote servers Network IP is **10.0.0.9**, it can be configured with:\n\n```csharp\nvar sentinel = new RedisSentinel(new[]{ \"10.0.0.9:26380\" }) {\n    IpAddressMap = {\n        {\"127.0.0.1\", \"10.0.0.9\"},\n    }\n};\ncontainer.Register\u003cIRedisClientsManager\u003e(c =\u003e sentinel.Start());\n```\n\nThe `IpAddressMap` is used to transparently map any returned local **127.0.0.1** IP Address to the **10.0.0.9** Network IP that any other computer on the same network can connect to. \n\nThe `RedisSentinel` client also just needs to be configured with a single Sentinel IP which it uses to automatically resolve the IP Addresses of the remaining Sentinels.\n\n## [3x Sentinels monitoring 1x Master and 2x Slaves](http://redis.io/topics/sentinel#example-2-basic-setup-with-three-boxes)\n\nWith the efficiency and headroom provided from a single redis-server instance, we expect this minimal Sentinel configuration required to achieve high-availability will be the most popular configuration used for Sentinel, which this repository focuses on.\n\nFrom the [redis Sentinel docs](http://redis.io/topics/sentinel#example-2-basic-setup-with-three-boxes) this setup consists of 1x redis master and 2x redis slaves instances with a Redis Sentinel monitoring each redis instance:\n\n```\n            +----------+\n            | M1:6380  |\n            | S1:26380 |\n            +----------+\n                 |\n +----------+    |    +----------+\n | R2:6381  |----+----| R3:6382  |\n | S2:26381 |         | S3:26382 |\n +----------+         +----------+\n Configuration: quorum = 2\n```\n\n\u003e The above diagram also shows the ports for each of the master, slave and sentinel instances.\n\nThis setup enables a \"highly-available\" configuration which can survive a single **redis-server** or **redis-sentinel** instance or entire server failing. \n\n### Redis Slave Server failing\n\nIf any of the slave servers fail it's business as usual with just one less replicated slave. This would still cause temporary disruption for any read-only clients that were connected to the failed slave. Redis clients [following the recommended client strategy](http://redis.io/topics/sentinel-clients) automatically recovers by asking one of the Sentinels for the next available address to connect to, resuming operations with one of the available instances.\n\n### Redis Master Server failing\n\nIt's more disruptive when the master server fails since that's where most of the clients are going to be connected to, who'll only be able to resume until the remaining 2 Sentinels both agree (quorum=2) the master is **objectively down** and begins the process of failing over promoting one of the replicated slaves to master. Since replication to slaves is asynchronous there's a small window for loss of writes to master that weren't replicated in time before it failed. \n\nThere's a greater potential for loss if there's instead a network split, cutting off the master server from the remaining sentinel servers, i.e:\n\n```\n         +----+\n         | M1 |\n         | S1 | \u003c- C1 (writes will be lost)\n         +----+\n            |\n            /\n            /\n+------+    |    +----+\n| [M2] |----+----| R3 |\n| S2   |         | S3 |\n+------+         +----+\n```\n\nIn this case the master continues to process redis commands but won't be able to successfully replicate them to any of its slaves. If the network split was long enough to cause the other 2 Sentinels to presume the master had failed and promoted one of the slaves to master, by the time the master rejoins the network it assumes the role as slave and ends up losing all the writes it processed before the network split. \n\nThis loss can be mitigated by configuring the master to stop accepting writes unless it has successfully replicated to a minimum number of slaves specified in the **min-slaves-to-write** and **min-slaves-max-lag** tolerance options. Consult the [Redis Sentinel docs](http://redis.io/topics/sentinel#example-2-basic-setup-with-three-boxes) for more info.\n\n### Google Cloud - [Click to Deploy Redis](https://cloud.google.com/solutions/redis/click-to-deploy)\n\nThe easiest Cloud Service we've found that can instantly setup a multi node Redis  Sentinel Configuration as described above is using Google Cloud's [click to deploy Redis feature](https://cloud.google.com/solutions/redis/click-to-deploy) available from the Google Cloud Console under **Deploy \u0026 Manage**:\n\n![](https://raw.githubusercontent.com/ServiceStack/Assets/master/img/redis/sentinel3-gcloud-01.png)\n\nClicking **Deploy** button will let you configure the type, size and location where you want to deploy the Redis VM's:\n\n![](https://raw.githubusercontent.com/ServiceStack/Assets/master/img/redis/sentinel3-gcloud-02.png)\n\nClicking **Deploy Redis** with a **3** Cluster node count will deploy 3 Linux VM instances containing:\n\n - A master (read/write) Redis instance (port 6379)\n - Two replicated slaves (read-only) Redis instances with data persistence (port 6379)\n - Redis Sentinel running in each instance (port 26379)\n\nAfter it's finished deploying you'll see the instance names and External IP's for each of the VM instances:\n\n![](https://raw.githubusercontent.com/ServiceStack/Assets/master/img/redis/sentinel3-gcloud-03.png)\n\nIt also includes a handy link to be able SSH directly into any of the instances from a browser. Whilst unnecessary since the deployed configuration is immediately ready for use, it still provides a convenient way to quickly explore the different configurations of each of the VM's.\n\nAfter its deployed you can get to a summary of all running VM's under `Compute \u003e Compute Engine \u003e VM instances` which provides a real-time graph monitoring the CPU usage of all the VM's:\n\n![](https://raw.githubusercontent.com/ServiceStack/Assets/master/img/redis/sentinel3-gcloud-04.png)\n\nClicking on any of the instances lets you drill-down to see more info like the Internal IP of the instance used in its internal network:\n\n![](https://raw.githubusercontent.com/ServiceStack/Assets/master/img/redis/sentinel3-gcloud-05.png)\n\nAnother way to access this info is via the great command-line support available in [Google Cloud SDK](https://cloud.google.com/sdk/) which after you've [authenticated and set the project](https://cloud.google.com/sdk/gcloud/#gcloud.auth) you want to target will let you see all running instances with:\n\n    gcloud compute instances list\n\nWhich returns the summary info of all running VM instances in that project, e.g:\n\n![](https://raw.githubusercontent.com/ServiceStack/Assets/master/img/redis/sentinel3-gcloud-06.png)\n\n\n### External vs Internal IP's\n\nSomething to keep in mind are that the Redis Sentinels are configured to monitor and report the redis instances internal IP's. Whilst this is ideal when accessing the redis instances from within the same internal network (e.g. from other deployed VM's), it will be an issue if you're connecting to the External IP from an external network (e.g. from your developer workstation) as the IP's returned from the Sentinels are the Internal IP's. \n\nOne solution would be to configure a virtual network so the Internal Network IP's are routed to the Google Cloud VM's. A less invasive solution would be to instead have those Internal IP's mapped to their equivalent External IP's. \n\nThis is available in the C# [ServiceStack.Redis](https://github.com/ServiceStack/ServiceStack.Redis) Client by specifying an Internal -\u003e External IP mapping when initializing a `RedisSentinel`:\n\n```csharp\nvar sentinel = new RedisSentinel(SentinelHosts, \"master\")\n{\n    IpAddressMap =\n    {\n        {\"10.240.9.29\", \"104.197.142.253\"},\n        {\"10.240.32.159\", \"104.197.132.102\"},\n        {\"10.240.170.236\", \"104.197.118.169\"},\n    }\n};\n\ncontainer.Register\u003cIRedisClientsManager\u003e(c =\u003e sentinel.Start());\n```\n\nNow when the Connection Manager asks the Sentinel for the IP of an available instance, the returned Internal IP gets transparently mapped to its equivalent External IP which the RedisClient can connect to.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fservicestack%2Fredis-config","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fservicestack%2Fredis-config","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fservicestack%2Fredis-config/lists"}