{"id":13721147,"url":"https://github.com/kuzzleio/kuzzle-plugin-cluster","last_synced_at":"2025-04-29T13:33:44.268Z","repository":{"id":71813393,"uuid":"60280849","full_name":"kuzzleio/kuzzle-plugin-cluster","owner":"kuzzleio","description":"Kuzzle cluster mode","archived":false,"fork":false,"pushed_at":"2020-09-07T09:23:20.000Z","size":580,"stargazers_count":7,"open_issues_count":0,"forks_count":2,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-04-05T17:43:15.014Z","etag":null,"topics":["high-availability","high-performance","kuzzle-cluster","kuzzle-plugin","masterless","scalability"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kuzzleio.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2016-06-02T16:42:27.000Z","updated_at":"2020-11-12T02:57:04.000Z","dependencies_parsed_at":"2023-06-11T00:15:34.896Z","dependency_job_id":null,"html_url":"https://github.com/kuzzleio/kuzzle-plugin-cluster","commit_stats":null,"previous_names":[],"tags_count":25,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuzzleio%2Fkuzzle-plugin-cluster","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuzzleio%2Fkuzzle-plugin-cluster/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuzzleio%2Fkuzzle-plugin-cluster/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuzzleio%2Fkuzzle-plugin-cluster/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kuzzleio","download_url":"https://codeload.github.com/kuzzleio/kuzzle-plugin-cluster/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251509934,"owners_count":21600726,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["high-availability","high-performance","kuzzle-cluster","kuzzle-plugin","masterless","scalability"],"created_at":"2024-08-03T01:01:12.923Z","updated_at":"2025-04-29T13:33:43.660Z","avatar_url":"https://github.com/kuzzleio.png","language":"JavaScript","funding_links":[],"categories":["Plugins"],"sub_categories":["Others"],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/7868838/66303815-2617b000-e8fc-11e9-8c3f-613574be1746.png\"/\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://david-dm.org/kuzzleio/kuzzle-plugin-cluster\"\u003e\n    \u003cimg src=\"https://david-dm.org/kuzzleio/kuzzle-plugin-cluster.svg\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://travis-ci.com/kuzzleio/kuzzle-plugin-cluster\"\u003e\n    \u003cimg alt=\"undefined\" src=\"https://travis-ci.com/kuzzleio/kuzzle-plugin-cluster.svg?branch=master\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://codecov.io/gh/kuzzleio/kuzzle-plugin-cluster\"\u003e\n    \u003cimg src=\"https://codecov.io/gh/kuzzleio/kuzzle-plugin-cluster/branch/master/graph/badge.svg\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/kuzzleio/kuzzle-plugin-cluster/blob/master/LICENSE\"\u003e\n    \u003cimg alt=\"undefined\" src=\"https://img.shields.io/github/license/kuzzleio/kuzzle-plugin-cluster.svg?style=flat\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n\n## About\n\n### Kuzzle Cluster Plugin\n\nThis plugin adds a masterless cluster mode to Kuzzle.\n\n\u003cp align=\"center\"\u003e\n  :books: \u003cb\u003e\u003ca href=\"https://docs.kuzzle.io/core/1/guides/kuzzle-depth/scalability\"\u003eDocumentation\u003c/a\u003e\u003c/b\u003e\n\u003c/p\u003e\n\n### Kuzzle\n\nKuzzle is a ready-to-use, **on-premises and scalable backend** that enables you to manage your persistent data and be notified in real-time on whatever happens to it. \nIt also provides you with a flexible and powerful user-management system.\n\n* :watch: __[Kuzzle in 5 minutes](https://kuzzle.io/company/about-us/kuzzle-in-5-minutes/)__\n* :octocat: __[Github](https://github.com/kuzzleio/kuzzle)__\n* :earth_africa: __[Website](https://kuzzle.io)__\n* :books: __[Documentation](https://docs.kuzzle.io)__\n* :email: __[Gitter](https://gitter.im/kuzzleio/kuzzle)__\n\n### Get trained by the creators of Kuzzle :zap:\n\nTrain yourself and your teams to use Kuzzle to maximize its potential and accelerate the development of your projects.  \nOur teams will be able to meet your needs in terms of expertise and multi-technology support for IoT, mobile/web, backend/frontend, devops.  \n:point_right: [Get a quote](https://hubs.ly/H0jkfJ_0)\n\n### Compatibility matrice\n\n| Kuzzle Version | Plugin Version |\n| -------------- | -------------- |\n| 1.8.x          | 3.x.x          | \n| 2.x.x          | 4.x.x          |\n\n\n## Try it\n\nTo run a kuzzle stack, you can use the provided compose file:\n\n```bash\ndocker-compose up --scale kuzzle=3\n```\n\n**NB: This compose stack is for tests and development only and should not be used as-is on production.** \n\n## Run a development stack\n\n```bash\ncd \u003cdir\u003e\ngit clone https://github.com/kuzzleio/kuzzle-plugin-cluster.git\n\ncd kuzzle-plugin-cluster\ncp docker-compose/my.env.sample docker-compose/my.env\n\n./dev-npm-install.sh\n./dev.sh\n```\n\nYou should now have a full Kuzzle clustered stack running 3 Kuzzle front nodes (and 3 servers).\nEach update on the cluster source should automatically restart kuzzle.\n\n**Note:** on some Linux environments, you may get `ENOSPC` errors from the filesystem watcher. If so, you need to raise the limits on the number of files that can be watched:\n\n`sudo sysctl -w fs.inotify.max_user_watches=524288`\n\n### Goodies\n\n* [http://localhost:7512/_plugin/cluster/status] =\u003e cluster status\n* `curl -XPOST http://localhost:7512/_plugin/cluster/reset` =\u003e resets redis state and force a new sync (blanks cluster state)\n* `bash docker-compose/scripts/devtools.sh` dumps to the standard output the urls to copy/paste in Google Chrome to live-debug the nodes\n\n## Run the cluster in production\n\n### Install\n\nThe cluster needs to be installed as a plugin. Please refer to [Kuzzle documentation](https://docs.kuzzle.io/guide/1/essentials/plugins/#installing-a-plugin) on how to proceed.\n\n### Network ports\n\nBy default, Kuzzle nodes communicate with each other using two channels on ports `7510` and `7511`.  \n\n**NB: These ports are used by the cluster only and do not need to be publicly exposed.**\n\nYou can configure the ports used in the `bindings` section of the plugin configuration (cf [below](#configuration)).\n\nEach Kuzzle node also needs to be able to access Redis and Elasticsearch services.\n\n### Healthcheck\n\nThe cluster exposes a healthcheck route on http://kuzzle:7512/_plugin/cluster/health\n\nThe route returns a 200 status code only if the `minimumNodes` set in the configuration is reached (cf [configuration](#configuration) below).\n\n## Configuration\n\n### Privileged context\n\nThis plugin needs privileged context to work. This context is granted by Kuzzle via the global configuration. Add the following to your configuration\n\n```javascript\nplugins: {\n    cluster: {\n        privileged: true\n    }\n}\n```\n\nFor more information on how to configure Kuzzle, [please refer to the Guide](http://docs.kuzzle.io/guide/#configuring-kuzzle).\n\n### Pipe plugin timeouts\n\nThis plugin registers some pipe plugins which induce some delay and will exceed default Kuzzle timeouts. \nMake sure you increase your pipe timeouts accordingly.\n\n```json\n  \"plugins\": {\n    \"common\": {\n      \"pipeWarnTime\": 5000,\n      \"pipeTimeout\": 10000\n    }\n```\n\n### Bindings\n\nThe bindings on which each node can be reached by the others can be configured in the `bindings` section of the cluster plugin configuration:\n\n```json\n\"plugins\": {\n  \"cluster\": {\n    \"bindings\": {\n      \"pub\": \"tcp://[_site_:ipv4]:7511\",\n      \"router\": \"tcp://[_site_:ipv4]:7510\"\n    }\n  }\n}\n```\n\nThe syntax is `tcp://[\u003cinterface\u003e:\u003cfamily\u003e]:\u003cport\u003e`, where\n\n- `interface` is either a network interface (i.e. `eth0`), an ip address (i.e. `0.0.0.0`) or `_site_`. If set to `_site_`, the first public ip will be used.\n- `family` is either set to `ipv4` (default) or `ipv6`\n- `port` is set to the port to listen to\n\n### Redis cluster\n\nRedis cluster comes with some limitations:\n\n1. Single database only.\n2. Cluster node arrays.\n\nThe latter implies the configuration cannot be set via environment variables.\nTo comply with the former, make sure to set only one database index (0).\n\n### Full configuration sample\n\nHere is a complete sample configuration using a 3 nodes redis cluster and a 2 elasticsearch nodes.\n\n```json\n\"plugins\": {\n  \"common\": {\n    \"pipeWarnTime\": 5000,\n    \"pipeTimeout\": 10000\n  },\n  \"cluster\": {\n    \"privileged\": true,\n    \"bindings\": {\n      \"pub\": \"tcp://[_site_:ipv4]:7511\",\n      \"router\": \"tcp://[_site_:ipv4]:7510\"\n    },\n    \"minimumNodes\": 1,\n    \"retryJoin\": 30,\n    \"timers\": {\n      \"discoverTimeout\": 3000,\n      \"joinAttemptIntervall\": 2000,\n      \"heartbeat\": 5000,\n      \"waitForMissingRooms\": 4500\n    }\n  }\n},\n\n\"services\": {\n  \"internalCache\": {\n    \"database\": 0,\n    \"nodes\": [\n      {\n        \"host\": \"redis1\",\n        \"port\": 6379\n      },\n      {\n        \"host\": \"redis2\",\n        \"port\": 6379\n      },\n      {\n        \"host\": \"redis3\",\n        \"port\": 6379\n      }\n    ]\n  },\n  \"memoryStorage\": {\n    \"database\": 0,\n    \"nodes\": [\n      {\n        \"host\": \"redis1\",\n        \"port\": 6379\n      },\n      {\n        \"host\": \"redis2\",\n        \"port\": 6379\n      },\n      {\n        \"host\": \"redis3\",\n        \"port\": 6379\n      }\n    ]\n  },\n  \"db\": {\n    \"client\": {\n      \"host\": false,\n      \"hosts\": [\n        {\n          \"protocol\": \"http\",\n          \"host\": \"elasticsearch1\",\n          \"port\": 9200\n        },\n        {\n          \"protocol\": \"http\",\n          \"host\": \"elasticsearch2\",\n          \"port\": 9200\n        }\n      ]\n    }\n  }\n}\n\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkuzzleio%2Fkuzzle-plugin-cluster","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkuzzleio%2Fkuzzle-plugin-cluster","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkuzzleio%2Fkuzzle-plugin-cluster/lists"}