{"id":13582305,"url":"https://github.com/stefanprodan/mongo-swarm","last_synced_at":"2025-04-06T14:30:42.397Z","repository":{"id":72481777,"uuid":"101558379","full_name":"stefanprodan/mongo-swarm","owner":"stefanprodan","description":"Bootstrapping MongoDB sharded clusters on Docker Swarm","archived":true,"fork":false,"pushed_at":"2017-09-06T07:55:48.000Z","size":990,"stargazers_count":125,"open_issues_count":5,"forks_count":40,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-11-05T22:40:57.152Z","etag":null,"topics":["docker","mongodb","swarm"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stefanprodan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-08-27T14:37:22.000Z","updated_at":"2024-06-02T14:24:34.000Z","dependencies_parsed_at":"2023-03-11T13:22:36.868Z","dependency_job_id":null,"html_url":"https://github.com/stefanprodan/mongo-swarm","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefanprodan%2Fmongo-swarm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefanprodan%2Fmongo-swarm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefanprodan%2Fmongo-swarm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefanprodan%2Fmongo-swarm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stefanprodan","download_url":"https://codeload.github.com/stefanprodan/mongo-swarm/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247495664,"owners_count":20948092,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","mongodb","swarm"],"created_at":"2024-08-01T15:02:35.159Z","updated_at":"2025-04-06T14:30:42.090Z","avatar_url":"https://github.com/stefanprodan.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# mongo-swarm\n\n[![Build Status](https://travis-ci.org/stefanprodan/mongo-swarm.svg?branch=master)](https://travis-ci.org/stefanprodan/mongo-swarm)\n[![Docker Image](https://images.microbadger.com/badges/image/stefanprodan/mongo-bootstrap.svg)](https://hub.docker.com/r/stefanprodan/mongo-bootstrap)\n\nMongo-swarm is a POC project that automates the bootstrapping process of a MongoDB cluster for production use.\nWith a single command you can deploy the _Mongos_, _Config_ and _Data_ replica sets onto Docker Swarm, \nforming a high-available MongoDB cluster capable of surviving multiple nodes failure without service interruption. \nThe Docker stack is composed of two MongoDB replica sets, two Mongos instances and the\nmongo-bootstrap service. Mongo-bootstrap is written in Go and handles the replication, sharding and \nrouting configuration.\n\n![Overview](https://github.com/stefanprodan/mongo-swarm/blob/master/diagrams/mongo-swarm.png)\n\n### Prerequisites \n\nIn oder to deploy the MongoDB stack you should have a Docker Swarm cluster made out of eleven nodes:\n\n* 3 Swarm manager nodes (prod-manager-1, prod-manager-2, prod-manager-3)\n* 3 Mongo data nodes (prod-mongodata-1, prod-mongodata-2, prod-mongodata-3)\n* 3 Mongo config nodes (prod-mongocfg-1, prod-mongocfg-2, prod-mongocfg-3)\n* 2 Mongo router nodes (prod-mongos-1, prod-mongos-2)\n\nYou can name your Swarm nodes however you want, \nthe bootstrap process uses placement restrictions based on the `mongo.role` label. \nFor the bootstrapping to take place you need to apply the following labels:\n\n**Mongo data nodes**\n\n```bash\ndocker node update --label-add mongo.role=data1 prod-mongodata-1\ndocker node update --label-add mongo.role=data2 prod-mongodata-2\ndocker node update --label-add mongo.role=data3 prod-mongodata-3\n```\n\n**Mongo config nodes**\n\n```bash\ndocker node update --label-add mongo.role=cfg1 prod-mongocfg-1\ndocker node update --label-add mongo.role=cfg2 prod-mongocfg-2\ndocker node update --label-add mongo.role=cfg3 prod-mongocfg-3\n```\n\n**Mongos nodes**\n\n```bash\ndocker node update --label-add mongo.role=mongos1 prod-mongos-1\ndocker node update --label-add mongo.role=mongos2 prod-mongos-2\n```\n\n### Deploy\n\nClone this repository and run the bootstrap script on a Docker Swarm manager node:\n\n```bash\n$ git clone https://github.com/stefanprodan/mongo-swarm\n$ cd mongo-swarm\n\n$ ./bootstrap.sh\n```\n\nThe bootstrap.sh script creates two overlay networks and deploys the mongo stack:\n\n```bash\ndocker network create --attachable -d overlay mongo\ndocker network create --attachable -d overlay mongos\n\ndocker stack deploy -c swarm-compose.yml mongo\n```\n\n**Networking**\n\nThe config and data replica sets are isolated from the rest of the swarm in the `mongo` overlay network. \nThe routers, Mongos1 and Mongos2 are connected to the `mongo` network and to the `mongos` network. \nYou should attach application containers to the `mongos` network in order to communicate with \nthe MongoDB Cluster.\n\n**Persistent storage** \n\nAt first run, each data and config node will be provisioned with a named Docker volume. This \nensures the MongoDB databases will not be purged if you restart or update the MongoDB cluster. Even if you \nremove the whole stack the volumes will remain on the disk. If you want to delete the MongoDB data and config \nyou have to run `docker volume purge` on each Swarm node.\n\n**Bootstrapping**\n\nAfter the stack has been deploy the mongo-bootstrap container will do the following:\n\n* waits for the data nodes to be online\n* joins the data nodes into a replica set (datars)\n* waits for the config nodes to be online\n* joins the config nodes into a replica set (cfgrs)\n* waits for the mongos nodes to be online\n* adds the data replica set shard to the mongos instances\n\nYou can monitor the bootstrap process by watching the mongo-bootstrap service logs:\n\n```bash\n$ docker service logs -f mongo_bootstrap\n\nmsg=\"Bootstrap started for data cluster datars members [data1:27017 data2:27017 data3:27017]\"\nmsg=\"datars member data1:27017 is online\"\nmsg=\"datars member data2:27017 is online\"\nmsg=\"datars member data3:27017 is online\"\nmsg=\"datars replica set initialized successfully\"\nmsg=\"datars member data1:27017 state PRIMARY\"\nmsg=\"datars member data2:27017 state SECONDARY\"\nmsg=\"datars member data3:27017 state SECONDARY\"\nmsg=\"Bootstrap started for config cluster cfgrs members [cfg1:27017 cfg2:27017 cfg3:27017]\"\nmsg=\"cfgrs member cfg1:27017 is online\"\nmsg=\"cfgrs member cfg2:27017 is online\"\nmsg=\"cfgrs member cfg3:27017 is online\"\nmsg=\"cfgrs replica set initialized successfully\"\nmsg=\"cfgrs member cfg1:27017 state PRIMARY\"\nmsg=\"cfgrs member cfg2:27017 state SECONDARY\"\nmsg=\"cfgrs member cfg3:27017 state SECONDARY\"\nmsg=\"Bootstrap started for mongos [mongos1:27017 mongos2:27017]\"\nmsg=\"mongos1:27017 is online\"\nmsg=\"mongos1:27017 shard added\"\nmsg=\"mongos2:27017 is online\"\nmsg=\"mongos2:27017 shard added\"\n```\n\n**High availability**\n\nA MongoDB cluster provisioned with mongo-swarm can survive node failures and will \nstart an automatic failover if:\n\n* the primary data node goes down\n* the primary config node goes down\n* one of the mongos nodes goes down\n\nWhen the primary data or config node goes down, the Mongos instances will detect the new \nprimary node and will reroute all the traffic to it. If a Mongos node goes down and your applications are \nconfigured to use both Mongos nodes, the Mongo driver will switch to the online Mongos instance. When you \nrecover a failed data or config node, this node will rejoin the replica set and resync if the oplog size allows it.\n\nIf you want the cluster to outstand more than one node failure per replica set, you can \nhorizontally scale up the data and config sets by modifying the swarm-compose.yml file. \nAlways have an odd number of nodes per replica set to avoid split brain situations. \n\nYou can test the automatic failover by killing or removing the primary data and config nodes:\n\n```bash\nroot@prod-data1-1:~# docker kill mongo_data1.1....\nroot@prod-cfg1-1:~# docker rm -f mongo_cfg1.1....\n```\n\nWhen you bring down the two instances Docker Swarm will start new containers to replace the killed ones. \nThe data and config replica sets will choose a new leader and the newly started instances will join the \ncluster as followers. \n\nYou can check the cluster state by doing an HTTP GET on mongo-bootstrap port 9090.\n\n```json\ndocker run --rm --network mongo tutum/curl:alpine curl bootstrap:9090\n```\n\n**Client connectivity**\n\nTo test the Mongos connectivity you can run an interactive mongo container attached to the mongos network:\n\n```bash\n$ docker run --network mongos -it mongo:3.4 mongo mongos1:27017 \n\nmongos\u003e use test\nswitched to db test\n\nmongos\u003e db.demo.insert({text: \"demo\"})\nWriteResult({ \"nInserted\" : 1 })\n\nmongos\u003e db.demo.find()\n{ \"_id\" : ObjectId(\"59a6fa01e33a5cec9872664f\"), \"text\" : \"demo\" }\n```\n\nThe Mongo clients should connect to all Mongos nodes that are running on the mongos overlay network. \nHere is an example with the mgo golang MongoDB driver:\n\n```go\nsession, err := mgo.Dial(\"mongodb://mongos1:27017,mongos2:27017/\")\n```\n\n**Load testing**\n\nYou can run load tests for the MongoDB cluster using the loadtest app. \n\nStart 3 loadtest instances on the mongos network:\n\n```bash\ndocker stack deploy -c swarm-loadtest.yml lt\n``` \n\nThe loadtest app is a Go web service that connects to the two Mongos nodes and does an insert and select:\n\n```go\nhttp.HandleFunc(\"/\", func(w http.ResponseWriter, req *http.Request) {\n    session := s.Repository.Session.Copy()\n    defer session.Close()\n\n    log := \u0026AccessLog{\n        Timestamp: time.Now().UTC(),\n        UserAgent: string(req.Header.Get(\"User-Agent\")),\n    }\n\n    c := session.DB(\"test\").C(\"log\")\n\n    err := c.Insert(log)\n    if err != nil {\n        http.Error(w, err.Error(), http.StatusInternalServerError)\n        return\n    }\n\n    logs := []AccessLog{}\n\n    err = c.Find(nil).Sort(\"-timestamp\").Limit(10).All(\u0026logs)\n    if err != nil {\n        http.Error(w, err.Error(), http.StatusInternalServerError)\n        return\n    }\n\n    b, err := json.MarshalIndent(logs, \"\", \"  \")\n    if err != nil {\n        http.Error(w, err.Error(), http.StatusInternalServerError)\n        return\n    }\n\n    w.Header().Set(\"Content-Type\", \"application/json\")\n    w.Write(b)\n})\n```\n\nThe loadtest service is exposed on the internet on port 9999. \nYou can run the load test using rakyll/hey or Apache bench.\n\n```bash\n#install hey\ngo get -u github.com/rakyll/hey\n\n#do 10K requests \nhey -n 10000 -c 100 -m GET http://\u003cSWARM-PUBLIC-IP\u003e:9999/\n```\n\nWhile running the load test you could kill a _Mongos_, _Data_ and _Config_ node and see \nwhat's the failover impact.\n\nRunning the load test with a single loadtest instance:\n\n```bash\nSummary:\n  Total:\t58.3945 secs\n  Slowest:\t2.5077 secs\n  Fastest:\t0.0588 secs\n  Average:\t0.5608 secs\n  Requests/sec:\t171.2490\n  Total data:\t8508290 bytes\n  Size/request:\t850 bytes\n\nResponse time histogram:\n  0.304 [1835]\t|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎\n  0.549 [3781]\t|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎\n  0.793 [2568]\t|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎\n  1.038 [1153]\t|∎∎∎∎∎∎∎∎∎∎∎∎\n  1.283 [400]\t|∎∎∎∎\n```\n\nRunning the load test with 3 loadtest instances:\n\n```bash\nSummary:\n  Total:\t35.5129 secs\n  Slowest:\t1.9471 secs\n  Fastest:\t0.0494 secs\n  Average:\t0.3223 secs\n  Requests/sec:\t281.5877\n  Total data:\t8508392 bytes\n  Size/request:\t850 bytes\n\nResponse time histogram:\n  0.239 [5040]\t|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎\n  0.429 [2358]\t|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎\n  0.619 [1235]\t|∎∎∎∎∎∎∎∎∎∎\n  0.808 [741]\t|∎∎∎∎∎∎\n  0.998 [396]\t|∎∎∎\n```\n\nScaling up the application from one instance to three instances made the load test 23 seconds faster and the \nrequests per second rate went from 171 to 281.\n\n**Monitoring with Weave Scope**\n\nMonitoring the load test with Weave Cloud shows how the traffic is being routed by the Docker Swarm \nload balancer and by the Mongos instances:\n\n![Traffic](https://github.com/stefanprodan/mongo-swarm/blob/master/diagrams/weave-scope.png)\n\nWeave Scope is a great tool for visualising network traffic between containers and/or Docker Swarm nodes. \nBesides traffic you can also monitor system load, CPU and memory usage. Recording multiple\nload test sessions with Scope you can determine what's the maximum load your infrastructure can take without\na performance degradation. \n\nMonitoring a Docker Swarm cluster with Weave Cloud is as simple as deploying a Scope container on each Swarm node. \nMore info on installing Weave Scope with Docker can be found [here](https://www.weave.works/docs/scope/latest/installing/).\n\n\n\n**Local deployment**\n\nIf you want to run the MongoDB cluster on a single Docker machine without Docker Swarm mode you can use \nthe local compose file. I use it for debugging on Docker for Mac.\n\n```bash\n$ docker-compose -f local-compose.yml up -d\n``` \n\nThis will start all the MongoDB services and mongo-bootstrap on the bridge network without persistent storage. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstefanprodan%2Fmongo-swarm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstefanprodan%2Fmongo-swarm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstefanprodan%2Fmongo-swarm/lists"}