{"id":13498197,"url":"https://github.com/xiaopeng163/Mongodb-Replication-Test","last_synced_at":"2025-03-28T23:30:47.131Z","repository":{"id":25898619,"uuid":"29339207","full_name":"xiaopeng163/Mongodb-Replication-Test","owner":"xiaopeng163","description":null,"archived":false,"fork":false,"pushed_at":"2016-05-31T09:03:34.000Z","size":1626,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-08-01T20:38:25.102Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xiaopeng163.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-01-16T08:20:11.000Z","updated_at":"2018-03-19T18:43:08.000Z","dependencies_parsed_at":"2022-08-24T14:16:32.787Z","dependency_job_id":null,"html_url":"https://github.com/xiaopeng163/Mongodb-Replication-Test","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xiaopeng163%2FMongodb-Replication-Test","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xiaopeng163%2FMongodb-Replication-Test/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xiaopeng163%2FMongodb-Replication-Test/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xiaopeng163%2FMongodb-Replication-Test/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xiaopeng163","download_url":"https://codeload.github.com/xiaopeng163/Mongodb-Replication-Test/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":222426447,"owners_count":16982684,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T20:00:53.682Z","updated_at":"2024-10-31T14:31:36.101Z","avatar_url":"https://github.com/xiaopeng163.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# MongoDB HA (Replication) Test with Pymongo\n\n\n## 1 Environment Setup\n\nOne CentOS 7 machine\n\n``` bash\n$ more /etc/redhat-release \nCentOS Linux release 7.2.1511 (Core)\n```\n\nMongodb version\n\n``` bash\n$ mongod --version\ndb version v3.2.6\ngit version: 05552b562c7a0b3143a729aaa0838e558dc49b25\nOpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013\nallocator: tcmalloc\nmodules: none\nbuild environment:\n    distmod: rhel70\n    distarch: x86_64\n    target_arch: x86_64\n```\n\nPymongo version\n\n``` bash\n\u003e\u003e\u003e import pymongo\n\u003e\u003e\u003e pymongo.version\n'3.2'\n\u003e\u003e\u003e \n```\n\n\n### 1.1 Install mongodb\n\nPlease see https://docs.mongodb.com/manual/tutorial/install-mongodb-on-red-hat/\n\n### 1.2 Configure mongodb\n\nWe will configure three nodes replication in one machine，like:\n\n![main](_image/1.png)\n\n1) Prepare three mongodb config file\n\n``` bash\n$ ls /etc/ | grep mongo\nmongod1.conf\nmongod2.conf\nmongod3.conf\n```\n\nEach config file is:\n\n``` bash\n$ more /etc/mongod1.conf \n# mongod.conf\n\nsystemLog:\n  destination: file\n  logAppend: true\n  path: /var/log/mongodb1/mongod.log\n\nstorage:\n  dbPath: /var/lib/mongo1\n  journal:\n    enabled: true\n    \nprocessManagement:\n  fork: true  # fork and run in background\n  pidFilePath: /var/run/mongodb1/mongod.pid  # location of pidfile\n\nnet:\n  port: 27017\n\nreplication:\n  replSetName: rs0\n\n$ more /etc/mongod2.conf \n# mongod.conf\n\nsystemLog:\n  destination: file\n  logAppend: true\n  path: /var/log/mongodb2/mongod.log\n\nstorage:\n  dbPath: /var/lib/mongo2\n  journal:\n    enabled: true\n    \nprocessManagement:\n  fork: true  # fork and run in background\n  pidFilePath: /var/run/mongodb2/mongod.pid  # location of pidfile\n\nnet:\n  port: 27018\n\nreplication:\n  replSetName: rs0\n\n$ more /etc/mongod3.conf \n# mongod.conf\n\nsystemLog:\n  destination: file\n  logAppend: true\n  path: /var/log/mongodb3/mongod.log\n\nstorage:\n  dbPath: /var/lib/mongo3\n  journal:\n    enabled: true\n    \nprocessManagement:\n  fork: true  # fork and run in background\n  pidFilePath: /var/run/mongodb3/mongod.pid  # location of pidfile\n\nnet:\n  port: 27019\n\nreplication:\n  replSetName: rs0\n```\n\n### 1.3 Start 3 mongodb instances and init them\n\n``` bash\n$ sudo /usr/bin/mongod -f /etc/mongod1.conf\n$ sudo /usr/bin/mongod -f /etc/mongod2.conf\n$ sudo /usr/bin/mongod -f /etc/mongod3.conf\n\n```\n\nConnect to one of your mongod instances through the mongo shell. MongoDB initiates a set that consists of the current member and that uses the default replica set configuration.\n\n### 1.4 Add the remaining members to the replica set.\n\n``` bash\nrs.add(\"127.0.0.1:27018\")\nrs.add(\"127.0.0.1:27019\")\n```\n\nthrough `rs.status` we can see the status of the replica set\n\n``` bash\nrs0:PRIMARY\u003e rs.status()\n{\n        \"set\" : \"rs0\",\n        \"date\" : ISODate(\"2016-05-22T04:45:22.195Z\"),\n        \"myState\" : 1,\n        \"term\" : NumberLong(3),\n        \"heartbeatIntervalMillis\" : NumberLong(2000),\n        \"members\" : [\n                {\n                        \"_id\" : 0,\n                        \"name\" : \"127.0.0.1:27017\",\n                        \"health\" : 1,\n                        \"state\" : 1,\n                        \"stateStr\" : \"PRIMARY\",\n                        \"uptime\" : 166805,\n                        \"optime\" : {\n                                \"ts\" : Timestamp(1463737764, 8),\n                                \"t\" : NumberLong(3)\n                        },\n                        \"optimeDate\" : ISODate(\"2016-05-20T09:49:24Z\"),\n                        \"electionTime\" : Timestamp(1463725527, 1),\n                        \"electionDate\" : ISODate(\"2016-05-20T06:25:27Z\"),\n                        \"configVersion\" : 3,\n                        \"self\" : true\n                },\n                {\n                        \"_id\" : 1,\n                        \"name\" : \"127.0.0.1:27018\",\n                        \"health\" : 1,\n                        \"state\" : 2,\n                        \"stateStr\" : \"SECONDARY\",\n                        \"uptime\" : 166799,\n                        \"optime\" : {\n                                \"ts\" : Timestamp(1463737764, 8),\n                                \"t\" : NumberLong(3)\n                        },\n                        \"optimeDate\" : ISODate(\"2016-05-20T09:49:24Z\"),\n                        \"lastHeartbeat\" : ISODate(\"2016-05-22T04:45:20.570Z\"),\n                        \"lastHeartbeatRecv\" : ISODate(\"2016-05-22T04:45:20.570Z\"),\n                        \"pingMs\" : NumberLong(0),\n                        \"syncingTo\" : \"127.0.0.1:27019\",\n                        \"configVersion\" : 3\n                },\n                {\n                        \"_id\" : 2,\n                        \"name\" : \"127.0.0.1:27019\",\n                        \"health\" : 1,\n                        \"state\" : 2,\n                        \"stateStr\" : \"SECONDARY\",\n                        \"uptime\" : 166794,\n                        \"optime\" : {\n                                \"ts\" : Timestamp(1463737764, 8),\n                                \"t\" : NumberLong(3)\n                        },\n                        \"optimeDate\" : ISODate(\"2016-05-20T09:49:24Z\"),\n                        \"lastHeartbeat\" : ISODate(\"2016-05-22T04:45:21.356Z\"),\n                        \"lastHeartbeatRecv\" : ISODate(\"2016-05-22T04:45:21.356Z\"),\n                        \"pingMs\" : NumberLong(0),\n                        \"syncingTo\" : \"127.0.0.1:27017\",\n                        \"configVersion\" : 3\n                }\n        ],\n        \"ok\" : 1\n}\nrs0:PRIMARY\u003e \n```\n\n### 1.5 Add authentication\n\n1) Create the keyfile your deployment will use to authenticate to members to each other.\n\n``` bash\n$ openssl rand -base64 741 \u003e /home/mongodb/mongodb-keyfile\n$ chmod 600 mongodb-keyfile\n```\n\t\n2)Enable authentication for each member of the sharded cluster or replica set.\n\nIn each replica member's configure file, please add this:\n\n```\nsecurity:\n  keyFile: /home/mongodb/mongodb-keyfile\n```\nIf the replica members are in different machines, Pleas copy the key file to the host machine where the replica member located in.\n\n3) Create user administrator\n\n```\nuse admin\ndb.createUser(\n  {\n    user: \"myUserAdmin\",\n    pwd: \"abc123\",\n    roles: [ { role: \"userAdminAnyDatabase\", db: \"admin\" } ]\n  }\n)\n```\n\n4) Restart all mongodb instances\n\n```\n$ mongo\nMongoDB shell version: 3.2.6\nconnecting to: test\nrs0:PRIMARY\u003e rs.status()\n{\n        \"ok\" : 0,\n        \"errmsg\" : \"not authorized on admin to execute command { replSetGetStatus: 1.0 }\",\n        \"code\" : 13\n}\nrs0:PRIMARY\u003e use admin\nswitched to db admin\nrs0:PRIMARY\u003e db.auth(\"myUserAdmin\",\"abc123\")\n1\n```\n\n## 2 Testing\n\n### 2.1 Test case 1: Basic operations\n\nWe use [pymongo](http://api.mongodb.org/python/current/examples/high_availability.html) to do some basic testing.\n\n![Failover](_image/election.png)\n\nConnecting to a Replica Set\n\n```python\n\u003e\u003e\u003e from pymongo import MongoClient\n\u003e\u003e\u003e MongoClient(\"10.75.44.10\", replicaset='rs1')\nMongoClient([u'mongodb2:27017', u'mongodb1:27017', u'mongodb3:27017'])\n\u003e\u003e\u003e \n```\n\nWrite operation：\n\n``` python\n\u003e\u003e\u003e from pymongo import MongoClient\n\u003e\u003e\u003e db = MongoClient(\"127.0.0.1\", replicaset='rs1').demo\n\u003e\u003e\u003e db\nDatabase(MongoClient([u'mongodb2:27017', u'mongodb1:27017', u'mongodb3:27017']), u'demo')\n\u003e\u003e\u003e db.connection.host\n'10.75.44.10'\n\u003e\u003e\u003e db.connection.port\n27017\n\u003e\u003e\u003e\n```\n\n看到目前对于数据库的操作是`PRIMARY`，也就是host `mongodb1`。 然后对数据库写入一条record：\n\n```python\n\u003e\u003e\u003e db.test.insert({'x':1})\nObjectId('54b8b1a1c77b3b3b354869a3')\n\u003e\u003e\u003e db.test.find_one()\n{u'x': 1, u'_id': ObjectId('54b8b1a1c77b3b3b354869a3')}\n\u003e\u003e\u003e \n```\n\n此时，把`mongodb1`的mongod stop掉。\n\n```python\n\u003e\u003e\u003e db.test.find_one()\nTraceback (most recent call last):\n  File \"\u003cstdin\u003e\", line 1, in \u003cmodule\u003e\n  File \"/usr/local/lib/python2.7/dist-packages/pymongo/collection.py\", line 713, in find_one\n    for result in cursor.limit(-1):\n  File \"/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py\", line 1038, in next\n    if len(self.__data) or self._refresh():\n  File \"/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py\", line 982, in _refresh\n    self.__uuid_subtype))\n  File \"/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py\", line 906, in __send_message\n    res = client._send_message_with_response(message, **kwargs)\n  File \"/usr/local/lib/python2.7/dist-packages/pymongo/mongo_client.py\", line 1186, in _send_message_with_response\n    sock_info = self.__socket(member)\n  File \"/usr/local/lib/python2.7/dist-packages/pymongo/mongo_client.py\", line 913, in __socket\n    \"%s %s\" % (host_details, str(why)))\npymongo.errors.AutoReconnect: could not connect to 10.75.44.10:27017: [Errno 111] Connection refused\n\u003e\u003e\u003e db.test.find_one()\n{u'x': 1, u'_id': ObjectId('54b8b1a1c77b3b3b354869a3')}\n\u003e\u003e\u003e db.connection.host\nu'mongodb3'\n\u003e\u003e\u003e db.connection.port\n27017\n\u003e\u003e\u003e \n```\n\n发现有个`pymongo.errors.AutoReconnect`的异常，不过马上恢复了，而且此时的操作数据库变成了`mongodb3`，也就是现在的`PRIMARY`.\n\n如果需要从Secondary读取数据，可以设置`ReadPreference`.\n``` python\n\u003e\u003e\u003e from pymongo.read_preferences import ReadPreference\n\u003e\u003e\u003e db.test.find_one()\n{u'x': 1, u'_id': ObjectId('54b8b1a1c77b3b3b354869a3')}\n\u003e\u003e\u003e db.test.find_one(read_preference=ReadPreference.SECONDARY)\n{u'x': 1, u'_id': ObjectId('54b8b1a1c77b3b3b354869a3')}\n\u003e\u003e\u003e \n```\n\n\n### 2.2 Test case 2: PRIMARY lost connection with all SECONDARY\n\n假如PRIMARY和其它所有的SECONDARY失去联系了，那么PRIMARY就无法进行读写操作了。\n\n```python\n\u003e\u003e\u003e db.test.insert({'x':3})\nTraceback (most recent call last):\n  File \"\u003cstdin\u003e\", line 1, in \u003cmodule\u003e\n  File \"/usr/local/lib/python2.7/dist-packages/pymongo/collection.py\", line 402, in insert\n    gen(), check_keys, self.uuid_subtype, client)\n  File \"/usr/local/lib/python2.7/dist-packages/pymongo/mongo_client.py\", line 1125, in _send_message\n    raise AutoReconnect(str(e))\npymongo.errors.AutoReconnect: not master\n\u003e\u003e\u003e \n\u003e\u003e\u003e \n\u003e\u003e\u003e \n\u003e\u003e\u003e \n\u003e\u003e\u003e db.test.insert({'x':3})\nTraceback (most recent call last):\n  File \"\u003cstdin\u003e\", line 1, in \u003cmodule\u003e\n  File \"/usr/local/lib/python2.7/dist-packages/pymongo/collection.py\", line 363, in insert\n    client._ensure_connected(True)\n  File \"/usr/local/lib/python2.7/dist-packages/pymongo/mongo_client.py\", line 924, in _ensure_connected\n    self.__ensure_member()\n  File \"/usr/local/lib/python2.7/dist-packages/pymongo/mongo_client.py\", line 797, in __ensure_member\n    member, nodes = self.__find_node()\n  File \"/usr/local/lib/python2.7/dist-packages/pymongo/mongo_client.py\", line 888, in __find_node\n    raise AutoReconnect(', '.join(errors))\npymongo.errors.AutoReconnect: [Errno 111] Connection refused, [Errno 111] Connection refused, mongodb3:27017 is not primary or master\n\u003e\u003e\u003e \n```\n\n直到有至少两个Replica Set的host连接，然后选出新的PRIMARY。\n\n### 2.3 Test case 3: Write Concern\n\n根据[Write Concern for Replica Sets](http://docs.mongodb.org/manual/core/replica-set-write-concern/)的介绍：\n\n![WriteConcern](_image/writeconcern.png)\n\n[pymongo write concern](http://api.mongodb.org/python/current/api/pymongo/database.html#pymongo.database.Database.write_concern)\n\n```python\n\u003e\u003e\u003e db\nDatabase(MongoClient([u'mongodb2:27017', u'mongodb1:27017', u'mongodb3:27017']), u'demo')\n\u003e\u003e\u003e db.write_concern\n{}\n\u003e\u003e\u003e db.write_concern = {'w':2, 'wtimeout':5000}\n\u003e\u003e\u003e db.write_concern\n{'wtimeout': 5000, 'w': 2}\n\u003e\u003e\u003e db.test.insert({'y':2})\nObjectId('54b8b89dc77b3b3b354869a4')\n```\n\n默认的write concern是空的配置。write concern有四个参数：`w`,`wtimeout`,`j`, `fsync`。\n\n其中比较重要的是`w`和`wtimeout`。\n\n`w`: (integer or string)If this is a replica set, write operations will block until they have been replicated to the specified \nnumber or tagged set of servers. w=\u003cint\u003e always includes the replica set primary (e.g. w=3 means write to \nthe primary and wait until replicated to two secondaries). Setting w=0 disables write acknowledgement and \nall other write concern options.\n\n`wtimeout`: (integer) Used in conjunction with w. Specify a value in milliseconds to control how long to wait for write \npropagation to complete. If replication does not complete in the given timeframe, a timeout exception is raised.\n\n\n## Reference\n\n[Three Member Replica Sets from mongodb.org](http://docs.mongodb.org/manual/core/replica-set-architecture-three-members/)\n\n[http://docs.mongodb.org/manual/replication/](http://docs.mongodb.org/manual/replication/)\n\n[How to Setup MongoDB Replication Using Replica Set and Arbiters](http://www.thegeekstuff.com/2014/02/mongodb-replication/)\n\n[pymongo documentation](http://api.mongodb.org/python/current/api/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxiaopeng163%2FMongodb-Replication-Test","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxiaopeng163%2FMongodb-Replication-Test","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxiaopeng163%2FMongodb-Replication-Test/lists"}