{"id":22725638,"url":"https://github.com/livetocode/mongodb_nodejs_bug_01","last_synced_at":"2025-03-29T23:42:52.581Z","repository":{"id":76004173,"uuid":"556509317","full_name":"livetocode/mongodb_nodejs_bug_01","owner":"livetocode","description":"Bug report: occasional loss of connectivity with a replicaset of 3 nodes, after a primary node change","archived":false,"fork":false,"pushed_at":"2022-11-25T17:06:53.000Z","size":227,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-05T01:31:23.213Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/livetocode.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-24T01:40:34.000Z","updated_at":"2022-11-15T03:03:17.000Z","dependencies_parsed_at":"2023-07-03T21:25:30.322Z","dependency_job_id":null,"html_url":"https://github.com/livetocode/mongodb_nodejs_bug_01","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/livetocode%2Fmongodb_nodejs_bug_01","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/livetocode%2Fmongodb_nodejs_bug_01/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/livetocode%2Fmongodb_nodejs_bug_01/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/livetocode%2Fmongodb_nodejs_bug_01/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/livetocode","download_url":"https://codeload.github.com/livetocode/mongodb_nodejs_bug_01/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246258862,"owners_count":20748573,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-10T16:13:26.206Z","updated_at":"2025-03-29T23:42:52.574Z","avatar_url":"https://github.com/livetocode.png","language":"Shell","readme":"# Bug report\n\n## Title\n\nOccasional loss of connectivity with a replicaset of 3 nodes, after a primary node change\n\n## Details\n\nIn some rare cases, the MongoDB driver for NodeJS is not able to \"survive\" a change of the primary node, in the MongoDB cluster.\n\nThe following test cases will demonstrate the issue but you might have to run it for hours before it gets triggered!\n\nApparently, after a topology change of type ReplicaSetNoPrimary, the driver would not trigger a ReplicaSetWithPrimary and thus would miss the currently active primary node and the MongoDB queries would fails with the following errors:\n- Server selection timed out after ...\n- not primary and secondaryOk=false\n\nThis is a regression introduced in version 4.7 of the NodeJS MongoDB driver.\n\n### Fix\n\nThe MongoDB team acknowledged this issue by releasing the new [4.12.0](https://github.com/mongodb/node-mongodb-native/releases/tag/v4.12.0) version, which contains the following statement:\n\nVersion 4.7.0 of the Node driver released an improvement to our server monitoring in FAAS environments by allowing the driver to skip monitoring events if there were more than one monitoring events in the queue when the monitoring code restarted. When skipping monitoring events that contained a topology change, the driver would incorrectly fail to update its view of the topology.\n\nVersion 4.12.0 fixes this issue by ensuring that the topology is always updated when monitoring events are processed.\n\n\n## Test\n\nWe made a test suite to demonstrate the issues.\n\nWe deploy a MongoDB cluster configured as a Replicaset of 3 nodes, using the Percona operator for Kubernetes.\n\nThen we run a chaos script that will change the version of the cluster every 10 minutes, in order to force a change of the primary node while upgrading the nodes.\n\nIn parallel, we run several tests cases that will issue 2 queries per second to the server.\nWe have a test case for each version of the MongoDB driver that we want to study, and we test both NodeJS and Python in order to demonstrate that this is not a bug related to MongoDB but to specific versions of its NodeJS drivers.\n\nFinally, there is a Grafana dashboard to observe the results in live and compare the performance of those test cases.\n\n### Requirements\n\n#### Tools\n\nRun the following bash script to verify that the required tools are properly installed:\n\n`bash scripts/check-prerequisites.sh`\n\n#### Cluster\n\nYou need a kubernetes cluster with at least 3 nodes and for each node 8 GB of Ram and 2 vCPUs.\n\nYou can provision such a cluster in GCP with GKE, or any other cloud provider.\n\nYou could also run your own cluster on your machine as long as you have 3 nodes (see k3d/k3s for instance).\n\nMake sure the current configuration for the kubectl command points to the right cluster.\n\n### Configuration\n\nYou can edit the file scripts/config.sh to adjust some options. \n\n### Provisioning\n\nRun the following script in order to provision the required services, such as MongoDB, Prometheus and Grafana:\n\n`bash scripts/provision.sh`\n\n### Deployment\n\nRun the following script in order to deploy the different test cases in the cluster:\n\n`bash scripts/deploy.sh`\n\n### Dashboard\n\nYou can observe the results in live using the provided Grafana dashboard.\nJust run this script and it will open a tunnel to the Grafana app and open it in your browser.\nThe username is 'admin' and the password is also 'admin'.\n\n`bash scripts/open-dashboard.sh`\n\n## Build\n\nIf you want to rebuild the images, you can change the DOCKER_ORG variable in the scripts/config.sh file to use your own account.\n\nThen just run this script:\n\n`bash scripts/build.sh`\n\nNote that this script will build the images using two platforms (Arm64 and Amd64), to support both Apple M1 chips and regular AMD chips.\n\nIf you don't want Arm64, edit the scripts/utils.sh file (and the buildExampleImage function).\nThere is another build script to update in tools/mongo-chaos/scripts/build.sh\n\n## Run locally\n\nIf you want to test or debug an example, then cd to the right folder, install the dependencies and run it.\n\nAlso, make sure that you define the MONGO_URL environment variable to point to your MongoDB cluster. Your connection string should specify the replicaset nodes, the DB name and the user credentials.\n\nFor example, for running mongodb-4.7.0:\n```shell\ncd examples/nodejs/mongodb-4.7.0\nnpm install\nexport MONGO_URL=...\nnpm start\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flivetocode%2Fmongodb_nodejs_bug_01","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flivetocode%2Fmongodb_nodejs_bug_01","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flivetocode%2Fmongodb_nodejs_bug_01/lists"}