{"id":19044097,"url":"https://github.com/himanshub16/mongodb-notifications-server","last_synced_at":"2026-05-06T03:34:52.774Z","repository":{"id":112516797,"uuid":"87727693","full_name":"himanshub16/mongodb-notifications-server","owner":"himanshub16","description":"Demonstration of real-time notifications based on changes in a mongodb collection.","archived":false,"fork":false,"pushed_at":"2017-05-29T09:53:19.000Z","size":15,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-25T22:53:06.340Z","etag":null,"topics":["database","mongodb","nodejs","oplog","replica-set"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/himanshub16.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2017-04-09T17:52:06.000Z","updated_at":"2023-09-08T17:23:26.000Z","dependencies_parsed_at":"2023-05-04T13:48:41.772Z","dependency_job_id":null,"html_url":"https://github.com/himanshub16/mongodb-notifications-server","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/himanshub16/mongodb-notifications-server","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/himanshub16%2Fmongodb-notifications-server","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/himanshub16%2Fmongodb-notifications-server/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/himanshub16%2Fmongodb-notifications-server/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/himanshub16%2Fmongodb-notifications-server/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/himanshub16","download_url":"https://codeload.github.com/himanshub16/mongodb-notifications-server/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/himanshub16%2Fmongodb-notifications-server/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32677930,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-06T02:33:58.958Z","status":"ssl_error","status_checked_at":"2026-05-06T02:33:39.611Z","response_time":117,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","mongodb","nodejs","oplog","replica-set"],"created_at":"2024-11-08T22:44:50.111Z","updated_at":"2026-05-06T03:34:52.756Z","avatar_url":"https://github.com/himanshub16.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Realtime MongoDB database monitoring\n#### Author: Himanshu Shekhar\n---\n\n* [Development platform](#development-platform)\n* [Running TLDR](#tldr)\n* [Running Detailed description](#detailed-description)\n* [Problem solving tactics](#the-problems)\n* [Code Structure](#code-structure)\n* [References](#references)\n\n### Development platform\n| Operating system | Linux |\n| Distribution | Ubuntu 16.04 (KDE) |\n| Node Version | 6.10.2 |\n| MongoDB Version | 3.4.3 |\n| Architecture | 64 bit Intel |\n\n### Preparing to run\n#### TLDR\nInside the folder having this code\n```\nmongoimport --db test --collection subscribers --drop --file subscribers.json\nmongoimport --db test --collection people --drop --file people.json\nnpm install\nsudo mongod --port 27017 --dbpath /var/lib/mongodb --replSet rs0\nStart mongo shell and type \"rs.initiate()\"\nnode index.js test\n```\nOpen http://localhost:4000/\n\n#### Detailed explanation\n* Sample datasets are available in files *people.json*, and *subscribers.json*.\n* Use the following commands to import the collections to MongoDB. You can change the name of database and collection as required, but remember to make the corresponding changes in *configs.js*.\n\n```\nmongoimport --db test --collection subscribers --drop --file subscribers.json\nmongoimport --db test --collection people --drop --file people.json\n```\n* Install required node modules using *package.json*.\n```\nnpm install\n```\n* Start your mongodb instances with a replication instance. Make sure the **dbpath** matches your installation.\n  Any change in configuration needs to made in **configs.js**.\n```\nsudo mongod --port 27017 --dbpath /var/lib/mongodb --replSet rs0\n```\n* Initiate the replication from *mongoclient*.\n```\nmongo\nand then type\nrs.initiate()\n```\n* Start the HTTP server using index.js, which will run on port 4000 by default.\n```\nnode index.js\n```\n* To use the automated test,\n```\nnode index.js test\n```\n* Open `localhost:4000` to see the events as database changes.\n\n---\n\n## How I figured out the to this problem?\n\n### The problems\n* **Triggers**: MongoDB doesn't provide triggers, which can be used to handle events like database updates.\n  **Solution**: However, Mongo has something called *tailable cursors*, which are similar to *tail -f* on Unix.\n                But, the [official docs](https://docs.mongodb.com/manual/core/tailable-cursors/) doesn't recommend tailable cursors for high volume of writes. Rather, they provide a method of replication of databases, and *oplogs*, which is kinda log of whatever query is made. One can smartly utilize this log in his/her interest.\n\n    For NodeJS, there were many oplog watchers already available, of which some were obsolete, and some were poorly documented and unreliable. This reduced the number of choices after trying most of them, making me finally step onto [mongo-oplog-watch](https://github.com/sachinb94/mongo-oplog-watch). It worked like charm, credits to proper documentation.\n\n\n* **Realtime communication**: The notifications need to be sent to clients in real time.\n  **Solution**: [Socket.IO](https://socket.io/) is the most reliable way in Node to handle real-time communication. \n                With plenty of examples available out there, it made the task easy.\n                But, we had to handle many socket connections. This was done by making a hashtable with userId as keys (easy to implement as JSON).\n                The references are properly mentioned in *references.txt*.\n\n\n\n### Code structure\nThe codebase is built upon **Node JS**. There are three main javascript files:\n1. **index.js** : Creates **ExpressJS** and **Socket.IO** server, sets up routes and handle interactions with the client.\n2. **notifier.js** : Handles the direct interaction with MongoDB instance, like subscribe, unsubscribe, watch a collection for changes, get user list from database, etc.\n3. **configs.js**: Has configuration variables so that configuration can be easily changed.\n\nThere are two collections.\n1. **People's collection**: This contains the objects for all users, and their basic data.\n2. **Subscriber's collection**: This contains the objects for all subscribers, so that other instance of this code can see which users are being served currently.\n\n\n### REFERENCES\n\n**expressjs and socketio references**\n* https://blog.joshsoftware.com/2012/01/30/push-notifications-using-express-js-and-socket-io/\n* https://scotch.io/tutorials/use-expressjs-to-get-url-and-post-parameters\n* https://blog.joshsoftware.com/2012/01/30/push-notifications-using-express-js-and-socket-io/\n* https://code.tutsplus.com/tutorials/real-time-chat-with-nodejs-socketio-and-expressjs--net-31708\n \n**setting up replication in mongodb and oplog**\n* https://docs.mongodb.com/getting-started/shell/import-data/\n* https://docs.mongodb.com/getting-started/shell/update/\n* https://www.tutorialspoint.com/mongodb/mongodb_replication.htm\n* http://stackoverflow.com/questions/22629462/does-mongodb-have-the-properties-such-as-trigger-and-procedure-in-a-relational-d\n* https://github.com/sachinb94/mongo-oplog-watch\n \n**XHR in plain javascript**\n* https://developer.mozilla.org/en/docs/Web/API/XMLHttpRequest/Using_XMLHttpRequest\n\n**Gossip girl characters for sample dataset**\n* https://en.wikipedia.org/wiki/List_of_Gossip_Girl_characters#Main","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhimanshub16%2Fmongodb-notifications-server","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhimanshub16%2Fmongodb-notifications-server","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhimanshub16%2Fmongodb-notifications-server/lists"}