{"id":18649811,"url":"https://github.com/ashwanthkumar/meghaduta","last_synced_at":"2025-11-05T10:30:31.452Z","repository":{"id":88757891,"uuid":"42767443","full_name":"ashwanthkumar/meghaduta","owner":"ashwanthkumar","description":"Notification system built for Amazon Hackathon (Sept 2015)","archived":false,"fork":false,"pushed_at":"2015-09-21T00:11:39.000Z","size":348,"stargazers_count":3,"open_issues_count":0,"forks_count":2,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-12-27T12:44:52.553Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ashwanthkumar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-09-19T09:33:27.000Z","updated_at":"2022-11-14T12:41:45.000Z","dependencies_parsed_at":"2023-06-18T05:15:48.965Z","dependency_job_id":null,"html_url":"https://github.com/ashwanthkumar/meghaduta","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashwanthkumar%2Fmeghaduta","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashwanthkumar%2Fmeghaduta/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashwanthkumar%2Fmeghaduta/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashwanthkumar%2Fmeghaduta/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ashwanthkumar","download_url":"https://codeload.github.com/ashwanthkumar/meghaduta/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239456408,"owners_count":19641843,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T06:40:44.970Z","updated_at":"2025-11-05T10:30:31.374Z","avatar_url":"https://github.com/ashwanthkumar.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Build Status](https://snap-ci.com/ashwanthkumar/meghaduta/branch/master/build_image)](https://snap-ci.com/ashwanthkumar/meghaduta/branch/master)\n\n# Meghaduta\n\n### Dependencies\n- JDK7\n- Redis\n\n### Design\nFile Changes are monitored through a Storm topology that propogates the events. \n![Meghaduta Storm Topology](https://raw.githubusercontent.com/ashwanthkumar/meghaduta/master/docs/storm_topology.png)\n\nThe event gets propagated through each stage where the item is built incrementally and finally the notifier is notified (pluggable).\n\n### Getting Started\n```bash\n$ mvn clean package\n\n# To start the storm Topology\n$ java -cp target/meghaduta-1.0.0-SNAPSHOT.jar meghaduta.storm.MeghaDutaTopology\n\n# To start the HTTP Service\n$java -cp target/meghaduta-1.0.0-SNAPSHOT.jar meghaduta.service.MeghaDutaService -server\n```\n\nThe HTTP service runs on port 8080 by default and you can query the data\n\n| API | Operation |\n| --- | --- |\n| /{id} | id - itemId to query for |\n\nSample output JSON would be\n```json\n{\n    \"publisher\": \"William Morrow\",\n    \"authors\": \"Levitt \u0026 Dubner\",\n    \"title\": \"freakonomics\",\n    \"list price\": \"7\",\n    \"release date\": \"20-09-2011\"\n}\n```\n\nSample notification-file.log\n```\n[1442738940456] Notifing subscriber-test about Event{itemId='13579', name='title', value='freakonomics', timestamp=1442738934000} from \nitem=Item{itemId='13579', lastUpdated=1442738934000, attributes={publisher=William Morrow, authors=Levitt \u0026 Dubner, title=freakonomics}}\n[1442738940462] Notifing subscriber1 about Event{itemId='13579', name='release date', value='20-09-2011', timestamp=1442738934000} from \nitem=Item{itemId='13579', lastUpdated=1442738934000, attributes={publisher=William Morrow, authors=Levitt \u0026 Dubner, title=freakonomics, list price=7, release date=20-09-2011}}\n[1442738940463] Notifing subscriber2 about Event{itemId='1248', name='list price', value='10', timestamp=1442738934000} from \nitem=Item{itemId='1248', lastUpdated=1442738934000, attributes={title=Effective Java, authors=Joshua Bloch, publisher=Addison-Wesley, list price=10}}\n```\n\n### Subscription Configuration\n```hocon\n# Add all your subscriptions in here. The format is as follows\n#  {\n#   notifier = \"whom should I notify?\"\n#   filters = [{\n#     name = \"attribute name\"\n#     type = \"type of the attribute\" # accepted values - int, string, date\n#     operator = \"eq\" # accepted values - eq, gt, lt, ge, le\n#     value = \"value of the attribute to check\"\n#   }]\n#  }\n#\n# You can specify multiple filters and by default we'll always AND them.\n#\n# FIXME - Move this to a store so that we can in-corporate updates while the system is running\n\nsubscriptions = [{\n  notifier = \"subscriber-test\"\n  filters = [{\n    name = \"title\"\n    type = \"string\"\n    operator = \"eq\"\n    value = \"freakonomics\"\n  }]\n}, {\n  notifier = \"subscriber1\"\n  filters = [{\n    name = \"release date\"\n    type = \"date\"\n    operator = \"gt\"\n    value = \"01-01-2000\"\n  }]\n}, {\n  notifier = \"subscriber2\"\n  filters = [{\n    name = \"publisher\"\n    type = \"string\"\n    operator = \"eq\"\n    value = \"Addison-Wesley\"\n  }, {\n    name = \"list price\"\n    type = \"int\"\n    operator = \"ge\"\n    value = \"10\"\n  }]\n}]\n# You can specify multiple filters and by default we'll always AND them.\n```\n\n### Configuration\n```\nmeghaduta {\n  # Store implementation - redis, rocksdb (experimental)\n  store-type = \"redis\"\n  db-location = \"localhost\"\n\n  # Relative to the project root - can also be an absolute path\n  shared-folder = \"shared-folder/IN\"\n\n  # Notifier implementation\n  notifier-type = \"file\"\n  notify-output-file = \"notifications.log\"\n}\n```\n\n\u003chr /\u003e\n\n### Problem Description\nThe goal is to build a configurable notification system, which receives inputs in a CSV file and notifies its subscribers based on the condition set.\n\n### System Requirements\n\n- The system should consume CSV files which are dropped into a shared folder which are of the form item,attribute_name,attribute_value. The CSV files will be dropped into the shared folder at any point in time.\n- The system should be able to notify the attribute changes for items to different subscribers based on different configurable conditions\n- The notifications can be sent on any asynchronous messaging system\n- The system should also be able to support queries to get all the attributes for a given item at any time\n- The system should be scalable to\n-- accept large number of/large sized CSV files\n-- support large number of subscriber configurations\n-- support high throughput and low latency on the get query\n\n### Sample Files:\n\n`file_1.csv`\n\n```csv\n13579,title,freakonomics\n13579,authors,Levitt \u0026 Dubner\n13579,release date,20-09-2011\n13579,list price,7 USD\n13579,publisher,William Morrow\n1248,title,Effective Java\n1248,authors,Joshua Bloch\n1248,list price,10 USD\n1248,publisher,Addison-Wesley\n```\n\n`file_2.csv`\n\n```csv\n13579, list price,11 USD\n```\n\n### Sample Subscriber Configurations\n\n#### Sample conditions are: \n```\nif release date \u003e= 01-01-2000, notify to subscriber1 \nif publisher is Addison-Wesley and list price is \u003e= 10, notify to subscriber2\n```\n\n#### Sample Query\n```\nGET 13579\n```\n\n##### Result Set:\n```json\ntitle=freakonomics\nauthors=Levitt \u0026 Dubner\nrelease date = 20-09-2011\nlist price=11 USD\npublisher=William Morrow\n````\n\n### Technology\n\nProgramming Language: Java, but using C#/ C++ is acceptable \n\nOther Technologies: AWS\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fashwanthkumar%2Fmeghaduta","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fashwanthkumar%2Fmeghaduta","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fashwanthkumar%2Fmeghaduta/lists"}