{"id":30165246,"url":"https://github.com/agkloop/products-compare","last_synced_at":"2025-08-11T18:21:51.592Z","repository":{"id":120978670,"uuid":"326830443","full_name":"agkloop/products-compare","owner":"agkloop","description":null,"archived":true,"fork":false,"pushed_at":"2021-01-07T12:21:48.000Z","size":325,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-15T09:45:09.510Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/agkloop.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-01-04T23:14:29.000Z","updated_at":"2024-09-20T09:16:11.000Z","dependencies_parsed_at":null,"dependency_job_id":"254758d0-6b94-4101-9412-576ea5d7f3f1","html_url":"https://github.com/agkloop/products-compare","commit_stats":null,"previous_names":["agkloop/products-compare"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/agkloop/products-compare","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agkloop%2Fproducts-compare","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agkloop%2Fproducts-compare/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agkloop%2Fproducts-compare/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agkloop%2Fproducts-compare/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/agkloop","download_url":"https://codeload.github.com/agkloop/products-compare/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agkloop%2Fproducts-compare/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269933073,"owners_count":24499190,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-11T02:00:10.019Z","response_time":75,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-11T18:21:49.702Z","updated_at":"2025-08-11T18:21:51.555Z","avatar_url":"https://github.com/agkloop.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# products-comparison\n\n##### This project is composed of 2 microservices\n   * REST API (Reading ) + projection consumer that updates the db in a realtime fashion.\n   * Sources scraping and mapping \n   \n## REST API (Reading ) + projection consumer ##\n   built using reactor reactive stream on jvm  and high performance batch db writing using jdbc template api\n\n## Sources scraping and mapping ##\n is built using reactive kafka that integrate reactive streams with kafka API \n it's also use protobuf serialization protocol in the published messages\n and communication between the producer and consumer. \n   \n   \n\n## Following the reactive architecture the high level structure ##   \n![Image of Yaktocat](digram.png)\n\n\n\n## How it works ##\n1. in the Sources/dataProvider microservice we have a scheduled task that run every day or week for each source [here](com.product.all/sources/src/main/java/com/task/sources/AppStartupRunner.java)\n2. the Sources/dataProvider fetch and map the data to a common structure then push it to kafka\n3. in the Projection there is a reactive consumer running to consume and update the DB in a real time with backpressure applied.\n4. finally, the REST API part [here](#Try)\n\n\n\n\n\n### what is missing ###\n1. Separating the projection consumer to be a separate service using kafka-connect\n2. Using elastic search cluster as a primary source of data\n3. Using kubernetes  \n4. More test coverage \n\t   \n\n\n\n## Requirement ##\n* jave 15 \n* maven \n* Docker\n \n## Build ## \n 1. run `mvn clean package` to create the jar file\n 2. run ` docker-compose up --build`\n\n## Try \n \n 1. call  `${localhost/docker-machine-ip}:8080/category ` to list all the available categories\n 2. call  `${localhost/docker-machine-ip}:8080/category/{Category_id}/product/{Product_name} ` to list all the available products that have similar name with prices\n eg: call  `http://192.168.99.100/:8080/category/1/product/iphone `\n \n \n \n  \n   \n    \n    \n        \n     \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagkloop%2Fproducts-compare","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fagkloop%2Fproducts-compare","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagkloop%2Fproducts-compare/lists"}