{"id":16193937,"url":"https://github.com/narayanbhat166/leetcode_datahouse","last_synced_at":"2025-03-19T04:30:44.893Z","repository":{"id":112262099,"uuid":"560519801","full_name":"Narayanbhat166/leetcode_datahouse","owner":"Narayanbhat166","description":"This project is to scrape leetcode database and gain useful insights","archived":false,"fork":false,"pushed_at":"2024-01-09T16:56:03.000Z","size":283,"stargazers_count":6,"open_issues_count":3,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-17T03:34:23.906Z","etag":null,"topics":["leetcode","rust","rust-lang"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Narayanbhat166.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-01T17:20:39.000Z","updated_at":"2024-10-03T07:22:48.000Z","dependencies_parsed_at":"2023-06-01T16:00:44.752Z","dependency_job_id":"897396ea-e6b7-43af-a052-68e72d2cc6d3","html_url":"https://github.com/Narayanbhat166/leetcode_datahouse","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Narayanbhat166%2Fleetcode_datahouse","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Narayanbhat166%2Fleetcode_datahouse/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Narayanbhat166%2Fleetcode_datahouse/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Narayanbhat166%2Fleetcode_datahouse/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Narayanbhat166","download_url":"https://codeload.github.com/Narayanbhat166/leetcode_datahouse/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244357400,"owners_count":20440320,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["leetcode","rust","rust-lang"],"created_at":"2024-10-10T08:17:06.485Z","updated_at":"2025-03-19T04:30:44.606Z","avatar_url":"https://github.com/Narayanbhat166.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# The Leetcode datahouse\n\nA highly distributed, robust, modular, scalable web data extractor. Bring your own machine to start contributing to the data.\n\nThis project is aimed to extract the data from leetcode for data analysis. A project for beginners in rust to get started to learn and contribute.\n\nServices\n- Producer: This service is what generates the data by extracting it from the web. It coordinates with the `Controller` extract the right data and push it to the controller. There can be multiple producers running at any given time. These are run on the client machines and are light weight.\n\n- Consumer: The service which pushes the data from a temporary storage or a message queue to a persistent storage. There will be only one consumer running at any given time.\n\n- Controller: Coordinates the producers to produce the right data. It stores the produced data in a temporary storage like a message queue or redis. \n\nThe communication between `Controller` and `Producer` happens by grpc.\n\n# Architecture\n![alt architecture diagram][diagram]\n\n[diagram]: https://github.com/Narayanbhat166/leetcode_datahouse/assets/48803246/d0224a74-25a1-49df-b4fa-8fb37d6e8cc1\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnarayanbhat166%2Fleetcode_datahouse","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnarayanbhat166%2Fleetcode_datahouse","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnarayanbhat166%2Fleetcode_datahouse/lists"}