{"id":29188752,"url":"https://github.com/aman-codes/scrapemaster","last_synced_at":"2026-04-14T23:32:19.398Z","repository":{"id":54699333,"uuid":"335531550","full_name":"Aman-Codes/ScrapeMaster","owner":"Aman-Codes","description":"A web scraping tool that shows cumulative ratings and ranking from online coding platforms. Built using node.js and puppeteer","archived":false,"fork":false,"pushed_at":"2021-02-04T10:20:30.000Z","size":44,"stargazers_count":9,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-01T22:45:48.768Z","etag":null,"topics":["codechef","hackerrank","nodejs","puppeteer","scraper","webscraping"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Aman-Codes.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"docs/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"docs/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-02-03T06:39:07.000Z","updated_at":"2025-03-30T19:11:38.000Z","dependencies_parsed_at":"2022-08-14T00:20:44.525Z","dependency_job_id":null,"html_url":"https://github.com/Aman-Codes/ScrapeMaster","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Aman-Codes/ScrapeMaster","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aman-Codes%2FScrapeMaster","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aman-Codes%2FScrapeMaster/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aman-Codes%2FScrapeMaster/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aman-Codes%2FScrapeMaster/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Aman-Codes","download_url":"https://codeload.github.com/Aman-Codes/ScrapeMaster/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aman-Codes%2FScrapeMaster/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31819718,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-14T18:05:02.291Z","status":"ssl_error","status_checked_at":"2026-04-14T18:05:01.765Z","response_time":153,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["codechef","hackerrank","nodejs","puppeteer","scraper","webscraping"],"created_at":"2025-07-01T22:37:02.153Z","updated_at":"2026-04-14T23:32:19.380Z","avatar_url":"https://github.com/Aman-Codes.png","language":"JavaScript","readme":"# ScrapeMaster\n[![code style: prettier](https://img.shields.io/badge/code_style-prettier-ff69b4.svg?style=flat-square)](https://github.com/prettier/prettier)\n![MIT](https://img.shields.io/github/license/Aman-Codes/ScrapeMaster?color=blue)\n\nScrapeMaster is a web scrapper which scrapes various online platforms. It is built using node.js and puppeteer.\n\n## Getting Started\n\nThese instructions will get you a copy of the project up and running on your local machine for development and testing purposes.\n\n### Prerequisites\n\nYour machine should have npm (or yarn) and Node.js installed to use it locally.\n\n## Setup and Installation\n\n### Setting up the repository locally\n\n1. First fork the repo to your account.  \n   Go to the forked repo and clone it to your local machine:\n\n```sh\ngit clone https://github.com/Your_Username/ScrapeMaster.git\n```\n\nThis will make a copy of the code to your local machine.\n\n2. Now move to the `ScrapeMaster` directory.\n\n```sh\ncd ScrapeMaster\n```\n\n3. Now check the remote of your local code by:\n\n```sh\ngit remote -v\n```\n\nThe response should look like:\n\n```sh\norigin\thttps://github.com/Your_Username/ScrapeMaster.git (fetch)\norigin\thttps://github.com/Your_Username/ScrapeMaster.git (push)\n```\n\nTo add upstream to remote, run:\n\n```sh\ngit remote add upstream https://github.com/Aman-Codes/ScrapeMaster.git\n```\n\nAgain run `git remote -v`, the response should look like:\n\n```sh\norigin\thttps://github.com/Your_Username/ScrapeMaster.git (fetch)\norigin\thttps://github.com/Your_Username/ScrapeMaster.git (push)\nupstream\thttps://github.com/Aman-Codes/ScrapeMaster (fetch)\nupstream\thttps://github.com/Aman-Codes/ScrapeMaster (push)\n```\n\n4. Once the remote is set, install all the necessary dependencies by the following command:\n\n```sh\nnpm install\n```\n### Run locally\n\nRun the below command to start the server:\n\n```sh\nnpm run dev\n```\n\n### Contributing\n\nWe really like contributions in several forms, see [CONTRIBUTING.md](https://github.com/Aman-Codes/ScrapeMaster/blob/main/docs/CONTRIBUTING.md) and [CODE_OF_CONDUCT.md](https://github.com/Aman-Codes/ScrapeMaster/blob/main/docs/CODE_OF_CONDUCT.md)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faman-codes%2Fscrapemaster","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faman-codes%2Fscrapemaster","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faman-codes%2Fscrapemaster/lists"}