https://github.com/aman-codes/scrapemaster
A web scraping tool that shows cumulative ratings and ranking from online coding platforms. Built using node.js and puppeteer
https://github.com/aman-codes/scrapemaster
codechef hackerrank nodejs puppeteer scraper webscraping
Last synced: 1 day ago
JSON representation
A web scraping tool that shows cumulative ratings and ranking from online coding platforms. Built using node.js and puppeteer
- Host: GitHub
- URL: https://github.com/aman-codes/scrapemaster
- Owner: Aman-Codes
- License: mit
- Created: 2021-02-03T06:39:07.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2021-02-04T10:20:30.000Z (over 4 years ago)
- Last Synced: 2024-05-02T04:32:31.745Z (about 1 year ago)
- Topics: codechef, hackerrank, nodejs, puppeteer, scraper, webscraping
- Language: JavaScript
- Homepage:
- Size: 43 KB
- Stars: 4
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: docs/CONTRIBUTING.md
- License: LICENSE
- Code of conduct: docs/CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# ScrapeMaster
[](https://github.com/prettier/prettier)
ScrapeMaster is a web scrapper which scrapes various online platforms. It is built using node.js and puppeteer.
## Getting Started
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.
### Prerequisites
Your machine should have npm (or yarn) and Node.js installed to use it locally.
## Setup and Installation
### Setting up the repository locally
1. First fork the repo to your account.
Go to the forked repo and clone it to your local machine:
```sh
git clone https://github.com/Your_Username/ScrapeMaster.git
```
This will make a copy of the code to your local machine.
2. Now move to the `ScrapeMaster` directory.
```sh
cd ScrapeMaster
```
3. Now check the remote of your local code by:
```sh
git remote -v
```
The response should look like:
```sh
origin https://github.com/Your_Username/ScrapeMaster.git (fetch)
origin https://github.com/Your_Username/ScrapeMaster.git (push)
```
To add upstream to remote, run:
```sh
git remote add upstream https://github.com/Aman-Codes/ScrapeMaster.git
```
Again run `git remote -v`, the response should look like:
```sh
origin https://github.com/Your_Username/ScrapeMaster.git (fetch)
origin https://github.com/Your_Username/ScrapeMaster.git (push)
upstream https://github.com/Aman-Codes/ScrapeMaster (fetch)
upstream https://github.com/Aman-Codes/ScrapeMaster (push)
```
4. Once the remote is set, install all the necessary dependencies by the following command:
```sh
npm install
```
### Run locally
Run the below command to start the server:
```sh
npm run dev
```
### Contributing
We really like contributions in several forms, see [CONTRIBUTING.md](https://github.com/Aman-Codes/ScrapeMaster/blob/main/docs/CONTRIBUTING.md) and [CODE_OF_CONDUCT.md](https://github.com/Aman-Codes/ScrapeMaster/blob/main/docs/CODE_OF_CONDUCT.md)