https://github.com/mayankagnihotri7/data-crawler
IMDB data scrapper for scrapping the top 100 movies from the website.
https://github.com/mayankagnihotri7/data-crawler
axios cheerio imdb-webscrapping nodejs react-hooks reactjs scrapper
Last synced: 6 months ago
JSON representation
IMDB data scrapper for scrapping the top 100 movies from the website.
- Host: GitHub
- URL: https://github.com/mayankagnihotri7/data-crawler
- Owner: mayankagnihotri7
- Created: 2020-10-26T05:43:48.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2021-08-19T15:47:20.000Z (about 4 years ago)
- Last Synced: 2025-02-15T18:52:18.480Z (8 months ago)
- Topics: axios, cheerio, imdb-webscrapping, nodejs, react-hooks, reactjs, scrapper
- Language: CSS
- Homepage:
- Size: 1.77 MB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Data-crawler
## Getting started
- Clone the repo using 'https://github.com/maxagno3/data-crawler.git'.
- Install the dependencies by going into respective folders using 'yarn install' or if you don't have yarn installed on your machine then delete the package-lock.json from the respective folders and install dependencies using 'npm i' or 'npm install'.
- Start the server by going into the client or server folder or both and use 'yarn start' or 'npm start'.
## Routes
Movie list can be seen on http://localhost:3000/ \
Route for the API is http://localhost:4000/ \
For Scrapping data from IMDB's website, you can enter http://localhost:4000/movie url on the browser, it will scrape data of 100 top movies and will save it into a JSON file.
## Library Used
- Axios ( https://www.npmjs.com/package/axios ).
- Cheerio ( https://www.npmjs.com/package/cheerio ).
- TailwindCSS ( https://tailwindcss.com/ ).
## Developer
#### Mayank Agnihotri
- Twitter: https://twitter.com/max_AgNO3
- LinkedIn: https://www.linkedin.com/in/mayank-agnihotri/