https://github.com/fahimfba/web-scraper

Extract data from websites using the web-scrapper. Made with nodejs, ExpressJS, axios & cheerio.
https://github.com/fahimfba/web-scraper

axios cheerio cheeriojs javascript js npm npm-package webscrape webscraping webscraping-data webscraping-search webscrapper

Last synced: 6 months ago
JSON representation

Extract data from websites using the web-scrapper. Made with nodejs, ExpressJS, axios & cheerio.

• Host: GitHub
• URL: https://github.com/fahimfba/web-scraper
• Owner: FahimFBA
• License: mit
• Created: 2021-09-27T16:51:08.000Z (about 4 years ago)
• Default Branch: main
• Last Pushed: 2024-09-13T09:44:57.000Z (about 1 year ago)
• Last Synced: 2025-03-27T14:54:59.290Z (6 months ago)
• Topics: axios, cheerio, cheeriojs, javascript, js, npm, npm-package, webscrape, webscraping, webscraping-data, webscraping-search, webscrapper
• Language: JavaScript
• Homepage: https://fahimfba.github.io/Web-Scraper/
• Size: 1.06 MB
• Stars: 4
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Web Scraper



⭐ the repo if you like this project 😀




You can check the live feed from [here](https://youtu.be/NvXpo41vNrQ) as well. 😀






What is a Web-Scraper?

According to [Wikipedia](https://en.wikipedia.org/wiki/Web_scraping), " Web scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites. The web scraping software may directly access the World Wide Web using the Hypertext Transfer Protocol or a web browser. "

# Used languages & framework:

![JavaScript](https://badges.aleen42.com/src/javascript.svg)

![nodejs](https://badges.aleen42.com/src/node.svg)

![ExpressJS](https://img.shields.io/badge/-ExpressJS-yellow)

![npm](https://badges.aleen42.com/src/npm.svg)

![axios](https://img.shields.io/badge/-axios-lightgrey)

![cheerio](https://img.shields.io/badge/-cheerio-lightgrey)

# Run the scrapper

- Clone the repository

    - Using SSH 

      ```

      git clone git@github.com:FahimFBA/Web-Scraper.git

      ```

    - Using HTTPS

      ```

      git clone https://github.com/FahimFBA/Web-Scraper.git

      ```

- Go to the Web-Scraper directory

```

cd Web-Scraper

```

- Run the project using the following command

```

npm run start

```

By default, it would scrap from The Guardian as I used [The Guardian](https://www.theguardian.com/international) to experiment with the web scrapper.

To experiment on different websites, change the url in the index.js and customize the class in the axios as well.

##  Output (Using VS Code)

![Output](img/output.png)

Special thanks goes to [Ania Kubów](https://www.youtube.com/channel/UC5DNytAJ6_FISueUfzZCVsw)