https://github.com/ztf666/web-scraper

A small page scraper , NO DYNAMIC SCRAPING tho :tired_face:
https://github.com/ztf666/web-scraper

api-call axios axios-rest axios-restful cheerio page scraper scraperjs

Last synced: 6 months ago
JSON representation

A small page scraper , NO DYNAMIC SCRAPING tho :tired_face:

• Host: GitHub
• URL: https://github.com/ztf666/web-scraper
• Owner: ZTF666
• License: mit
• Created: 2020-04-23T02:13:40.000Z (about 6 years ago)
• Default Branch: master
• Last Pushed: 2021-05-11T03:15:09.000Z (almost 5 years ago)
• Last Synced: 2023-03-06T16:11:20.666Z (about 3 years ago)
• Topics: api-call, axios, axios-rest, axios-restful, cheerio, page, scraper, scraperjs
• Language: JavaScript
• Homepage:
• Size: 143 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: readme.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# 💩Scrapy💩

A small page scraper , still a WiP .

No dynamic scraping ...

This script uses :









Axios



## How to use

- **Install and run**

```javascript

npm install

```

```javascript

npm run scrapy

```

- **Change the website and add yours**

```javascript

axios.get("https://chouftv.ma/press");

```

- **Change the elements by the ones you desire**

```javascript

$(".description").each((index, element) => {

  const title = $(element).children().first().text();

  const links = $(element).children("a").attr("href");

});

```

![Screenshot](scr/res.png)

```

It looks weird because i used it on a local news website.

```

- **Limitations**

  

    

    
This is a shitty scrapper , i'm still learning.

    
It doesn't scrap unloaded links. 


  ![Screenshot](scr/lm.png)

In the screenshot above , the button litteraly translates to : LOAD MORE

Since i suck at this, i can't make it load more so i can grab the links

So it only grabs the latest news articles .

That's a blessing and a curse , beacause if clicked , it will load EVERY ARTICLE WRITTEN 

since the deployement of the website... 


## Contact

```

you can contact me at ZTF666@protonmail.ch

```



  

    
ZTF666

  



## License



**💩Scrapy💩** released under the [MIT](LICENSE) License.





Made with 💘 by a 👨‍💻 on a 💻 | 2020 | ZTF666 - N.EA