https://github.com/datadiggerjay/loadmore-web-scrapper
This repository contains a Selenium & BeautifulSoup-based web scraper designed to extract dynamically loaded content from websites with "Load More" buttons. It efficiently clicks the button, scrapes articles, and saves the data to an Excel file, ensuring incremental backups to prevent data loss .
https://github.com/datadiggerjay/loadmore-web-scrapper
beautifulsoup loadmore python selenium web-scraping
Last synced: 6 months ago
JSON representation
This repository contains a Selenium & BeautifulSoup-based web scraper designed to extract dynamically loaded content from websites with "Load More" buttons. It efficiently clicks the button, scrapes articles, and saves the data to an Excel file, ensuring incremental backups to prevent data loss .
- Host: GitHub
- URL: https://github.com/datadiggerjay/loadmore-web-scrapper
- Owner: DataDiggerJay
- License: mit
- Created: 2025-02-03T10:40:18.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2025-02-03T11:12:50.000Z (8 months ago)
- Last Synced: 2025-02-03T11:38:21.674Z (8 months ago)
- Topics: beautifulsoup, loadmore, python, selenium, web-scraping
- Language: Python
- Homepage: https://github.com/DataDiggerJay/Loadmore-Web-scrapper
- Size: 0 Bytes
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0