https://github.com/datadiggerjay/loadmore-web-scrapper
This repository contains a Selenium & BeautifulSoup-based web scraper designed to extract dynamically loaded content from websites with "Load More" buttons. It efficiently clicks the button, scrapes articles, and saves the data to an Excel file, ensuring incremental backups to prevent data loss .
https://github.com/datadiggerjay/loadmore-web-scrapper
beautifulsoup loadmore python selenium web-scraping
Last synced: 2 months ago
JSON representation
This repository contains a Selenium & BeautifulSoup-based web scraper designed to extract dynamically loaded content from websites with "Load More" buttons. It efficiently clicks the button, scrapes articles, and saves the data to an Excel file, ensuring incremental backups to prevent data loss .
- Host: GitHub
- URL: https://github.com/datadiggerjay/loadmore-web-scrapper
- Owner: DataDiggerJay
- License: mit
- Created: 2025-02-03T10:40:18.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-02-03T11:12:50.000Z (over 1 year ago)
- Last Synced: 2025-02-03T11:38:21.674Z (over 1 year ago)
- Topics: beautifulsoup, loadmore, python, selenium, web-scraping
- Language: Python
- Homepage: https://github.com/DataDiggerJay/Loadmore-Web-scrapper
- Size: 0 Bytes
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0