https://github.com/soumil-biswas/amazon-web-scraper-selenium
An web scraping bot for retrieving mobile phone prices. Built using Python, selenium and BeautifulSoup
https://github.com/soumil-biswas/amazon-web-scraper-selenium
beautifulsoup4 python selenium web-scraping
Last synced: 2 months ago
JSON representation
An web scraping bot for retrieving mobile phone prices. Built using Python, selenium and BeautifulSoup
- Host: GitHub
- URL: https://github.com/soumil-biswas/amazon-web-scraper-selenium
- Owner: Soumil-Biswas
- Created: 2025-03-15T04:35:20.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-03-15T05:41:52.000Z (over 1 year ago)
- Last Synced: 2025-03-15T06:28:14.820Z (over 1 year ago)
- Topics: beautifulsoup4, python, selenium, web-scraping
- Language: Python
- Homepage:
- Size: 8.6 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Amazon-Web-Scraper-Selenium
An web scraping bot for retrieving mobile phone prices. Built using Python, selenium and BeautifulSoup
# Dependencies
The following python dependencies need to be installed before this script can be run:
1. selenium
2. bs4
3. pandas
# How to run
Run any one among `main_Selenium.py` , `main_BeutifulSoup.py` , `main_Selenium_Stage_2.py` in a python IDE of your choice.
`main_Selenium.py` scrapes a single page using Selenium and saves the records in `amazon_mobiles_Selenium.csv`. The search Query and price range is hardcoded into the amazon.in search URL.
`main_BeutifulSoup.py` scrapes a single page using Selenium, parses the information using BeautifulSoup (bs4) and saves the records in `amazon_mobiles_BeutifulSoup.csv`. The search Query and price range is hardcoded into the amazon.in search URL.
`main_Selenium_Stage_2.py` scrapes 20 pages using Selenium and BeutifulSoup saves the records in `amazon_mobiles_Selenium_Stage_2.csv`. The bot manually searches for mobile phones using Amazon's search bar and sets the price range in Amazon's price range slider.
