https://github.com/vanguardmaster01/price-tracking-web-scraper
Extract price from websites
https://github.com/vanguardmaster01/price-tracking-web-scraper
flask python scraping web-scraping
Last synced: about 2 months ago
JSON representation
Extract price from websites
- Host: GitHub
- URL: https://github.com/vanguardmaster01/price-tracking-web-scraper
- Owner: vanguardmaster01
- Created: 2023-11-10T16:55:42.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-11-10T16:57:47.000Z (over 2 years ago)
- Last Synced: 2025-01-08T17:23:25.799Z (over 1 year ago)
- Topics: flask, python, scraping, web-scraping
- Language: Python
- Homepage:
- Size: 200 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Project Information
This project provides a user interface to interact with an automated price tracking web scraper. Currently the tracker scrapes amazon.ca, but could be configured to scrape multiple sources.
## Libraries/Frameworks/Modules
This project uses:
- React
- Flask
- Playwright
- Bright Data (Web Scraping Browser)
## Using the Scraper
Install all dependencies, create the `auth.json` file, start the flask backend, run the react frontend and interact with the tool.
### auth.json
Fill in your [Bright Data Scraping Browser](https://brightdata.com/products/scraping-browser) credentials in a `backend/scraper/auth.json` file (see `auth_example.json`).
### Python Flask Backend
- `cd backend`
- `pip install -r requirements.txt`
- `playwright install`
- `python app.py` or `python3 app.py`
### Running the React Frontend
- `cd frontend`
- `npm i`
- `npm run start`
## Setting Up Automation
To automate the collection of prices from this software simply run the `scheduler/main.py` file at your desired increment while the python flask backend is running.
### Windows
I have created a simple `.bat` script called `run.bat` that you can schedule to execute using the Windows Task Scheduler that will automatically run the backend api and send the appropriate request to it.