https://github.com/marcusjhang/scraper

Google Sheets -> Python Scraper -> Postgres DB -> React Frontend
https://github.com/marcusjhang/scraper

beautifulsoup googlesheetsapi nodejs postgresql reactjs

Last synced: 13 days ago
JSON representation

Google Sheets -> Python Scraper -> Postgres DB -> React Frontend

• Host: GitHub
• URL: https://github.com/marcusjhang/scraper
• Owner: marcusjhang
• Created: 2024-06-18T03:40:51.000Z (6 months ago)
• Default Branch: main
• Last Pushed: 2024-06-18T05:48:50.000Z (6 months ago)
• Last Synced: 2024-06-19T05:27:29.012Z (6 months ago)
• Topics: beautifulsoup, googlesheetsapi, nodejs, postgresql, reactjs
• Language: Python
• Homepage:
• Size: 5.63 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
## To run website locally

run npm install when needed

1. cd backend

2. node index.js 

- server is run on 5001

3. open new terminal

4. cd frontent 

5. npm start

- server is run on 300

## To run script that adds entries

1. cd scripts

2. source venv/bin/activate 

- activate virtual environment

3. pip install psycopg2 

- may not be needed

4. python add_to_db.py

- reset local environment, should see the updated version

may be needed after initialising virtual environment:

pip install google-api-python-client google-auth google-auth-oauthlib google-auth-httplib2

pip install requests beautifulsoup4