https://github.com/jacknormand/tfrrs-topqualifer

A data collection program for college track
https://github.com/jacknormand/tfrrs-topqualifer

beautifulsoup postgresql python webscraping

Last synced: 12 days ago
JSON representation

A data collection program for college track

• Host: GitHub
• URL: https://github.com/jacknormand/tfrrs-topqualifer
• Owner: jacknormand
• License: apache-2.0
• Created: 2023-12-27T19:38:33.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2024-03-18T17:15:59.000Z (10 months ago)
• Last Synced: 2024-11-11T09:18:32.646Z (2 months ago)
• Topics: beautifulsoup, postgresql, python, webscraping
• Language: Python
• Homepage:
• Size: 43.9 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Overview

A python program that grabs select data from TFRRS.com (online track & field database) and stores in a PostgresSQL database. A database.ini file is used to read in credentials. Collects the top 100 marks for each event for a given range of indoor/outdoor seasons (startyear-endyear). Ranges from 2012 to current marks. Formats and stores rows in a very functional format. BeauitfulSoup and PostgreSQL used. Loading animation and progress bar shown below. Also has several mechanisms for preventing invalid rows (runs queries to remove errors and skips over bad data).

![running](https://github.com/jacknormand/TFRRS-TopQualifer/assets/21299000/c6669730-820b-4fa6-a97e-84a5be6e6790)

# Usage

'python3 scrape.py 2012 2024' 


Supports scraping just one year also, just make startyear = endyear 


**ONLY WORKS FROM 2012-ON**


# Disclaimer

This project is for educational purposes only. Please do not use irresponsibly.