https://github.com/next-li/long-island-datasets

https://github.com/next-li/long-island-datasets

Last synced: 18 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/next-li/long-island-datasets
• Owner: next-LI
• Created: 2022-08-17T17:23:06.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2022-08-25T20:22:59.000Z (over 2 years ago)
• Last Synced: 2024-11-08T22:35:06.517Z (2 months ago)
• Language: Python
• Size: 25.4 KB
• Stars: 0
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# long-island-datasets

Scrapes data from several websites.

# Scrapers to be refactored

 - [x] cms-data.py

 - [x] cms-provider-data.py

 - [x] college-scorecard.py

 - [x] epa-echo.py

 - [x] epa-sems-envirofacts.py

 - [ ] usace-fuds-arcgis.py

 - [ ] suffolk-county-food-establishment-inspections.py

 - [ ] socrata.py

 - [ ] open-fda.py

 - [ ] nyscjc-determinations.py

 - [ ] ntsb-carol.py

 - [ ] nhtsa-fars.py

 - [x] irs_exempt_organizations.py

 - [x] hhs_oig_exclusions.py

 - [x] fhwa_nbi_arcgis.py

 - [x] fra_nsrt.py

 - [x] dol_osha.py

# To-do

 - move all configuration to config.py

 - assess if we require a wrapper for requests

 - refactor `li_scraper.py` itself

 - utility function to check for LI zips

 - possibly consolidate hhs, irs, nyscjc into a standardized config-based scraper

 - rename scrapers to be more pythonic