{"id":24330910,"url":"https://github.com/code-jl/nfl-point-kicker-data-scraper","last_synced_at":"2025-09-06T11:45:16.606Z","repository":{"id":264402093,"uuid":"893270632","full_name":"Code-JL/NFL-Point-Kicker-Data-Scraper","owner":"Code-JL","description":"A Python-based web scraping toolkit that extracts and processes NFL kicking statistics from Pro-Football-Reference. This project automates the collection of comprehensive game data, with a particular focus on field goal attempts and environmental conditions.","archived":false,"fork":false,"pushed_at":"2025-01-05T22:44:47.000Z","size":362,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-27T23:51:04.537Z","etag":null,"topics":["automation","beautifulsoup","csv","data-analysis","data-collection","field-goals","football-statistics","kicking-stats","nfl","python","selenium","sports-analysis","statistics","weather-data","web-scraping"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Code-JL.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-24T01:20:06.000Z","updated_at":"2025-01-09T02:14:26.000Z","dependencies_parsed_at":"2025-03-11T11:19:27.913Z","dependency_job_id":"8126d6d6-9b26-4efe-9920-e4b7f3a2a85e","html_url":"https://github.com/Code-JL/NFL-Point-Kicker-Data-Scraper","commit_stats":null,"previous_names":["code-jl/nfl-point-kicker-data-scraper"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Code-JL%2FNFL-Point-Kicker-Data-Scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Code-JL%2FNFL-Point-Kicker-Data-Scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Code-JL%2FNFL-Point-Kicker-Data-Scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Code-JL%2FNFL-Point-Kicker-Data-Scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Code-JL","download_url":"https://codeload.github.com/Code-JL/NFL-Point-Kicker-Data-Scraper/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248863593,"owners_count":21174032,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation","beautifulsoup","csv","data-analysis","data-collection","field-goals","football-statistics","kicking-stats","nfl","python","selenium","sports-analysis","statistics","weather-data","web-scraping"],"created_at":"2025-01-18T01:15:36.209Z","updated_at":"2025-04-14T10:35:47.191Z","avatar_url":"https://github.com/Code-JL.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NFL Point Kicker Data Scraper\n\nA Python-based web scraping toolkit that extracts and processes NFL kicking statistics from Pro-Football-Reference. This project automates the collection of comprehensive game data, with a particular focus on field goal attempts and environmental conditions.\n\n## Overview\n\nThe NFL Point Kicker Data Scraper systematically collects and processes data through a pipeline of specialized Python scripts, each handling a specific aspect of the data collection and formatting process. The result is a series of well-structured CSV files ready for analysis.\n\n## Key Features\n\n- **Comprehensive Data Collection**\n  - Stadium information\n  - Roof type and playing surface\n  - Weather conditions (temperature, humidity, wind speed)\n  - Field goal attempts and success rates\n\n- **Intelligent Data Processing**\n  - Automatic rate limiting to prevent IP bans\n  - Incremental data saving\n  - Error handling and recovery\n  - Multiple output formats for different analysis needs\n\n- **Specialized Output Files**\n  - Complete dataset (NFLKicksInfo2000-2022.csv)\n  - Weather-specific analysis (NFLKicksInfo2000-2022WeatherSplit.csv)\n  - Dome-specific analysis (NFLKicksInfo2000-2022NonSplitClosedDome.csv)\n  - Wind chill analysis (NFLKicksInfo2000-2022WeatherSplitWindChill.csv)\n\n## Technical Architecture\n\nThe system operates through a series of specialized scripts:\n\n1. `FootballInfoScraper.py` - Initializes the scraping pipeline and collects game URLs\n2. `FootballGameInfoScraper.py` - Extracts detailed game data\n3. `GameUrlDateSeparator.py` - Processes game dates\n4. `DateToCSVFormat.py` - Standardizes date formats\n5. `TotalDataFormatting.py` - Creates the master dataset\n6. `SplitDataFormatting.py` - Generates specialized analysis files\n\n## Installation\n\n```bash\n# Clone the repository\ngit clone https://github.com/Code-JL/NFL-Point-Kicker-Data-Scraper.git\n\n# Navigate to project directory\ncd NFL-Point-Kicker-Data-Scraper\n\n# Install required packages\npip install -r requirements.txt\n```\n\n## Usage\n\nExecute the scripts in the following order:\n\n```bash\npython FootballInfoScraper.py\npython FootballGameInfoScraper.py\npython GameUrlDateSeparator.py\npython DateToCSVFormat.py\npython TotalDataFormatting.py\npython SplitDataFormatting.py\n```\n\nNote: `FootballGameInfoScraper.py` may take significant time to complete due to rate limiting.\n\n## Output Files\n\nThe scraper generates several CSV files optimized for different analysis scenarios:\n\n- `NFLKicksInfo2000-2022.csv` - Complete dataset\n- `NFLKicksInfo2000-2022WeatherSplit.csv` - Weather-specific analysis\n- `NFLKicksInfo2000-2022NonSplit.csv` - General analysis\n- `NFLKicksInfo2000-2022NonSplitClosedDome.csv` - Indoor game analysis\n- `NFLKicksInfo2000-2022WeatherSplitWindChill.csv` - Wind chill impact analysis\n\n## Data Fields\n\nEach record includes:\n- Game date and location\n- Stadium characteristics\n- Environmental conditions\n- Kicking performance metrics\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## Acknowledgments\n\n- Data sourced from [Pro-Football-Reference](https://www.pro-football-reference.com/)\n- Built with Python and BeautifulSoup4\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcode-jl%2Fnfl-point-kicker-data-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcode-jl%2Fnfl-point-kicker-data-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcode-jl%2Fnfl-point-kicker-data-scraper/lists"}