{"id":26153467,"url":"https://github.com/an4pdm/sportstats-pipeline","last_synced_at":"2026-04-16T19:39:53.161Z","repository":{"id":277597559,"uuid":"932937177","full_name":"An4PDM/SportStats-Pipeline","owner":"An4PDM","description":"SportStats Pipeline é um projeto de ETL que coleta, processa e armazena dados de eventos esportivos diários utilizando a API TheSportsDB. O pipeline é orquestrado com Apache Airflow e armazena os dados em um banco de dados MySQL.","archived":false,"fork":false,"pushed_at":"2025-02-14T19:54:06.000Z","size":0,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-14T20:29:03.598Z","etag":null,"topics":["airflow","airflow-dags","api","etl","mysql","pandas","pipeline","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/An4PDM.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-14T19:52:39.000Z","updated_at":"2025-02-14T19:54:09.000Z","dependencies_parsed_at":"2025-02-14T20:39:08.364Z","dependency_job_id":null,"html_url":"https://github.com/An4PDM/SportStats-Pipeline","commit_stats":null,"previous_names":["an4pdm/sportstats-pipeline"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/An4PDM%2FSportStats-Pipeline","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/An4PDM%2FSportStats-Pipeline/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/An4PDM%2FSportStats-Pipeline/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/An4PDM%2FSportStats-Pipeline/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/An4PDM","download_url":"https://codeload.github.com/An4PDM/SportStats-Pipeline/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242995913,"owners_count":20218828,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["airflow","airflow-dags","api","etl","mysql","pandas","pipeline","python"],"created_at":"2025-03-11T07:58:04.873Z","updated_at":"2025-10-03T23:08:07.373Z","avatar_url":"https://github.com/An4PDM.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SportStats Pipeline 📊 (in progress)\n\nSportStats Pipeline is a data pipeline project designed to collect, process, and store sports-related data. This project aims to automate the extraction and transformation of sports events, results, and statistics, providing structured and reliable datasets for analysis.\n\n## 🚀 Features\n\n- ⏱️ Daily automated extraction of sports events\n- 🔄 ETL process (Extract, Transform, Load)\n- 💾 Storage of structured data in a database or file system\n- 🧹 Data cleaning and transformation logic\n- 🔁 Scheduled execution with Apache Airflow\n\n## 🛠️ Technologies Used\n\n- **Python**\n- **Apache Airflow**\n- **Pandas / NumPy**\n- **Requests**\n- **MySQL** (adaptable)\n- **TheSportsDB API** *(or other public sports API)*\n\n## ⚙️ How to Run\n\n1. Clone the repository:\n  ```bash\n  git clone https://github.com/An4PDM/SportStats-Pipeline.git\n  cd SportStats-Pipeline\n   ```\n\n2. Create and activate a virtual environment:\n  ```bash\n  python -m venv venv\n  source venv/bin/activate  # or venv\\Scripts\\activate on Windows\n  ```\n\n3. Install dependencies:\n  ```bash\n  pip install -r requirements.txt\n  ```\n\n4. Set up and start Apache Airflow (basic setup):\n  \n  ```bash\n  export AIRFLOW_HOME=$(pwd)/airflow\n  airflow db init\n  airflow users create \\\n    --username admin \\\n    --password admin \\\n    --firstname First \\\n    --lastname Last \\\n    --role Admin \\\n    --email admin@example.com\n  airflow webserver --port 8080\n  airflow scheduler\n  ```\n5. Open http://localhost:8080 and trigger the sports DAG.\n\n## Tests\n\nYou can test individual components (e.g., scripts in scripts/) by running them directly:\n\n  ```bash\n  python scripts/extract.py\n  python scripts/transform.py\n  ```\n\n## 📄 License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fan4pdm%2Fsportstats-pipeline","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fan4pdm%2Fsportstats-pipeline","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fan4pdm%2Fsportstats-pipeline/lists"}