{"id":47339950,"url":"https://github.com/maugus0/sats-flight-data-fetcher","last_synced_at":"2026-03-17T22:43:45.608Z","repository":{"id":334188524,"uuid":"1140406110","full_name":"maugus0/sats-flight-data-fetcher","owner":"maugus0","description":"A simple Python tool to fetch and analyze flight data for 15+ major airlines using the AirLabs API.","archived":false,"fork":false,"pushed_at":"2026-02-04T09:04:52.000Z","size":66,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-04T21:36:57.717Z","etag":null,"topics":["airline-data","cli-tool","data-analysis","flight-data","python3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/maugus0.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-01-23T08:30:58.000Z","updated_at":"2026-02-04T09:04:54.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/maugus0/sats-flight-data-fetcher","commit_stats":null,"previous_names":["maugus0/sats-flight-data-fetcher"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/maugus0/sats-flight-data-fetcher","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maugus0%2Fsats-flight-data-fetcher","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maugus0%2Fsats-flight-data-fetcher/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maugus0%2Fsats-flight-data-fetcher/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maugus0%2Fsats-flight-data-fetcher/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/maugus0","download_url":"https://codeload.github.com/maugus0/sats-flight-data-fetcher/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maugus0%2Fsats-flight-data-fetcher/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30633799,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-17T22:38:22.569Z","status":"ssl_error","status_checked_at":"2026-03-17T22:38:11.804Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["airline-data","cli-tool","data-analysis","flight-data","python3"],"created_at":"2026-03-17T22:43:44.978Z","updated_at":"2026-03-17T22:43:45.601Z","avatar_url":"https://github.com/maugus0.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ✈️ Airlines Flight Data Fetcher\n\n[![CI/CD Pipeline](https://github.com/maugus0/sats-flight-data-fetcher/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/maugus0/sats-flight-data-fetcher/actions/workflows/ci.yml)\n![Python](https://img.shields.io/badge/Python-3.11+-blue.svg)\n![License](https://img.shields.io/badge/License-MIT-green.svg)\n![API](https://img.shields.io/badge/API-AirLabs-orange.svg)\n![Tests](https://img.shields.io/badge/Tests-Passing-brightgreen.svg)\n\nA simple, user-friendly Python tool to fetch flight data for any major airline. Perfect for data analysis, research, or operational tracking.\n\n## 🎯 Features\n\n- **15+ Airlines Supported** - Singapore Airlines, Emirates, Qatar Airways, British Airways, and more\n- **Multiple Output Formats** - Excel (with summary), CSV, or JSON\n- **Date Range Support** - Fetch single day or multiple days at once\n- **Interactive Mode** - No command line knowledge needed\n- **Progress Tracking** - Visual progress bar for multi-day fetches\n- **Automatic Checkpoints** - Raw JSON data saved for each day (recovery \u0026 backup)\n- **Retry Logic** - Automatic retries with exponential backoff (up to 3 attempts)\n- **Summary Statistics** - On-time performance, delays, top routes, status breakdown\n- **CI/CD Pipeline** - Automated testing (94.84% coverage), formatting, and security checks\n- **Pre-commit Hooks** - Catch issues before pushing to GitHub\n- **67+ Comprehensive Tests** - Full test suite covering all functionality (including hub filter with 5 dedicated tests)\n- **Hub Airport Filter** - Focus on flights to/from specific airports with input validation\n\n## ⚠️ AirLabs API Limits (Free Tier)\n\nBefore using this tool, understand the free tier limitations:\n\n| Limit Type | Free Tier | Paid Tier |\n|------------|-----------|-----------|\n| Results per page | 50 | 100-1000 |\n| **Total results per airline/day** | **~100 max** | 200+ |\n| API calls per month | 1,000 | 25,000+ |\n\n**What this means:**\n- Singapore Airlines operates 300+ flights daily\n- Free tier only returns approximately **100 flights** per day\n- Use `--hub SIN` filter to get the most relevant flights within this limit\n\n\u003e **💡 Tip:** The `--hub` filter is **highly recommended** to focus on flights to/from your airport of interest.\n\n## 📋 Quick Start\n\n### For Non-Technical Users\n\n1. **Download** this project and extract it to a folder\n2. **Install Python** from [python.org](https://python.org) (version 3.11 or higher)\n3. **Get an API Key** from [AirLabs](https://airlabs.co) (free tier available)\n4. **Open Terminal/Command Prompt** in the project folder\n5. **Run these commands:**\n\n```bash\npip install -r requirements.txt\npython fetch_flights.py\n```\n\n6. **Follow the prompts** - the script will guide you through selecting an airline and dates\n\n### For Technical Users\n\n```bash\n# Clone and setup\ngit clone https://github.com/maugus0/sats-flight-data-fetcher.git\ncd sats-flight-data-fetcher\npip install -r requirements.txt\n\n# Add your API key\necho \"AIRLABS_API_KEY=your_key_here\" \u003e .env\n\n# Fetch yesterday's Singapore Airlines flights (RECOMMENDED: use --hub for best results)\npython fetch_flights.py --airline SQ --yesterday --hub SIN\n\n# Fetch date range for Emirates flights to/from Dubai\npython fetch_flights.py --airline EK --start-date 2025-01-01 --end-date 2025-01-07 --hub DXB --format csv\n```\n\n## 🔑 API Key Setup\n\n1. Go to [AirLabs.co](https://airlabs.co) and create a free account\n2. Navigate to your Dashboard → API Key\n3. Create a `.env` file in the project folder:\n\n```\nAIRLABS_API_KEY=your_actual_api_key_here\n```\n\n\u003e **Free Tier:** 1,000 requests/month (1 request = 1 day of data)\n\n## 🎯 Hub Airport Filter (Recommended)\n\nFilter flights to only include those to/from a specific airport:\n\n```bash\n# Singapore Airlines flights to/from Singapore only\npython fetch_flights.py --airline SQ --yesterday --hub SIN\n\n# Emirates flights to/from Dubai only\npython fetch_flights.py --airline EK --yesterday --hub DXB\n\n# Qatar Airways flights to/from Doha only\npython fetch_flights.py --airline QR --last-week --hub DOH\n```\n\n**Why use `--hub`?**\n- API returns max ~100 flights/day (free tier limit)\n- **Without filter:** Random ~100 flights from global network\n- **With `--hub SIN`:** ~100 flights filtered to Singapore routes only ✓\n\n**Validation:**\n- Hub code must be exactly 3 letters (e.g., `SIN`, `DXB`, `LHR`)\n- Case-insensitive (automatically converted to uppercase)\n- Invalid codes will show a helpful error message\n\n## 📖 Usage Examples\n\n### Interactive Mode (Easiest)\n\n```bash\npython fetch_flights.py\n```\n\nThe script will ask you:\n- Which airline? (type `list` to see all options)\n- Start date?\n- End date?\n- Output format?\n\n\u003e **Note:** Hub filter is only available in command-line mode\n\n### Command Line Mode\n\n```bash\n# Fetch yesterday's flights (with hub filter - RECOMMENDED)\npython fetch_flights.py --airline SQ --yesterday --hub SIN\n\n# Fetch last week\npython fetch_flights.py --airline EK --last-week --hub DXB\n\n# Fetch last month\npython fetch_flights.py --airline QR --last-month --hub DOH\n\n# Fetch specific date range\npython fetch_flights.py --airline BA --start-date 2025-01-01 --end-date 2025-01-31 --hub LHR\n\n# Output as CSV instead of Excel\npython fetch_flights.py --airline LH --yesterday --hub FRA --format csv\n\n# Output as JSON\npython fetch_flights.py --airline AF --yesterday --hub CDG --format json\n\n# List all available airlines\npython fetch_flights.py --list-airlines\n```\n\n## 📚 Command Reference\n\n| Option | Description | Example |\n|--------|-------------|---------|\n| `--airline`, `-a` | Airline IATA code | `--airline SQ` |\n| `--start-date`, `-s` | Start date (YYYY-MM-DD) | `--start-date 2025-12-01` |\n| `--end-date`, `-e` | End date (YYYY-MM-DD) | `--end-date 2025-12-31` |\n| `--hub` | Filter to/from airport (3-letter IATA code, validated) | `--hub SIN` |\n| `--yesterday` | Fetch yesterday's flights | `--yesterday` |\n| `--last-week` | Fetch last 7 days | `--last-week` |\n| `--last-month` | Fetch last 30 days | `--last-month` |\n| `--format`, `-f` | Output format (csv/excel/json) | `--format csv` |\n| `--verbose`, `-v` | Show debug info | `--verbose` |\n| `--list-airlines` | Show all airlines | `--list-airlines` |\n\n## 📋 Common Use Cases\n\n### Singapore Airlines flights to/from Singapore\n\n```bash\npython fetch_flights.py --airline SQ --last-month --hub SIN --format excel\n```\n\n### Yesterday's flights with detailed output\n\n```bash\npython fetch_flights.py --airline SQ --yesterday --hub SIN --verbose\n```\n\n### Export to CSV for data analysis\n\n```bash\npython fetch_flights.py --airline SQ --start-date 2025-12-01 --end-date 2025-12-31 --hub SIN --format csv\n```\n\n### Interactive mode (easiest for beginners)\n\n```bash\npython fetch_flights.py\n```\n\n*Note: Hub filter is only available in command-line mode*\n\n## ✈️ Supported Airlines\n\n| Code | Airline | Country |\n|------|---------|---------|\n| SQ | Singapore Airlines | Singapore |\n| EK | Emirates | UAE |\n| QR | Qatar Airways | Qatar |\n| BA | British Airways | UK |\n| LH | Lufthansa | Germany |\n| AF | Air France | France |\n| AA | American Airlines | USA |\n| UA | United Airlines | USA |\n| DL | Delta Air Lines | USA |\n| CX | Cathay Pacific | Hong Kong |\n| NH | All Nippon Airways | Japan |\n| JL | Japan Airlines | Japan |\n| QF | Qantas | Australia |\n| TK | Turkish Airlines | Turkey |\n| KE | Korean Air | South Korea |\n\nRun `python fetch_flights.py --list-airlines` for the full list.\n\n## 📊 Output Files\n\n### Excel Output (Default) ✨ **Two Sheets Included!**\n\nCreates a `.xlsx` file with **two separate sheets**:\n\n#### **Sheet 1: \"Flight Data\"** 📊\nContains all individual flight records with complete details:\n\n| Flight | From | To | Sched. Dep | Actual Dep | Sched. Arr | Actual Arr | Delay (min) | Status |\n|--------|------|----|-----------:|-----------:|------------|------------|------------:|--------|\n| SQ123 | SIN | LHR | 08:00 | 08:15 | 15:30 | 15:45 | 15 | landed |\n| SQ317 | SIN | LHR | 23:30 | 23:42 | 06:15 | 06:25 | 12 | landed |\n\n**Features:**\n- All flight records in one table\n- Styled headers (blue background, white text)\n- Auto-adjusted column widths\n- Easy to filter, sort, and analyze\n\n#### **Sheet 2: \"Summary\"** 📈\nContains aggregated statistics and insights:\n\n**Key Metrics:**\n- Total Flights\n- Average Delay (minutes)\n- On-Time Performance (%)\n- Delayed Flights (≥15 min)\n- Cancelled Flights\n\n**Breakdowns:**\n- **Flights by Status**: Count of flights by status (landed, scheduled, cancelled, etc.)\n- **Top 10 Routes**: Most frequent routes with flight counts\n\n**Features:**\n- Formatted title with airline name\n- Clear sections with bold labels\n- Easy-to-read layout for reporting\n\n### CSV Output\n\n**Note:** CSV is a single flat file format - it does NOT support multiple sheets. If you need two sheets (flight data + summary), use Excel format instead.\n\n**CSV contains:**\n- All flight records in a simple comma-separated format\n- Easy to import into Excel, databases, or other tools\n- One row per flight with all details\n\n**To get summary statistics with CSV:**\n- Use JSON format which includes both flights and summary\n- Or use Excel format (default) which has both sheets\n\n### JSON Output\n\nComplete data structure with:\n- `flights`: Array of all flight records\n- `summary`: Statistics object with all metrics\n\n### Checkpoint Files\n\nRaw JSON responses are automatically saved as `checkpoint_AIRLINE_DATE.json` in the `outputs/` folder. These can be used for:\n- Data recovery if export fails\n- Re-processing data without API calls\n- Debugging API responses\n\n## 📁 Project Structure\n\n```\nsats-flight-data-fetcher/\n├── .env                    # Your API key (create this)\n├── .env.example            # API key template\n├── .gitignore              # Files to ignore in git\n├── README.md               # This file\n├── CONTRIBUTING.md         # Contributor guidelines\n├── requirements.txt        # Python dependencies\n├── requirements-dev.txt    # Dev/test dependencies\n├── airlines_config.json    # Airline definitions\n├── fetch_flights.py        # Main script\n├── pytest.ini              # Test configuration\n├── .flake8                  # Flake8 linting config\n├── .coveragerc              # Coverage reporting config\n├── .pre-commit-config.yaml  # Pre-commit hooks config\n├── pre-commit-check.ps1    # Pre-commit checks (Windows)\n├── pre-commit-check.sh     # Pre-commit checks (Linux/macOS)\n├── fix-formatting.ps1      # Auto-fix formatting (Windows)\n├── fix-formatting.sh       # Auto-fix formatting (Linux/macOS)\n├── tests/                  # Test suite (67 tests)\n│   ├── __init__.py\n│   ├── conftest.py         # Pytest fixtures\n│   ├── test_fetch_flights.py  # Main test file\n│   └── test_config.py      # Config validation tests\n├── .github/\n│   ├── workflows/\n│   │   └── ci.yml          # CI/CD pipeline\n│   ├── ISSUE_TEMPLATE/     # Issue templates\n│   └── PULL_REQUEST_TEMPLATE.md\n└── outputs/                # Generated files (auto-created)\n```\n\n## ❓ FAQ\n\n**Q: Why am I only getting ~100 flights per day?**\nA: This is an AirLabs free tier limitation. Use `--hub` filter to focus on flights to/from your airport of interest (e.g., `--hub SIN` for Singapore).\n\n**Q: What does `--hub` do?**\nA: It filters results to only include flights departing from OR arriving at the specified airport. For example, `--hub SIN` shows only flights connected to Singapore. The hub code must be a valid 3-letter IATA airport code (e.g., SIN, DXB, LHR) and is automatically validated before use.\n\n**Q: How many API calls do I need?**\nA: 1 API call per day of data. A week needs 7 calls, a month needs ~30 calls.\n\n**Q: Is the free tier enough?**\nA: Yes for basic usage! 1,000 requests/month is sufficient for tracking multiple airlines. Note that results are capped at ~100 flights/day per airline.\n\n**Q: Can I add more airlines?**\nA: Yes! Edit `airlines_config.json` and add any airline with their IATA code.\n\n**Q: What if I get rate limited or timeout?**\nA: The script automatically retries up to 3 times with exponential backoff. Rate limits are handled gracefully with automatic waiting.\n\n**Q: Where are my files saved?**\nA: In the `outputs/` folder with timestamps (e.g., `SQ_2025-01-20_143052.xlsx`). Checkpoint files (raw JSON) are also saved as `checkpoint_AIRLINE_DATE.json` for data recovery.\n\n**Q: My CI/CD pipeline failed. What should I do?**\nA: Run `.\\pre-commit-check.ps1` (or `./pre-commit-check.sh`) locally to see the same errors. Fix them with `.\\fix-formatting.ps1` if needed, then commit and push again.\n\n## 🔧 Troubleshooting\n\n### \"Only getting ~100 flights per day\"\n\nThis is an **AirLabs free tier limitation**, not a bug.\n\n- Free tier caps results at approximately 100 flights per airline per day\n- Singapore Airlines operates 300+ daily flights, but you'll only get ~100\n- **Solution:** Use `--hub` filter to get the most relevant flights\n\n```bash\n# Without filter: Random ~100 flights from global network\npython fetch_flights.py --airline SQ --yesterday\n\n# With filter: ~100 flights focused on Singapore routes ✓\npython fetch_flights.py --airline SQ --yesterday --hub SIN\n```\n\nConsider upgrading to [AirLabs paid tier](https://airlabs.co/pricing) for more data.\n\n### \"API key not found\"\n\n1. Create a `.env` file in the project folder\n2. Add: `AIRLABS_API_KEY=your_key_here`\n3. Get your free key at [airlabs.co](https://airlabs.co)\n\n### \"No flights found\"\n\n- Check if the airline operated on that date\n- Verify the date format is `YYYY-MM-DD`\n- Try without the `--hub` filter first to confirm data exists\n- Some dates may have fewer or no flights (holidays, etc.)\n\n### \"Rate limited\"\n\n- Free tier: 1,000 requests/month\n- Script automatically waits and retries\n- Check your usage at [airlabs.co/account](https://airlabs.co/account)\n\n### Other Common Issues\n\n| Issue | Solution |\n|-------|----------|\n| \"Unknown airline\" | Run `--list-airlines` to see valid codes |\n| \"Invalid date\" | Use format `YYYY-MM-DD` (e.g., 2025-01-20) |\n| \"Hub must be a 3-letter airport code\" | Use valid IATA codes like SIN, DXB, LHR (exactly 3 letters) |\n| Timeout errors | Check internet connection, retry in a few minutes |\n| CI/CD badge shows \"no status\" | Push code, wait for workflow to complete. Check [Actions tab](https://github.com/maugus0/sats-flight-data-fetcher/actions) |\n\n## 🚀 CI/CD Pipeline\n\nThis project includes a complete CI/CD pipeline that runs automatically on every push and pull request:\n\n- **Code Formatting** - Ensures consistent code style with Black\n- **Linting** - Catches code quality issues with flake8 and pylint\n- **Unit Tests** - Runs comprehensive test suite (67 tests) with 94.84% code coverage\n- **Security Scanning** - Checks for vulnerabilities with Bandit and Safety\n- **Config Validation** - Validates JSON configs and requirements files\n- **Integration Tests** - Verifies script imports and CLI commands work\n\nView the pipeline status in the badge at the top of this README, or check the [Actions tab](https://github.com/maugus0/sats-flight-data-fetcher/actions) on GitHub.\n\n## 🧪 Development\n\n### Run Tests\n\n```bash\n# Install dev dependencies\npip install -r requirements-dev.txt\n\n# Run all tests (67 tests)\npytest tests/ -v\n\n# Run with coverage report\npytest tests/ --cov=fetch_flights --cov-report=html --cov-report=term\n\n# Check coverage threshold (must be \u003e= 70%)\ncoverage report --fail-under=70\n```\n\n**Current Test Coverage: 94.84%** ✅\n\n### Code Quality\n\n```bash\n# Format code (Black)\npython -m black fetch_flights.py tests/\n\n# Sort imports (isort with Black profile)\npython -m isort --profile=black --line-length=120 fetch_flights.py tests/\n\n# Lint (flake8)\npython -m flake8 fetch_flights.py tests/ --max-line-length=120 --extend-ignore=E203,W503\n\n# Code quality (pylint)\npylint fetch_flights.py --max-line-length=120 --disable=C0111,R0903,W0212,C0103\n```\n\n### Pre-commit Checks\n\n**Always run pre-commit checks before pushing to GitHub!** This prevents CI/CD pipeline failures.\n\n#### Run Pre-commit Checks\n\n**Windows (PowerShell):**\n```powershell\n.\\pre-commit-check.ps1\n```\n\n**Linux/macOS:**\n```bash\nchmod +x pre-commit-check.sh\n./pre-commit-check.sh\n```\n\nThe script checks:\n- ✅ Code formatting (Black)\n- ✅ Import sorting (isort)\n- ✅ Linting (flake8)\n- ✅ Python syntax validation\n- ✅ JSON config validation\n- ✅ Security (no hardcoded API keys)\n\n#### Auto-fix Formatting Issues\n\nIf formatting issues are found, auto-fix them:\n\n**Windows:**\n```powershell\n.\\fix-formatting.ps1\n```\n\n**Linux/macOS:**\n```bash\nchmod +x fix-formatting.sh\n./fix-formatting.sh\n```\n\nThen run the pre-commit check again to verify everything passes.\n\n#### Workflow\n\n```bash\n# 1. Make your changes\n# 2. Run pre-commit check\n.\\pre-commit-check.ps1  # or ./pre-commit-check.sh\n\n# 3. If issues found, auto-fix\n.\\fix-formatting.ps1  # or ./fix-formatting.sh\n\n# 4. Run check again to verify\n.\\pre-commit-check.ps1\n\n# 5. Commit and push\ngit add .\ngit commit -m \"your message\"\ngit push\n```\n\n\u003e **Tip:** Add these scripts to your git workflow to catch issues early and keep your CI/CD pipeline green! 🟢\n\n## 📝 License\n\nMIT License - feel free to use, modify, and distribute.\n\n## 🤝 Contributing\n\nContributions welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines.\n\n### Quick Contribution Guide\n\n1. **Fork the repo** and clone your fork\n2. **Create a feature branch**: `git checkout -b feature/your-feature`\n3. **Install dev dependencies**: `pip install -r requirements-dev.txt`\n4. **Make your changes**\n5. **Run pre-commit checks**: `.\\pre-commit-check.ps1` (or `./pre-commit-check.sh`)\n6. **Run tests**: `pytest tests/ -v`\n7. **Commit and push**: `git push origin feature/your-feature`\n8. **Create a Pull Request**\n\n\u003e **Important:** All PRs must pass pre-commit checks and tests. The CI/CD pipeline will verify this automatically.\n\n---\n\n**Happy Flight Tracking! ✈️**","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaugus0%2Fsats-flight-data-fetcher","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaugus0%2Fsats-flight-data-fetcher","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaugus0%2Fsats-flight-data-fetcher/lists"}