{"id":34600724,"url":"https://github.com/tekrop/overfast-api","last_synced_at":"2026-04-26T23:02:53.211Z","repository":{"id":61100306,"uuid":"429216369","full_name":"TeKrop/overfast-api","owner":"TeKrop","description":"⚡ Unofficial Overwatch 2 API, built with FastAPI, provides data on heroes, game modes, maps, and player careers","archived":false,"fork":false,"pushed_at":"2026-01-31T14:07:44.000Z","size":19489,"stargazers_count":152,"open_issues_count":1,"forks_count":19,"subscribers_count":7,"default_branch":"main","last_synced_at":"2026-01-31T21:38:55.345Z","etag":null,"topics":["api","blizzard","blizzard-profile-pages","fastapi","nginx","openresty","overwatch","overwatch-2","overwatch-api","overwatch-heroes","overwatch-maps","overwatch-stats","python","selectolax","valkey"],"latest_commit_sha":null,"homepage":"https://overfast-api.tekrop.fr","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TeKrop.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null},"funding":{"ko_fi":"tekrop"}},"created_at":"2021-11-17T22:11:27.000Z","updated_at":"2026-01-23T06:57:14.000Z","dependencies_parsed_at":"2025-11-27T13:03:32.398Z","dependency_job_id":null,"html_url":"https://github.com/TeKrop/overfast-api","commit_stats":null,"previous_names":[],"tags_count":144,"template":false,"template_full_name":null,"purl":"pkg:github/TeKrop/overfast-api","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TeKrop%2Foverfast-api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TeKrop%2Foverfast-api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TeKrop%2Foverfast-api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TeKrop%2Foverfast-api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TeKrop","download_url":"https://codeload.github.com/TeKrop/overfast-api/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TeKrop%2Foverfast-api/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28992629,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-01T20:57:35.821Z","status":"ssl_error","status_checked_at":"2026-02-01T20:57:29.580Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","blizzard","blizzard-profile-pages","fastapi","nginx","openresty","overwatch","overwatch-2","overwatch-api","overwatch-heroes","overwatch-maps","overwatch-stats","python","selectolax","valkey"],"created_at":"2025-12-24T12:41:13.499Z","updated_at":"2026-04-26T23:02:53.206Z","avatar_url":"https://github.com/TeKrop.png","language":"Python","funding_links":["https://ko-fi.com/tekrop"],"categories":[],"sub_categories":[],"readme":"# ⚡ OverFast API\n![Python](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/TeKrop/15a234815aa74059953a766a10e92688/raw/python-version.json)\n[![Build Status](https://github.com/TeKrop/overfast-api/actions/workflows/build.yml/badge.svg?branch=main)](https://github.com/TeKrop/overfast-api/actions/workflows/build.yml)\n[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=TeKrop_overfast-api\u0026metric=alert_status)](https://sonarcloud.io/summary/new_code?id=TeKrop_overfast-api)\n![Coverage](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/TeKrop/1362ebafcd51d3f65dae7935b1d322eb/raw/pytest.json)\n[![Issues](https://img.shields.io/github/issues/TeKrop/overfast-api)](https://github.com/TeKrop/overfast-api/issues)\n[![Documentation](https://img.shields.io/badge/documentation-yes-brightgreen.svg)](https://overfast-api.tekrop.fr)\n[![License: MIT](https://img.shields.io/github/license/TeKrop/overfast-api)](https://github.com/TeKrop/overfast-api/blob/master/LICENSE)\n![Mockup OverFast API](static/logo.png)\n\n\u003e OverFast API provides comprehensive data on Overwatch heroes, game modes, maps, and player statistics by scraping Blizzard pages. Built with **FastAPI** and **Selectolax**, **PostgreSQL** for persistent storage, **Stale-While-Revalidate caching** via **Valkey** and **nginx (OpenResty)**, **taskiq** background workers, and **TCP Slow Start + AIMD throttling** for Blizzard requests.\n\n## Table of contents\n* [✨ Live instance](#-live-instance)\n* [🐋 Run for production](#-run-for-production)\n* [💽 Run as developer](#-run-as-developer)\n* [👨‍💻 Technical details](#-technical-details)\n* [🐍 Architecture](#-architecture)\n* [📊 Monitoring](#-monitoring)\n* [🤝 Contributing](#-contributing)\n* [🚀 Community projects](#-community-projects)\n* [🙏 Credits](#-credits)\n* [📝 License](#-license)\n\n\n## ✨ [Live instance](https://overfast-api.tekrop.fr)\nThe live instance operates with a rate limit applied per second, shared across all endpoints. You can view the current rate limit on the home page, and this limit may be adjusted as needed. For higher request throughput, consider hosting your own instance on a dedicated server 👍\n\n- Live instance (Redoc documentation) : https://overfast-api.tekrop.fr/\n- Swagger UI : https://overfast-api.tekrop.fr/docs\n- Status page : https://uptime-overfast-api.tekrop.fr/\n\n## 🐋 Run for production\nRunning the project is straightforward. Ensure you have `docker` and `docker compose` installed. Next, generate a `.env` file using the provided `.env.dist` template. Finally, if `just` is already installed on your machine, execute the following command :\n\n```shell\njust up\n```\n\nYou can also use the `Makefile` alternative :\n\n```shell\nmake up\n```\n\n## 💽 Run as developer\nSame as earlier, ensure you have `docker` and `docker compose` installed, and generate a `.env` file using the provided `.env.dist` template. You can customize the `.env` file according to your requirements to configure the volumes used by the OverFast API.\n\nThen, execute the following commands to launch the dev server (you can still use the `make` alternative if `just` is not installed on your machine) :\n\n```shell\njust build          # Build the images, needed for all further commands\njust start          # Launch OverFast API (dev mode with autoreload)\njust start_testing  # Launch OverFast API (testing mode, with reverse proxy)\n```\nThe dev server will be running on the port `8000`. Reverse proxy will be running on the port `8080` in testing mode. You can use the `just down` command to stop and remove the containers. Feel free to type `just` or `just help` to access a comprehensive list of all available commands for your reference.\n\n### Generic settings\nShould you wish to customize according to your specific requirements, here is a detailed list of available settings:\n\n- `APP_VOLUME_PATH`: Folder for shared app data like logs, Valkey save file and dotenv file (app settings)\n- `APP_PORT`: Port for the app container (default is `80`).\n- `APP_BASE_URL` : Base URL for exposed links in endpoints like player search and maps listing.\n- `POSTGRES_HOST`, `POSTGRES_PORT`, `POSTGRES_DB`, `POSTGRES_USER`, `POSTGRES_PASSWORD`: PostgreSQL connection settings for persistent storage.\n\nYou likely won't need to modify other generic settings, but if you're curious about their functionality, consult the docstrings within the `app/config.py` file for further details.\n\n### Code Quality\nThe code quality is checked using `ruff` for linting and formatting, and `ty` for type checking. I'm also using `ruff format` for imports ordering and code formatting, enforcing PEP-8 convention on my code. To check the quality of the code, you just have to run the following commands :\n\n```shell\njust check     # Run ty type checker\njust lint      # Run ruff linter\njust format    # Run ruff formatter\n```\n\n### Testing\nThe code has been tested using unit testing, except some rare parts which are not relevant to test. There are tests on the parsers, the domain services, the adapters, and the API views (using FastAPI TestClient class). Tests are organized to mirror the DDD layer structure: `tests/domain/`, `tests/adapters/`, `tests/infrastructure/`, `tests/players/`, `tests/heroes/`, etc.\n\nRunning tests with coverage (default)\n```shell\njust test\n```\n\nRunning tests with given args (without coverage)\n```shell\njust test tests/domain/services\nmake test PYTEST_ARGS=\"tests/domain/services\"\n```\n\n\n### Pre-commit\nThe project is using [pre-commit](https://pre-commit.com/) framework to ensure code quality before making any commit on the repository. After installing the project dependencies, you can install the pre-commit by using the `pre-commit install` command.\n\nThe configuration can be found in the `.pre-commit-config.yaml` file. It runs the following checks on modified files before each commit:\n- `ruff` for linting and auto-fixing\n- `ruff format` for code formatting\n- `ty` for type checking\n\n## 👨‍💻 Technical details\n\n### Computed statistics values\n\nIn player career statistics, various conversions are applied for ease of use:\n- **Duration values** are converted to **seconds** (integer)\n- **Percent values** are represented as **integers**, omitting the percent symbol\n- Integer and float string representations are converted to their respective types\n\n### Valkey caching\n\nOverFast API integrates a **Valkey**-based cache system with two main components:\n- **API Cache**: This high-level cache associates URIs (cache keys) with a **SWR envelope** — a JSON object containing the response payload alongside metadata (`stored_at`, `staleness_threshold`, `stale_while_revalidate`). Nginx reads this envelope directly to serve `Age` and `Cache-Control: stale-while-revalidate` headers without calling FastAPI when data is stale but within the SWR window.\n- **Player Cache**: Stores persistent player profiles. This is backed by **PostgreSQL**, with Valkey used for short-lived negative caching (unknown players).\n\nBelow is the current list of TTL values configured for the API cache. The latest values are available on the API homepage.\n* Heroes list : 1 day\n* Hero specific data : 1 day\n* Roles list : 1 day\n* Gamemodes list : 1 day\n* Maps list : 1 day\n* Players career : 1 hour\n* Players search : 10 min\n\n## 🐍 Architecture\n\n### Request flow (Stale-While-Revalidate)\n\nEvery cached response is stored in Valkey as an **SWR envelope** containing the payload and three timestamps: `stored_at`, `staleness_threshold`, and `stale_while_revalidate`. Nginx/OpenResty inspects the envelope on every request:\n\n- **Fresh** (age \u003c `staleness_threshold`): Nginx returns cached data immediately, no App involved.\n- **Stale** (age ≥ `staleness_threshold` but \u003c `stale_while_revalidate`): Nginx returns the stale data immediately *and* fires an async enqueue to the background worker to refresh it.\n- **Expired / missing**: Nginx forwards to the App, which fetches from Blizzard, stores a new envelope, and returns the response.\n\n```mermaid\nsequenceDiagram\n    autonumber\n    actor User\n    participant Nginx\n    participant Valkey\n    participant App\n    participant Worker\n    participant Blizzard\n\n    User-\u003e\u003eNginx: Make an API request\n    Nginx-\u003e\u003eValkey: Check API cache (SWR envelope)\n\n    alt Cache hit (fresh or stale SWR window)\n        Valkey--\u003e\u003eNginx: Return cached data (with Age header)\n        Nginx--\u003e\u003eUser: 200 OK — Cache-Control: stale-while-revalidate set by Nginx\n    else Cache miss\n        Valkey--\u003e\u003eNginx: No result\n        Nginx-\u003e\u003e+App: Forward request\n        alt Fresh data in storage\n            App-\u003e\u003eValkey: Store fresh SWR envelope\n        else Stale data in storage\n            App-\u003e\u003eWorker: Enqueue background refresh\n            App-\u003e\u003eValkey: Store stale SWR envelope (short TTL)\n            Note over Worker,Blizzard: Async background refresh\n            Worker-\u003e\u003e+Blizzard: Fetch updated data\n            Blizzard--\u003e\u003e-Worker: Return data\n            Worker-\u003e\u003eValkey: Overwrite with fresh SWR envelope\n        else No data yet — first request\n            App-\u003e\u003e+Blizzard: Fetch data\n            Blizzard--\u003e\u003e-App: Return data\n            App-\u003e\u003eApp: Parse response\n            App-\u003e\u003eValkey: Store fresh SWR envelope\n        end\n        App--\u003e\u003e-Nginx: Return data\n        Nginx--\u003e\u003eUser: 200 OK\n    end\n```\n\n### Player profile flow\n\nPlayer profiles follow the same SWR logic, with the addition that parsed profile data is persisted in **PostgreSQL**. The worker compares the current `lastUpdated` value from the Blizzard search endpoint before deciding whether to re-parse the HTML.\n\n```mermaid\nsequenceDiagram\n    autonumber\n    actor User\n    participant Nginx\n    participant Valkey\n    participant App\n    participant Worker\n    participant PostgreSQL\n    participant Blizzard\n\n    User-\u003e\u003eNginx: Make player profile request\n    Nginx-\u003e\u003eValkey: Check API cache (SWR envelope)\n\n    alt Cache hit (fresh or stale SWR window)\n        Valkey--\u003e\u003eNginx: Return cached data (with Age header)\n        Nginx--\u003e\u003eUser: 200 OK — Cache-Control: stale-while-revalidate set by Nginx\n    else Cache miss\n        Valkey--\u003e\u003eNginx: No result\n        Nginx-\u003e\u003e+App: Forward request\n        alt Fresh profile in PostgreSQL\n            App-\u003e\u003eValkey: Store fresh SWR envelope\n        else Stale profile in PostgreSQL\n            App-\u003e\u003eWorker: Enqueue refresh_player_profile\n            App-\u003e\u003eValkey: Store stale SWR envelope (short TTL)\n            Note over Worker,Blizzard: Async background refresh\n            Worker-\u003e\u003e+Blizzard: Fetch search data (lastUpdated)\n            Blizzard--\u003e\u003e-Worker: Return search data\n            Worker-\u003e\u003e+PostgreSQL: Load stored profile\n            PostgreSQL--\u003e\u003e-Worker: Return stored profile\n            alt lastUpdated unchanged\n                Worker-\u003e\u003eValkey: Refresh SWR envelope (no re-parse)\n            else Profile changed\n                Worker-\u003e\u003e+Blizzard: Fetch player HTML\n                Blizzard--\u003e\u003e-Worker: Return HTML\n                Worker-\u003e\u003eWorker: Parse HTML\n                Worker-\u003e\u003ePostgreSQL: Upsert player profile\n                Worker-\u003e\u003eValkey: Store fresh SWR envelope\n            end\n        else No profile yet — first request\n            App-\u003e\u003e+Blizzard: Fetch search + player HTML\n            Blizzard--\u003e\u003e-App: Return data\n            App-\u003e\u003eApp: Parse HTML\n            App-\u003e\u003ePostgreSQL: Upsert player profile\n            App-\u003e\u003eValkey: Store fresh SWR envelope\n        end\n        App--\u003e\u003e-Nginx: Return data\n        Nginx--\u003e\u003eUser: 200 OK\n    end\n```\n\n### Background worker\n\nOverFast API runs a separate **taskiq** worker process alongside the FastAPI app:\n\n```shell\n# Worker\ntaskiq worker app.adapters.tasks.worker:broker\n# Scheduler\ntaskiq scheduler app.adapters.tasks.worker:scheduler\n```\n\n**On-demand tasks** (enqueued via SWR stale hits):\n- `refresh_heroes`, `refresh_hero`, `refresh_roles`, `refresh_maps`, `refresh_gamemodes`\n- `refresh_player_profile`\n\n**Scheduled cron tasks**:\n- `cleanup_stale_players` — daily at 03:00 UTC (removes expired profiles from PostgreSQL)\n- `check_new_hero` — daily at 02:00 UTC (detects newly released heroes)\n\nThe broker is a custom `ValkeyListBroker` backed by Valkey lists. Deduplication is handled by `ValkeyTaskQueue`, which uses `SET NX` so the same entity (e.g. a player battletag) is never enqueued twice for the same task type.\n\n```mermaid\nflowchart LR\n    Nginx --\u003e|stale hit| App\n    App --\u003e|LPUSH task| ValkeyQueue\n    ValkeyQueue --\u003e|BRPOP| Worker\n    Worker --\u003e|fetch| Blizzard\n    Worker --\u003e|upsert| PostgreSQL\n    Worker --\u003e|store envelope| Valkey\n```\n\n### Blizzard throttle (TCP Slow Start + AIMD)\n\nThe `BlizzardThrottle` component manages a self-adjusting inter-request delay that maximises throughput without triggering Blizzard 403s. **Only the HTTP status code is used as a signal** — response latency is intentionally ignored because player profiles are inherently slow and do not indicate rate limiting.\n\nThrottle state (`throttle:delay`, `throttle:ssthresh`, `throttle:streak`, `throttle:last_403`, `throttle:last_request`) is persisted in Valkey so it survives restarts and is shared between the API and worker processes.\n\n**Two phases:**\n\n| Phase | Condition | Behaviour |\n|---|---|---|\n| **Slow Start** | `delay \u003e ssthresh` | Halve delay every N consecutive 200s — fast exponential convergence |\n| **AIMD** | `delay ≤ ssthresh` | Subtract `delta` (50 ms) every M consecutive 200s — cautious linear probe |\n| **Penalty** | Any 403 | Double delay (min `penalty_delay`), set `ssthresh = delay × 2`, reset streak, block recovery for `penalty_duration` s |\n\n```mermaid\nstateDiagram-v2\n    [*] --\u003e SlowStart : startup / post-penalty\n    SlowStart --\u003e SlowStart : 200 (streak \u003c N) — increment streak\n    SlowStart --\u003e SlowStart : 200 (streak = N) — halve delay, reset streak\n    SlowStart --\u003e AIMD : delay ≤ ssthresh\n    AIMD --\u003e AIMD : 200 (streak \u003c M) — increment streak\n    AIMD --\u003e AIMD : 200 (streak = M) — delay −= delta, reset streak\n    AIMD --\u003e AIMD : delay = min_delay — stay at floor\n    SlowStart --\u003e Penalty : 403\n    AIMD --\u003e Penalty : 403\n    Penalty --\u003e SlowStart : penalty_duration elapsed\n    SlowStart --\u003e SlowStart : non-200 — reset streak\n    AIMD --\u003e AIMD : non-200 — reset streak\n```\n\n## 📊 Monitoring\n\nOverFast API ships an optional observability stack built on **Prometheus** and **Grafana**. When enabled, metrics are collected from two sources: **Nginx/OpenResty** (all requests, including Valkey cache hits) via a Lua module, and **FastAPI middleware** (requests that reach the app — cache misses only). Both sources normalize dynamic path segments (player IDs, hero keys) to prevent cardinality explosion, using matching Python and Lua implementations.\n\n![Grafana dashboard screenshot](static/monitoring/grafana_dashboard.png)\n\nKey metrics tracked include request rates and status code distribution, cache hit/miss ratios, Blizzard upstream call rates, AIMD throttle state, and background task throughput. Auto-provisioned Grafana dashboards cover API usage, API health, Blizzard calls, tasks \u0026 rate limiting, and system metrics.\n\nEnable the full stack with a single command:\n\n```shell\njust up monitoring=true\n```\n\nFor detailed metric definitions, normalization rules, dashboard descriptions, and troubleshooting, see [app/monitoring/README.md](app/monitoring/README.md).\n\n## 🤝 Contributing\n\nContributions, issues and feature requests are welcome ! Do you want to update the heroes data (health, armor, shields, etc.) or the maps list ? Don't hesitate to consult the dedicated [CONTRIBUTING file](https://github.com/TeKrop/overfast-api/blob/main/CONTRIBUTING.md).\n\n\n## 🚀 Community projects\nProjects using OverFast API as a data source are listed below. Using it in your project? Reach out via email with your project link, and I'll add it!\n\n- Counterwatch, an Overwatch overlay and stat tracker (https://www.counterwatch.gg)\n- Discord Bot OW2 for stats (https://github.com/polsojac/ow2discordbot)\n- Futuregamers Interface, including an Overwatch profile tracker (https://fi.futuregamers.eu/overwatch)\n- OverFast API client, a TypeScript package (https://github.com/Sipixer/overfast-api-client)\n- Overwatch Career Profile (https://github.com/EliaRenov/ow-career-profile)\n- overwatch_py, an async Python client for the OverFast API (https://github.com/Leo890728/overwatch_py)\n- OverwatchPy, a Python wrapper for the API (https://github.com/alexraskin/overwatchpy)\n- Watch Over, mobile app by @Backxtar (https://gitlab.backxtar.de/backxtar/watch-over-app)\n\n## 🙏 Credits\n\nAll maps screenshots hosted by the API are owned by Blizzard. Sources :\n- Blizzard Press Center (https://blizzard.gamespress.com)\n- Overwatch Wiki (https://overwatch.fandom.com/wiki/)\n\n\n## 📝 License\n\nCopyright © 2021-2026 [Valentin PORCHET](https://github.com/TeKrop).\n\nThis project is [MIT](https://github.com/TeKrop/overfast-api/blob/master/LICENSE) licensed.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftekrop%2Foverfast-api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftekrop%2Foverfast-api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftekrop%2Foverfast-api/lists"}