{"id":15722154,"url":"https://github.com/falkben/steam-to-sqlite","last_synced_at":"2025-08-22T21:32:09.655Z","repository":{"id":38374655,"uuid":"428096134","full_name":"falkben/steam-to-sqlite","owner":"falkben","description":"Public Steam app and achievement data in a sqlite database","archived":false,"fork":false,"pushed_at":"2025-06-07T22:07:19.000Z","size":280,"stargazers_count":3,"open_issues_count":6,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-08-15T03:54:34.643Z","etag":null,"topics":["datasette","sqlite","steam"],"latest_commit_sha":null,"homepage":"https://steam-to-sqlite.fly.dev","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/falkben.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-11-15T02:20:47.000Z","updated_at":"2025-06-07T22:07:21.000Z","dependencies_parsed_at":"2023-10-01T21:31:08.365Z","dependency_job_id":"6ba80e1f-fdb3-48a4-b77c-a0e29d7f2620","html_url":"https://github.com/falkben/steam-to-sqlite","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/falkben/steam-to-sqlite","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/falkben%2Fsteam-to-sqlite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/falkben%2Fsteam-to-sqlite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/falkben%2Fsteam-to-sqlite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/falkben%2Fsteam-to-sqlite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/falkben","download_url":"https://codeload.github.com/falkben/steam-to-sqlite/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/falkben%2Fsteam-to-sqlite/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271703775,"owners_count":24806527,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-22T02:00:08.480Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["datasette","sqlite","steam"],"created_at":"2024-10-03T22:04:33.997Z","updated_at":"2025-08-22T21:32:09.620Z","avatar_url":"https://github.com/falkben.png","language":"Python","readme":"# steam-to-sqlite\n\n![main branch](https://github.com/falkben/steam-to-sqlite/actions/workflows/test.yml/badge.svg?branch=main) [![codecov](https://codecov.io/gh/falkben/steam-to-sqlite/branch/main/graph/badge.svg?token=ZPVU94M3XE)](https://codecov.io/gh/falkben/steam-to-sqlite)\n\nPublic [Steam](https://store.steampowered.com/) app and achievement data in a sqlite database\n\n## Quick links\n\nThe database and deployments are updated automatically through GitHub actions.\n\n| | |\n|-|-|\n|SQLite3 File|[database.db ⬇️](https://www.dropbox.com/s/i47qt3chrp9lr9e/database.db?dl=1) |\n|[Datasette](https://datasette.io/) deployment|\u003chttps://steam-to-sqlite.fly.dev/\u003e|\n|GraphQL interface|\u003chttps://steam-to-sqlite.fly.dev/graphql\u003e|\n\nTo preview what's included in the database schema, you can view [models.py](/steam2sqlite/models.py), which contains the [SQLModel](https://sqlmodel.tiangolo.com/) representation, or just explore on Datasette.\n\n## Data exploration\n\nThe following are some plots generated from the data. Follow the Datasette links to view what's most recent.\n\n- Steam games over time (and you thought _your_ Steam library was large!)\n\n   ![Steam games over time](https://user-images.githubusercontent.com/653031/199382416-cf8c43f0-2cc5-47a5-99d7-ad7b32b41af9.png)\n\n   [Datasette Link](https://steam-to-sqlite.fly.dev/database?sql=select%0D%0A++strftime%28%27%25Y%27%2C+steam_app.release_date%29+as+year%2C%0D%0A++sum%28count%28steam_app.pk%29%29+over+%28%0D%0A++++order+by%0D%0A++++++steam_app.release_date%0D%0A++%29+as+total%0D%0Afrom%0D%0A++steam_app%0D%0Awhere%0D%0A++steam_app.release_date+is+not+NULL%0D%0A++and+steam_app.release_date+%3E%3D+date%28%222003-01-01%22%29%0D%0A++and+steam_app.release_date+%3C+CURRENT_DATE%0D%0A++and+steam_app.type+%3D+%22game%22%0D%0Agroup+by%0D%0A++year%0D%0Aorder+by%0D%0A++steam_app.release_date+asc#g.mark=line\u0026g.x_column=year\u0026g.x_type=ordinal\u0026g.y_column=total\u0026g.y_type=quantitative)\n\n- Games per genre (who knew there were so many indie games?!)\n\n   ![Games per genre](https://user-images.githubusercontent.com/653031/199382566-bf2cc609-f2c3-4841-a871-3cb2605c32de.png)\n\n   [Datasette Link](https://steam-to-sqlite.fly.dev/database?sql=select+genre.description%2C+count(steam_app.pk)+as+apps%0D%0Afrom+genre%0D%0Ajoin+genresteammapplink+on+genre_pk+%3D+genre.pk%0D%0Ajoin+steam_app+on+genresteammapplink.steam_app_pk+%3D+steam_app.pk%0D%0AGROUP+by+genre.pk%0D%0Aorder+by+apps+desc%3B#g.mark=bar\u0026g.x_column=description\u0026g.x_type=ordinal\u0026g.y_column=apps\u0026g.y_type=quantitative)\n\n- Games with full controller support (stand-in for Steam Deck support)\n\n   ![Games with full controller support over time](https://user-images.githubusercontent.com/653031/199390356-ad488ecd-e64b-4ca1-a5ba-0bc6ff3dd4cd.png)\n\n   [Datasette Link](https://steam-to-sqlite.fly.dev/database?sql=select%0D%0A++strftime%28%27%25Y%27%2C+steam_app.release_date%29+as+year%2C%0D%0A++sum%28count%28steam_app.pk%29%29+over+%28%0D%0A++++order+by%0D%0A++++++steam_app.release_date%0D%0A++%29+as+total%0D%0Afrom%0D%0A++steam_app%0D%0Awhere%0D%0A++steam_app.release_date+is+not+NULL%0D%0A++and+steam_app.release_date+%3E%3D+date%28%222003-01-01%22%29%0D%0A++and+steam_app.release_date+%3C+CURRENT_DATE%0D%0A++and+steam_app.type+%3D+%22game%22%0D%0A++and+steam_app.controller_support+%3D%3D+%22full%22%0D%0Agroup+by%0D%0A++year%0D%0Aorder+by%0D%0A++steam_app.release_date+asc#g.mark=line\u0026g.x_column=year\u0026g.x_type=ordinal\u0026g.y_column=total\u0026g.y_type=quantitative)\n\nHopefully these inspire you to explore the data. If you find something you want to highlight, PRs are welcome!\n\n## Install\n\nTo install the project locally:\n\n1. Ensure you have Python \u003e= 3.10. [pyenv](https://github.com/pyenv/pyenv) recommended\n2. Create and activate a virtual environment: `python -m venv .venv \u0026\u0026 . .venv/bin/activate`\n3. Install:\n   - `pip install -r requirements.txt`\n   - With dev. dependencies: `pip install -r requirements.txt -r dev-requirements.txt`\n4. Install package: `pip install -e .` or w/ dev dependencies `pip install -e \".[dev]\"`\n\n## Manage dependencies\n\n1. install/upgrade uv: \u003chttps://docs.astral.sh/uv/getting-started/installation/\u003e\n2. Create lock files with:\n\n   ```sh\n   uv pip compile -o requirements.txt pyproject.toml --quiet \u0026\u0026 \\\n   uv pip compile --group dev -o dev-requirements.txt -c requirements.txt pyproject.toml --quiet\n   ```\n\n3. Upgrade a package:\n\n   ```sh\n   uv pip compile -o requirements.txt pyproject.toml --quiet --upgrade-package PACKAGE \u0026\u0026 \\\n   uv pip compile --group dev -o dev-requirements.txt -c requirements.txt pyproject.toml --quiet\n   ```\n\n4. Upgrade all packages with:\n\n   ```sh\n   uv pip compile -o requirements.txt pyproject.toml --quiet --upgrade \u0026\u0026 \\\n   uv pip compile --group dev -o dev-requirements.txt -c requirements.txt pyproject.toml --quiet --upgrade\n   ```\n\nMore here: \u003chttps://github.com/astral-sh/uv\u003e\n\n## Run\n\nDownload a copy of the database and save locally to `database.db`\n\nTo verify installation:\n\n```bash\npython steam2sqlite/main.py --help\nusage: main.py [-h] [-l [LIMIT]]\n\noptions:\n  -h, --help            show this help message and exit\n  -l [LIMIT], --limit [LIMIT]\n                        limit runtime (minutes)\n```\n\nTo run:\n\n```sh\npython steam2sqlite/main.py\n```\n\nDue to rate limits on the public Steam api, the program will take several days to iterate over all the Steam apps in the Steam catalog.\n\nLimit the runtime in minutes with the `-l` or `--limit` argument:\n\n```sh\npython steam2sqlite/main.py --limit 1\n```\n\nWill run for 1 minutes and then (hopefully) exit cleanly with a database partially updated.\n\n## Migrations\n\nTo upgrade db to current migration/revision\n\n```sh\nalembic upgrade head\n```\n\nGenerate a migration/revision\n\n```sh\nalembic revision --autogenerate -m \"MESSAGE\"\n```\n\nExamine generated file in `migrations/versions`. Pay attention to table/column renames (they will be dropped/created in migration file).\n\n## Deploy to Fly\n\nThe deployment runs automatically with [GitHub actions](/.github/workflows/main.yml). To manually deploy:\n\n1. install [flyctl](https://fly.io/docs/getting-started/installing-flyctl/)\n2. `flyctl auth login`\n3. Deploy:\n\n    ```sh\n    datasette publish fly database.db \\\n      --install datasette-graphql --install datasette-vega \\\n      --app steam-to-sqlite \\\n      --metadata datasette-data/metadata.json\n    ```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffalkben%2Fsteam-to-sqlite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffalkben%2Fsteam-to-sqlite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffalkben%2Fsteam-to-sqlite/lists"}