{"id":17820500,"url":"https://github.com/eve-ning/osu-data","last_synced_at":"2026-03-01T23:32:23.229Z","repository":{"id":177479593,"uuid":"657700874","full_name":"Eve-ning/osu-data","owner":"Eve-ning","description":"Docker Containerize osu!db as a MySQL Service","archived":false,"fork":false,"pushed_at":"2024-08-16T13:44:53.000Z","size":2362,"stargazers_count":1,"open_issues_count":2,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-09-28T23:16:52.057Z","etag":null,"topics":["docker-compose","mysql"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Eve-ning.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-23T16:31:08.000Z","updated_at":"2025-08-08T23:49:24.000Z","dependencies_parsed_at":null,"dependency_job_id":"b7788fb0-d685-4d8d-a987-3e015a2d0023","html_url":"https://github.com/Eve-ning/osu-data","commit_stats":null,"previous_names":["eve-ning/osu-data-docker","eve-ning/osu-data"],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/Eve-ning/osu-data","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Eve-ning%2Fosu-data","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Eve-ning%2Fosu-data/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Eve-ning%2Fosu-data/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Eve-ning%2Fosu-data/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Eve-ning","download_url":"https://codeload.github.com/Eve-ning/osu-data/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Eve-ning%2Fosu-data/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29987698,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-01T22:42:38.399Z","status":"ssl_error","status_checked_at":"2026-03-01T22:41:51.863Z","response_time":124,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker-compose","mysql"],"created_at":"2024-10-27T17:05:40.729Z","updated_at":"2026-03-01T23:32:23.215Z","avatar_url":"https://github.com/Eve-ning.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# osu! Data on Docker\n\n`pip install osu-data; osu-data -m mania -v top_1000 -ymd YYYY_MM_DD`\n\n**Docker must be installed and running on your machine.**\n\nRetrieves database data from https://data.ppy.sh/ and hosts it on a local MySQL\nserver.\nOptionally, store all ranked/loved `.osu` files in a service with the `-f` tag.\n\n## Get Started\n\n**IMPORTANT**: MySQL data persists across runs.\nRecreate the MySQL Service if you changed the data used.\n\n1) Install via pip `pip install osu-data`\n\n2) Minimally, specify:\n    - `-m`, `--mode`:\n      The game mode to build the database with. `osu`, `taiko`, `catch`\n      or `mania`\n    - `-v`, `--version`:\n      The database version. `top_1000`, `top_10000` or `random_10000`\n3) Optionally, specify:\n    - `-ymd`, `--year_month_day`:\n      The year, month, day of the database in the format `YYYY_MM_DD`\n    - `-p`, `--port`:\n      The port to expose MySQL on. Default is `3308`\n    - `-f`, `--files`:\n      Whether to download `.osu` files.\n    - `-np`, `--nginx-port`:\n      The port to expose the nginx service on. Default is `8080`.\n      Not used if `-f` is not specified.\n    - `--...`:\n      See below table, these are optional flags to include or exclude more\n      data. **By specifying the flag, will INVERT the default value.**\n\n| Option                            | Default Value |\n|-----------------------------------|---------------|\n| `--beatmap-difficulty-attribs`    | False         |\n| `--beatmap-difficulty`            | False         |\n| `--scores`                        | True          |\n| `--beatmap-failtimes`             | False         |\n| `--user-beatmap-playcount`        | False         |\n| `--beatmaps`                      | True          |\n| `--beatmapsets`                   | True          |\n| `--user-stats`                    | True          |\n| `--sample-users`                  | True          |\n| `--counts`                        | True          |\n| `--difficulty-attribs`            | True          |\n| `--beatmap-performance-blacklist` | True          |\n\nThese options are chosen to be the most useful for analysis, and performance.\n\nE.g.\n\n```bash\nosu-data \\\n  -m osu -v top_1000 -ymd 2023_08_01 -p 3308 -f \\\n  --beatmap-difficulty \n```\n\n- Download the top 1000 osu! standard beatmaps\n- from 1st August 2023 \n- expose MySQL on port 3308\n- download `.osu` files\n- include beatmap difficulty data\n\n4) Connect on:\n   - `localhost:\u003cMYSQL_PORT\u003e`\n   - `localhost:\u003cNGINX_PORT\u003e` (if `-f` is specified)\n\n## Common Issues\n\n- **Docker daemon is not running**. Make sure that Docker is installed and\n  running. If you're using Docker Desktop, make sure it's actually started.\n- **MySQL Data isn't incorrect**. A few reasons\n    - *Import was abruptly stopped*. This can cause some `.sql` files to be\n      missing / incomplete. Delete the whole compose project and try again.\n    - *Didn't specify the optional flags to include files*. By default, some\n      `.sql` files are not loaded. Take a look at `osu-data -h` and specify the\n      optional flags to include them.\n    - *Data is outdated*. By default, on every re-run of `osu-data`, the data\n      is\n      preserved. To update the data, you must delete the whole compose project\n      and try again.\n- **wget: server returned error: HTTP/1.1 404 Not Found**. This happens when\n  you try to pull a `YYYY_MM_DD` that doesn't exist, and happens often when the\n  data isn't yet ready on the start of each month.\n  Check on https://data.ppy.sh/ to see which `YYYY_MM_DD` are available.\n- **`rm: can't remove '../osu.mysql.init/*'`**: This is safe to ignore.\n- **MySQL Credentials**. By default, the MySQL doesn't have a password, so just\n  use `root` as the username and leave the password blank.\n- **No `files` service**. This is default, `files` service is optional and\n  must be activated with the `-f` tag. `osu-data -h` for more info.\n\n## `mysql.cnf`\n\nThe database is tuned to be fast in importing speed, thus shouldn't be used for\nproduction. Notably, we set `innodb_doublewrite = 0` which can compromise\ndata integrity in the event of a crash. If you want to use this for production,\nwe recommend to set this up from this Git repo, and tweak `mysql.cnf`.\n\n## Important Matters\n\n1) Do not distribute the built images as per peppy's request.\n   Instead, you can just share the code to build your image, which should yield\n   the same result.\n2) This database is meant to be for analysis, it's not tuned for production.\n   Tweak `mysql.cnf` after importing\n   for more MySQL customizations.\n3) Finally, be mindful on conclusions you make from the data.\n\n## Changelog\n\n- **0.1.5**:\n  - Allowed wider range of Python versions `3.9 ~ 4.0`.\n- **0.2.0**: \n  - Added GitHub Actions to automatically create dataset on workflow dispatch. \n  - Year, Month specification is now Year, Month, Day because some data dumps \n    don't fall exactly on day 1.\n    - `-ym` -\u003e `-ymd`, `--year-month` -\u003e `--year-month-day`\n    - Default of `-ymd` is removed to encourage users to check source of data.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feve-ning%2Fosu-data","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feve-ning%2Fosu-data","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feve-ning%2Fosu-data/lists"}