{"id":19777165,"url":"https://github.com/ppy/osu-queue-score-statistics","last_synced_at":"2025-06-23T21:38:50.942Z","repository":{"id":38138432,"uuid":"376730143","full_name":"ppy/osu-queue-score-statistics","owner":"ppy","description":"A queue processor to handle updating general statistics updates from user scores","archived":false,"fork":false,"pushed_at":"2024-10-29T08:59:21.000Z","size":1267,"stargazers_count":12,"open_issues_count":13,"forks_count":11,"subscribers_count":10,"default_branch":"master","last_synced_at":"2024-10-29T10:07:03.662Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ppy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-06-14T06:50:46.000Z","updated_at":"2024-10-29T08:59:26.000Z","dependencies_parsed_at":"2023-02-14T05:01:08.724Z","dependency_job_id":"5596ce0b-86c3-4ad9-b29c-d128252b7c46","html_url":"https://github.com/ppy/osu-queue-score-statistics","commit_stats":null,"previous_names":[],"tags_count":82,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ppy%2Fosu-queue-score-statistics","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ppy%2Fosu-queue-score-statistics/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ppy%2Fosu-queue-score-statistics/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ppy%2Fosu-queue-score-statistics/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ppy","download_url":"https://codeload.github.com/ppy/osu-queue-score-statistics/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251769351,"owners_count":21640887,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T05:23:27.472Z","updated_at":"2025-04-30T19:31:31.747Z","avatar_url":"https://github.com/ppy.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# osu-queue-score-statistics-processor [![dev chat](https://discordapp.com/api/guilds/188630481301012481/widget.png?style=shield)](https://discord.gg/ppy)\n\nThis project is at heart an [osu-queue-processor](https://github.com/ppy/osu-queue-processor) that handles various book-keeping tasks in response to scores arriving from users. This includes:\n\n- Awarding medals\n- Updating the pp values of individual scores\n- Updating user statistics (total scores / pp / rank counts / play time / play count / max combo)\n\nIt also offers commands to perform common maintenance on all the above, including tasks like:\n\n- Re-running pp calculations on scores (and users) after a change in algorithm\n- Re-running processing of individual scores after a change in processing (ie. a new medal or new tracked statistic).\n- Migrating scores from legacy tables to the new format\n\nSome things are not yet fully fleshed out:\n\n- Not all medals are handled\n- Ranked score processor is not added\n\n# Getting started\n\nTo start the queue processing functionality:\n\n```sh\ndotnet run -- queue watch\n```\n\nTo reprocess pp of all scores:\n\n```sh\ndotnet run -- performance all\n```\n\nTo pump all scores in `scores` back through the queue for reprocessing:\n\n```sh\ndotnet run -- queue pump-all\n```\n\nNote that you will want a queue processor running to handle all the scores that are going to appear in the queue.\n\n# Processs versioning\n\nProcessing is versioned (see `ScoreProcessed.ProcessedVersion`), so re-queueing scores which have already been processed previously is a safe operation – they will either be skipped or upgraded (revert-apply).\n\nEach processor has an apply and revert command, so in theory, it should be possible to upgrade scores to a newer version of processing. This was made with the assumption we would gradually be adding new pieces of the puzzle in until we had everything online. If everything is in a good state, this may be less useful.\n\n# Current Versions\n\nThis is part of a group of projects which are used in live deployments where the deployed version is critical to producing correct results. The `master` branch tracks ongoing developments. If looking to use the correct version for matching live values, please [consult this wiki page](https://github.com/ppy/osu-infrastructure/wiki/Star-Rating-and-Performance-Points) for the latest information.\n\n# Contributing\n\nContributions can be made via pull requests to this repository. We hope to credit and reward larger contributions via a [bounty system](https://www.bountysource.com/teams/ppy). If you're unsure of what you can help with, check out the [list of open issues](https://github.com/ppy/osu-queue-score-statistics-processor/issues).\n\nNote that while we already have certain standards in place, nothing is set in stone. If you have an issue with the way code is structured; with any libraries we are using; with any processes involved with contributing, *please* bring it up. I welcome all feedback so we can make contributing to this project as pain-free as possible.\n\n# Environment Setup\n\n## Environment Variables\n\n### BEATMAP_DOWNLOAD_PATH\n\nURI template to download .osu beatmap files. Used for [realtime processing](#realtime_difficulty).\n\nDefaults to `https://osu.ppy.sh/osu/{0}`.\n\n### DB_HOST\n\nHost for MySQL.\n\nDefaults to `localhost`.\n\n### DB_NAME\n\nDatabase name.\n\nDefaults to `osu`.\n\n### DB_NAME_DIFFICULTY\n\nDatabase name for reading difficulty attributes from (all `osu_beatmap_difficulty_attribs` table access).\n\nDefaults to same as `DB_NAME`.\n\n### DB_USER\n\nDatabase username.\n\nDefaults to `root`.\n\n### DB_PASS\n\nDatabase password.\n\n### DB_USERS_TABLE\n\nDatabase users table name; if using the dumps, set this environment variable to \"sample_users\".\n\nDefaults to `phpbb_users`.\n\n### DD_AGENT_HOST\n\nHost to submit DataDog/StatsD metrics to.\n\nDefaults to `localhost`.\n\n### DD_ENTITY_ID\n\nEnables DataDog origin detection when running in a container. See [DataDog documentation](https://docs.datadoghq.com/developers/dogstatsd/?tab=kubernetes\u0026code-lang=dotnet#origin-detection-over-udp).\n\n### SHARED_INTEROP_DOMAIN\n\nRoot domain to which shared interop requests should be directed.\n\n- In Debug configuration, this parameter is not required; when missing, all requests are no-ops.\n- In Release configuration, this parameter is required; all requests will hard-fail with an exception if missing.\n\n### SHARED_INTEROP_SECRET\n\nSecret key used to sign shared interop requests to osu-web. Required to award medals.\n\n- In Debug configuration, this parameter is not required; when missing, all requests are no-ops.\n- In Release configuration, this parameter is required; all requests will hard-fail with an exception if missing.\n\n### PROCESS_USER_MEDALS\n\nWhether to process user medals. Set to `0` to disable processing.\n\nDefault is unset (processing enabled).\n\n### PROCESS_USER_TOTALS\n\nWhether to process user total stats. Set to `0` to disable processing.\n\nDefault is unset (processing enabled).\n\n### REALTIME_DIFFICULTY\n\nWhether to use realtime processing (download beatmaps and compute their difficulty attributes on every processed score), or to rely on database data. Set to `0` to disable processing.\n\nDefault is unset (processing enabled).\n\n### REDIS_HOST\n\nRedis connection string; see [here](https://stackexchange.github.io/StackExchange.Redis/Configuration.html#configuration-options) for configuration options.\n\nDefaults to `localhost`\n\n### SCHEMA\n\nSchema version for the queue; see [Schema](https://github.com/ppy/osu-elastic-indexer#schema).\n\n### WRITE_LEGACY_SCORE_PP\n\nWhether to write a legacy score's PP to `osu_scores_high` if applicable to the score. Set to `0` to disable writing.\n\nDefault is unset (writing enabled).\n\n## BYO\n\nTo setup a testing environment, the minimum requirements are having a MySQL instance available at `localhost:3306` and a redis instance available at `localhost:6379`.\n\nMake sure all data is expendable. It ~~may~~ will be nuked without remorse.\n\n## Docker\n\nYou can setup an environment and have it map to the correct localhost ports:\n\n```shell\ndocker-compose up -d\n```\n\nYou can stop the environment using:\n```shell\ndocker-compose down\n```\n\nAlternatively, you can stop and remove all volumes (ie. nuke your test database) using:\n```shell\ndocker-compose down -v\n```\n\n# Licence\n\nThe osu! client code, framework, and server-side components are licensed under the [MIT licence](https://opensource.org/licenses/MIT). Please see [the licence file](LICENCE) for more information. [tl;dr](https://tldrlegal.com/license/mit-license) you can do whatever you want as long as you include the original copyright and license notice in any copy of the software/source.\n\nPlease note that this *does not cover* the usage of the \"osu!\" or \"ppy\" branding in any software, resources, advertising or promotion, as this is protected by trademark law.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fppy%2Fosu-queue-score-statistics","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fppy%2Fosu-queue-score-statistics","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fppy%2Fosu-queue-score-statistics/lists"}