{"id":22276815,"url":"https://github.com/jakjus/csgomath_engine","last_synced_at":"2026-05-04T23:33:32.664Z","repository":{"id":113987315,"uuid":"454850616","full_name":"jakjus/csgomath_engine","owner":"jakjus","description":"Backend of web application CS:GO Math - Data analysis of Counter-Strike: Global Offensive Market 📈","archived":false,"fork":false,"pushed_at":"2022-02-18T22:58:33.000Z","size":35,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-06-24T10:44:20.017Z","etag":null,"topics":["counter-strike-global-offensive","docker-compose","etl","mariadb","python","requests","scraper","statistics"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jakjus.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-02-02T16:28:50.000Z","updated_at":"2022-02-06T22:05:31.000Z","dependencies_parsed_at":null,"dependency_job_id":"5809cf6c-5ddc-4480-b248-cd0a8954a232","html_url":"https://github.com/jakjus/csgomath_engine","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jakjus/csgomath_engine","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakjus%2Fcsgomath_engine","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakjus%2Fcsgomath_engine/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakjus%2Fcsgomath_engine/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakjus%2Fcsgomath_engine/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jakjus","download_url":"https://codeload.github.com/jakjus/csgomath_engine/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakjus%2Fcsgomath_engine/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32628830,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-04T10:08:07.713Z","status":"ssl_error","status_checked_at":"2026-05-04T10:08:02.005Z","response_time":58,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["counter-strike-global-offensive","docker-compose","etl","mariadb","python","requests","scraper","statistics"],"created_at":"2024-12-03T14:16:14.747Z","updated_at":"2026-05-04T23:33:32.650Z","avatar_url":"https://github.com/jakjus.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## CS:GO Math Engine\n\nBackend of web application \"CS:GO Math\" - interacting with [Counter-Strike: Global Offensive Market](https://steamcommunity.com/market/search?appid=730).\n\nConsists of four main components:\n- **Scraper** - scrapes (reads) CS:GO market data from [Valve's website](https://steamcommunity.com/market/search?appid=730).\n\n- **Extractor** - extracts cases from all items and calculate expected value of each case.\n\n- **Uploader** - uploads extracted data to database *(MariaDB)*.\n\n- **API** - serves non-sensitive database data to frontend *(Flask)*.\n\n## Motivation\n*Valve* does not officially offer API for market, hence scraping techniques are used. \n\nBy using extractor, end users may know what is the percentage return on each type of case, by comparing `case + key price` to `expected value`. Additionally, they can compare across other cases and decide which case is most worthwhile to buy.\n\n\u003e Example presentation is available at [csgomath.com](https://csgomath.com) (to be rebuilt)\n\n## Installation\n### In-stack (docker-compose)\nInstall:\n- docker\n- docker-compose\n\n### Manual\nInstall necessary dependencies:\n```bash\npip install -r scraper_extractor/requirements.txt\npip install -r api/requirements.txt\n```\n\n## Usage\n### In-stack (docker-compose)\nCreate your `.env` file from template and adjust variables if needed:\n```\ncp .env.example .env\n```\n\nStack initialization will:\n- run scrape+extract+upload with cron every 12 hours (default)\n- init and run DB\n- run API\n\nUse:\n```\ndocker-compose up -d\n```\n\n### Manual\nRun scraper + extractor + upload:\n```\npython scraper_extractor/main.py -seu\n```\n\nFlags:\n- `--scrape, -s` Scrape.\n- `--extract, -e` Extract.\n- `--upload, -u` Upload.\n\nRun API:\n```\ncd api\ngunicorn -b 0.0.0.0:3010 app:app\n```\n\n## API endpoints\n- `/api/cases` case and key data with prices history (intended use: front page)\n- `/api/case_description/\u003ccase_id\u003e` case description with price history for each description element (item)\n- `/api/key_description/\u003ckey_id\u003e` key description\n\n*(By Valve's default, all keys have `sale_price: 250`, which is 2.5$. Keys prices are neither included in database nor api and are subject to interpret on frontend side.)*\n\n## Further Description\n**Expected value** is an arithmetic mean of a large number of independent realization of random variable, which in this case is opening a case or a more specific random variable, like obtaining some version of one weapon. *(Expected value may be also called **estimated value** in parts of code.)*\n\nIf calculated **expected value** of a `Wooden Case` is 1.23$, it means, that if `100 Wooden Case`'s are opened, the opener gets items worth \n\n```\n100 * 1.23$ = 123$\n```\n\nIf calculated **expected value** of a `Dragonbreath M4A1` is 8.92$, it means, that if `100 Dragonbreath M4A1` are looted, the opener gets `Dragonbreath M4A1`'s worth \n```\n100 * 8.92$ = 892$\n```\n(in many versions: more or less used, StatTrak or not).\n\n\n#### Scraper:\n1. Send `GET` request to market URL with `appid=\u003cCS:GO game id\u003e` as parameter.\n2. Loop over market pages with sleep period to avoid \n3. Append items from each page to instance's variable\n\n#### Extractor:\n1. Find `cases` in `all items`.\n2. Read through `case description` and parse all items, that may be in the case.\n3. Find each item from `description` and calculate `one item estimated value`\n    - *(Calculating `item value` depends on the item type. It is based on odds provided by Valve. StatTrak/Usage odds are included.)*\n    - Use `sale price` instead of `sell price`. \n\u003e**Sale** price is the price that the item was lately sold for; **sell** price is the current lowest sell offer. It is an important distinction especially in very rare items: If only one piece of `Superhuman Gloves` is on sale for 1000000$ - `sell_price` is 1000000$ (inaccurate). Two weeks ago, though, someone has bought `Superhuman Gloves` for just 80$ - `sale_price` is 80$ (accurate).\n4. Add `total` field in each of the element in case description\n5. Add `total` field and `timestamp` in each whole case.\n6. Altered list is available in scraper's instance variable `cases`.\n\n## Contributing\nPull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.\n\n## License\n[MIT](./LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjakjus%2Fcsgomath_engine","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjakjus%2Fcsgomath_engine","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjakjus%2Fcsgomath_engine/lists"}