{"id":18785273,"url":"https://github.com/andreped/super-ml-pets","last_synced_at":"2025-04-13T12:33:48.499Z","repository":{"id":53927000,"uuid":"521754709","full_name":"andreped/super-ml-pets","owner":"andreped","description":"🐢 AI for Super Auto Pets","archived":false,"fork":false,"pushed_at":"2024-01-12T23:38:01.000Z","size":288150,"stargazers_count":33,"open_issues_count":12,"forks_count":16,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-13T05:36:31.462Z","etag":null,"topics":["ai","artificial-intelligence","computer-vision","deep-learning","game-development","hacktoberfest","machine-vision","opencv","reinforcement-learning","sap","sapai","sapai-gym","sb3","super-auto-pets","super-ml-pets","template-matching"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"HJK-Z/Super-Auto-Pets","license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/andreped.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-08-05T19:28:46.000Z","updated_at":"2025-03-05T08:54:17.000Z","dependencies_parsed_at":"2024-01-13T04:56:57.297Z","dependency_job_id":"c19bc2e6-3af5-4790-be00-a9ea398e2e75","html_url":"https://github.com/andreped/super-ml-pets","commit_stats":null,"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andreped%2Fsuper-ml-pets","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andreped%2Fsuper-ml-pets/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andreped%2Fsuper-ml-pets/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andreped%2Fsuper-ml-pets/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/andreped","download_url":"https://codeload.github.com/andreped/super-ml-pets/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248714736,"owners_count":21149958,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","artificial-intelligence","computer-vision","deep-learning","game-development","hacktoberfest","machine-vision","opencv","reinforcement-learning","sap","sapai","sapai-gym","sb3","super-auto-pets","super-ml-pets","template-matching"],"created_at":"2024-11-07T20:46:07.143Z","updated_at":"2025-04-13T12:33:46.731Z","avatar_url":"https://github.com/andreped.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"assets/SMLP.svg\" width=\"50%\" alt='super-auto-pets'\u003e\n\u003ch1 align=\"center\"\u003esuper-ml-pets\u003c/h1\u003e\n\u003ch3 align=\"center\"\u003eFramework for training and deploying AIs for Super Auto Pets\u003c/h3\u003e\n\n[![License](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)\n[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.7834142.svg)](https://doi.org/10.5281/zenodo.7834142)\n![CI](https://github.com/andreped/super-ml-pets/workflows/CI/badge.svg)\n![CodeQL](https://github.com/andreped/super-ml-pets/workflows/CodeQL/badge.svg)\n[![PEP8](https://img.shields.io/badge/code%20style-pep8-orange.svg)](https://www.python.org/dev/peps/pep-0008/)\n[![codecov](https://codecov.io/gh/andreped/super-ml-pets/branch/main/graph/badge.svg?token=9YF7NANQTE)](https://codecov.io/gh/andreped/super-ml-pets)\n \nTrain AIs for Super Auto Pets through a simulated environment and test the trained model against real opponents in the actual game! AI is trained using reinforcement learning and a machine vision system is used to capture the screen to give information to the AI.\n\n\u003c/div\u003e\n\n## [Introduction](https://github.com/andreped/super-ml-pets#introduction)\n\nFramework supports Python `3.7-3.11` and works cross-platform (Ubuntu, Windows, macOS). Deployment is also compatible with the [web app](https://teamwood.itch.io/super-auto-pets).\n\nTraining has also been tested with [GitHub Codespaces](https://github.com/features/codespaces) and [Google Colab](https://colab.research.google.com/). A demonstration of model training can be seen in [this gist](https://colab.research.google.com/gist/andreped/cc0789bd711874f792c0991978b2f981/super-ml-pets-test.ipynb).\n\nWe recommend using **Windows for deployment** as the UNIX-based systems require root permissions to launch the program out-of-the-box.\n\n## [Getting started](https://github.com/andreped/super-ml-pets#getting-started)\n\n1. Clone the repo:\n```\ngit clone https://github.com/andreped/super-ml-pets.git\n```\n\n2. Setup virtual environment:\n```\ncd super-ml-pets/\nvirtualenv -ppython3 venv --clear\n./venv/Scripts/activate\n```\n\nTo activate the virtual environment on UNIX-based systems, instead of the last line run `source venv/bin/activate`\n\n3. Install requirements:\n```\npip install -r requirements.txt\n```\n\n4. Download all pets, food, and misc icons\n```\nwget https://github.com/andreped/super-ml-pets/releases/download/pets-01-2024/pets.zip -O pets.zip; Expand-Archive pets.zip -DestinationPath ./; Remove-Item pets.zip\nwget https://github.com/andreped/super-ml-pets/releases/download/food-01-2024/food.zip -O food.zip; Expand-Archive food.zip -DestinationPath ./; Remove-Item food.zip\nwget https://github.com/andreped/super-ml-pets/releases/download/misc-01-2024/misc.zip -O misc.zip; Expand-Archive misc.zip -DestinationPath ./; Remove-Item misc.zip\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eAdditional setup for Ubuntu only\u003c/summary\u003e\n\n```\nsudo apt install python3-tk\nsudo su\nsource venv/bin/activate\nxhost +\nexport DISPLAY=:0.0\n```\n\nNote that the command `sudo su` enables administrator rights. This seems to be required by `keyboard` as mentioned in issue https://github.com/andreped/super-ml-pets/issues/23. The xhost + DISPLAY stuff is needed as the screen might not be found, hence, initializing one solves this issue.\n\n\u003c/details\u003e\n\n## [Usage](https://github.com/andreped/super-ml-pets#usage)\nThis framework currently supports training and deploying RL models for SAP.\n\n\u003cdetails open\u003e\n\u003csummary\u003eTraining\u003c/summary\u003e\n\nFor training in simulated environment, using default arguments, simply run:\n```\npython main.py --task train\n```\n\nGiven an existing model, it is also possible to finetune it by (with example):\n```\npython main.py --task train --finetune /path/to/model_sap_gym_sb3_180822_checkpoint_2175_steps\n```\n\nThe script supports other arguments. To see what is possible, run:\n```\npython main.py --help\n```\n\n\u003c/details\u003e\n\n\u003cdetails open\u003e\n\u003csummary\u003eTesting\u003c/summary\u003e\n\n1. To use a trained model in battle, start the game Super Auto Pets.\n\n2. Ensure that the game is in full screen mode, disable all unneccessary prompts, enable auto name picker, and set speed to `200%` (you might also have to enable auto battle which can only be done in the first battle - if this is the first time you are playing this game).\n\n3. Change the UI style to classic for all options in customize including \"Food art\", \"Background art\", \"Menu background\", \"Buff style\", and \"Held food\".\n \n4. Change UI style for pets to classic by going to the `pets` settings.\n\n5. Enter the arena by clicking `Arena mode`.\n\n6. Go outside the game and download a pretrained model from [here](https://github.com/andreped/super-ml-pets/releases/tag/v0.0.2), or use any pretrained model you might have. For simplicity, you can also run the following to download a example model:\n```\nwget https://github.com/andreped/super-ml-pets/releases/download/v0.0.6/model_sap_gym_sb3_280822_finetuned_641057_steps.zip\n```\n\n7. Then, simply start the AI by running this command from the terminal (with example path to pretrained model, **without extension .zip**):  \n```\npython main.py --task deploy --infer_model /path/to/model_sap_gym_sb3_280822_finetuned_641057_steps\n```\n\n8. Go back into the game and press the `Space` keyboard button (when you are in the Arena (in team preparation, before battle).\n\nIt might take a few seconds, but you should now be able to see the AI start playing. Please, let it play in peace, or else it might get angry and you may have accidentally created [Skynet](https://en.wikipedia.org/wiki/Skynet_(Terminator)). If you accidentally exit the game, or dont have the game in fullscreen, the machine vision system will fail, and you will have to start a completely new game to use the AI (properly).\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eTraining history\u003c/summary\u003e\n\nTo plot training history, run:\n```\npython smp/plot_history.py --log /path/to/history/history_rl_model/progress.csv\n```\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/training_history_example.png\" width=\"60%\" alt='super-auto-pets'\u003e\n\u003c/p\u003e\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eTroubleshoot\u003c/summary\u003e\n\nTo install virtualenv, run:\n```\npip install virtualenv\n```\n\nIf you do not have virtualenv in the path, you can access it by:\n```\npython -m virtualenv -ppython3 venv --clear\n```\n\nTo activate virtual environment on UNIX-based systems (e.g., macOS or Ubuntu), run:\n```\nsource venv/bin/activate\n```\n\nIf you are using newer versions of Python (e.g., `3.10`), you might have issues with installing and/or using `numpy` with the other dependencies. If that happens, try downgrading numpy by:\n```\npip install numpy==1.23.2 --force-reinstall\n```\n \nOn both Ubuntu and macOS, it might require sudo permissions to run deployment. This has to do with keyboard events not being able to be recognized without\nsudo rights. On Windows, administrative rights is **not needed**. For more information, see [here](https://pynput.readthedocs.io/en/latest/limitations.html).\n \nOn macOS, when you are downloading the models (.zip files) from [Releases](https://github.com/andreped/super-ml-pets/releases), they might be unzipped automatically. This is **bad** as the model extension is `.zip`. To fix this, disable the `Open safe files after downloading` in the Safari Preferences (see [here](https://www.lifewire.com/disable-open-safe-files-after-downloading-in-safari-446562) for more information).\n \nIf deployment fails to start (no mouse movements or events), it may be because your screen resolution differ from the expected resolution. The current machine vision system expects the screen resolution to be `1920x1080`. Please, adjust the resolution to this. This will be fixed in the future.\n\n\u003c/details\u003e\n\n## [Acknowledgements](https://github.com/andreped/super-ml-pets#acknowledgements)\nThis implementation is based on multiple different projects. The core implementation is derived from [GoldExplosion](https://github.com/GoldExplosion/SuperAutoPets-RL-Agent), which further was based upon the super auto pets engine [sapai](https://github.com/manny405/sapai) and RL training through [sapai-gym](https://github.com/alexdriedger/sapai-gym).\n\nAll credit to [jpdefrutos](https://github.com/jpdefrutos) for designing the amazing [header figure](https://github.com/andreped/super-ml-pets/blob/main/assets/SMLP.svg).\n\n## [Citation](https://github.com/andreped/super-ml-pets#citation)\nIf you found this project relevant for your research, please, cite the following:\n```\n@software{andre_pedersen_2023_7834142,\n  author       = {André Pedersen and Javier Pérez de Frutos and laughinggaschambers and GoldExplosion},\n  title        = {andreped/super-ml-pets: v0.0.9},\n  month        = apr,\n  year         = 2023,\n  publisher    = {Zenodo},\n  version      = {v0.0.9},\n  doi          = {10.5281/zenodo.7834142},\n  url          = {https://doi.org/10.5281/zenodo.7834142}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandreped%2Fsuper-ml-pets","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fandreped%2Fsuper-ml-pets","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandreped%2Fsuper-ml-pets/lists"}