{"id":18004720,"url":"https://github.com/oscarknagg/wurm","last_synced_at":"2025-03-26T10:31:34.199Z","repository":{"id":37599531,"uuid":"167426236","full_name":"oscarknagg/wurm","owner":"oscarknagg","description":"Massively multiagent reinforcement learning in a slither.io like environment","archived":false,"fork":false,"pushed_at":"2022-12-08T05:07:37.000Z","size":925,"stargazers_count":24,"open_issues_count":4,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-21T15:12:26.525Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oscarknagg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-01-24T19:41:59.000Z","updated_at":"2024-05-19T08:52:49.000Z","dependencies_parsed_at":"2023-01-24T11:30:16.076Z","dependency_job_id":null,"html_url":"https://github.com/oscarknagg/wurm","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oscarknagg%2Fwurm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oscarknagg%2Fwurm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oscarknagg%2Fwurm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oscarknagg%2Fwurm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oscarknagg","download_url":"https://codeload.github.com/oscarknagg/wurm/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245636529,"owners_count":20647986,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-30T00:15:34.336Z","updated_at":"2025-03-26T10:31:33.886Z","avatar_url":"https://github.com/oscarknagg.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# wurm\n\nVectorised and massively scalable implementation of a snake-like game\nas a reinforcement learning environment. The environment is highly\ncustomisable and can be run in either single-agent (classic snake)\nor multi-agent (gridworld http://slither.io/) mode.\n\n### Multi-agent\n\nExperiments ongoing! Here are some prelimiary results.\n\n![Results](https://media.giphy.com/media/9DatBqefTicuQHVYVd/giphy.gif)\n\n### Single agent\n\nSee this Medium article for a discussion of how to solve the single\nagent mode of this environment.\n\nhttps://medium.com/@oknagg/learning-to-play-snake-at-1-million-fps-4aae8d36d2f1\n\nSome training results are shown below\n\n![Results](https://media.giphy.com/media/x003Vu0wXLvQQxq9ft/giphy.gif)\n\n## Setup\n\nClone this project, create a Python 3.6 virtualenv and install\nfrom `requirements.txt`.\n\n```\ngit clone https://github.com/oscarknagg/wurm.git\ncd wurm\nvirtualenv -p python3.6 venv\nsource venv/bin/activate\npip install -r requirements.txt\n```\n\n## Usage\n\n### main.py\n\n`main.py` is the entry point for training and visualising agents. Below\nis a list of arguments.\n\n* env: What environment to run. Choose from {snake, gridworld}\n    - gridworld: Simple gridworld containing only a single agent pixel\n    and a reward pixel that respawns in a random location when reached.\n    Useful for debugging as is solved very quickly.\n    - snake: Clone of the classic mobile game snake.\n* num-envs: How many environments to run in parallel.\n* size: Size of environment in pixels\n* agent: Either the agent architecture to use or a filepath to a\npretrained model.\n    - random: performs random actions only.\n    - convolutional: 4 convolutional layers.\n    - relational: 2 convolutional layers and 2 spatial self-attention layers.\n    - feedforward: 2 feedforward layers.\nrelational, feedforward}\n* train: Whether to train the agent with A2C or not\n* observation: What observation type to use.\n    - default: RGB image\n    - raw: 3 channel image (head, food, body)\n    - partial_n: Flattened version of the environment within n pixels of the\n    head of each snake. Makes the environment partially observable\n* coord-conv: Whether or not to add 2 channels indicating the (x, y) position\nof the pixel to the input.\n* render: Whether or not to render the environment in a human visible way\n* render-window-size: Size of each rendered environment in pixels\n* render-{rows, cols}: Number of environments to render in each direction.\nDefaults to (1, 1). Using larger values will render multiple envs\nsimultaneously provided num-envs is large enough.\n* lr: Learning rate.\n* gamma: Discount.\n* update-steps: Length of trajectories to use when calculating value and\npolicy loss.\n* entropy: Entropy regularisation coefficient.\n* total-{steps, episodes}: Total number of environment steps or episodes\nto run before exiting.\n* save-location: Where to save results such as logs, models and videos.\nThese will save in `logs/$SAVE_LOCATION.csv`, `models/$SAVE_LOCATION.pt`\n and `videos/$SAVE_LOCATION/$EPISODE.mp4` respectively. If left blank a\n save location will be automatically generated based on model parameters.\n* save-{logs, models, video}: Whether or not to save the specified objects.\n* device: Device to run model and environment.\n\n### Example 1 - Training an agent from scratch\n\nRun the following command from the root of this directory to train an\nagent with A2C. It should achieve an\naverage length of 10 in around 10 million steps. This takes about four\nminutes on a 1080 Ti. If this command requires too much memory reduce the\nnum-envs argument.\n\n```\npython -m experiments.main --agent feedforward --env snake --num-envs 512 --size 9 \\\n    --observation partial_2 --update-steps 40 --entropy 0.01 --total-steps 10e6 \\\n    --save-logs True --save-model True --lr 0.0005 --gamma 0.99\n```\n\nTraining curve should look something like this (1 run).\n\n![Imgur](https://i.imgur.com/bmQPWHy.png)\n\nRun this command to visualise the results\n```\npython -m experiments.main \\\n    --agent env=snake__num_envs=512__size=9__agent=feedforward__observation=partial_2__coord_conv=True__lr=0.0005__gamma=0.99__update_steps=40__entropy=0.01__total_steps=10000000.0.pt \\\n    --env snake --num-envs 1 --size 9 --total-episodes 5 --save-model False --save-logs False  --render True --train False\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foscarknagg%2Fwurm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foscarknagg%2Fwurm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foscarknagg%2Fwurm/lists"}