{"id":19813900,"url":"https://github.com/nowke/wumpus-rl","last_synced_at":"2026-03-07T17:03:07.860Z","repository":{"id":97511955,"uuid":"257751543","full_name":"nowke/wumpus-rl","owner":"nowke","description":"Wumpus World Reinforcement Learning - Deep Q Network, based on OpenAI-Gym","archived":false,"fork":false,"pushed_at":"2020-05-01T09:48:30.000Z","size":1132,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-11T07:35:36.166Z","etag":null,"topics":["deep-q-learning","openai-gym","reinforcement-learning","wumpusworld"],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nowke.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-04-22T00:46:05.000Z","updated_at":"2024-04-23T13:19:29.000Z","dependencies_parsed_at":"2024-03-15T07:32:35.217Z","dependency_job_id":null,"html_url":"https://github.com/nowke/wumpus-rl","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nowke%2Fwumpus-rl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nowke%2Fwumpus-rl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nowke%2Fwumpus-rl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nowke%2Fwumpus-rl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nowke","download_url":"https://codeload.github.com/nowke/wumpus-rl/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241162460,"owners_count":19920410,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-q-learning","openai-gym","reinforcement-learning","wumpusworld"],"created_at":"2024-11-12T09:37:37.235Z","updated_at":"2026-03-07T17:03:02.810Z","avatar_url":"https://github.com/nowke.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Wumpus RL\n\nThe codebase consists of 2 parts:\n\n1. Wumpus World environment ([gym-wumpus/](gym-wumpus/)) - compatible with [OpenAI Gym](https://gym.openai.com/)\n    \u003e This wraps around Code from [Project 3: Using Logic to Hunt the Wumpus](http://www.sista.arizona.edu/~clayton/courses/ai/projects/wumpus/)\n2. DQN algorithm([dqn/](dqn/)) - for training Wumpus world environment\n\n## Setup Python Environment\n\n#### Create new environment with Python 3.7\n\n```sh\nconda create -n wumpus-rl python=3.7\nconda activate wumpus-rl\n```\n\n#### Install packages\n\n```sh\npip install -r requirements.txt\npip install -e gym-wumpus\n```\n\n## Using Wumpus Environment\n\n* The sample code uses `wumpus-v0` environment that is defined inside `gym-wumpus/` folder. \n* This assumes you have already run `pip install -e gym-wumpus` command, which installs the wumpus world as a dependency inside the Python virtual environment.\n\n#### Sample usage execution\n\n```sh\n\u003e\u003e\u003e import gym_wumpus   # To be imported before `gym`\n\u003e\u003e\u003e import gym\n\n\u003e\u003e\u003e env = gym.make('wumpus-v0') # Initialize wumpus environment\n\u003e\u003e\u003e env.reset()\narray([1, 1, 0, 0, 0, 0, 0, 0], dtype=uint32)\n\n\u003e\u003e\u003e env.render()\nScores: \u003cExplorer\u003e=0\n  0   1   2   3   4   5    time_step=0\n|---|---|---|---|---|---|\n| # | # | # | # | # | # | 5\n|---|---|---|---|---|---|\n| # |   |   |   |   | # | 4\n|---|---|---|---|---|---|\n| # | W | G | P |   | # | 3\n|---|---|---|---|---|---|\n| # |   |   |   |   | # | 2\n|---|---|---|---|---|---|\n| # | ^ |   | P |   | # | 1\n|---|---|---|---|---|---|\n| # | # | # | # | # | # | 0\n|---|---|---|---|---|---|\n\n\u003e\u003e\u003e env.step(2)  # Forward action\n(array([1, 2, 0, 1, 0, 0, 0, 0], dtype=uint32), -1, False, {})\n\n\u003e\u003e\u003e env.render()\nScores: \u003cExplorer\u003e=-1\n  0   1   2   3   4   5    time_step=1\n|---|---|---|---|---|---|\n| # | # | # | # | # | # | 5\n|---|---|---|---|---|---|\n| # |   |   |   |   | # | 4\n|---|---|---|---|---|---|\n| # | W | G | P |   | # | 3\n|---|---|---|---|---|---|\n| # | ^ |   |   |   | # | 2\n|---|---|---|---|---|---|\n| # |   |   | P |   | # | 1\n|---|---|---|---|---|---|\n| # | # | # | # | # | # | 0\n|---|---|---|---|---|---|\n```\n\n#### Modifying `gym-wumpus` environment\n\n* [`gym-wumpus/gym_wumpus/envs/wumpus_env.py`](gym-wumpus/gym_wumpus/envs/wumpus_env.py) - Defines the `gym.Env` class skeleton for Wumpus World environment. Modify the class to change the behavior.\n\nOnce modified, you can use the new enviornment in **two** ways:\n\n1. Reinstall `gym-wumpus` package from the root of the repository\n\n```sh\npip install -e gym-wumpus\n```\n\n2. Use the environment directly from the folder (no need to reinstall again and again)\n\n```python\nimport sys\nimport gym\nsys.path.append('gym-wumpus')  \n# NOTE: This assumes you are running this in root of repository\n# You can give relative paths or absolute path to `gym-wumpus` folder\n\nfrom gym_wumpus.envs import WumpusWorld\n\nenv = WumpusWorld()\n\n# You can use `env` object just as a regular `gym` environment\nenv.render()\n\n# Pass `rgb_array` to the `render` method to get numpy array\n# of the rendered image --\u003e this can be used to generate GIFs\nnp_arr_img = env.render('rgb_array')\n```\n\n## Using DQN code\n\n#### Files\n\n* [`dqn/dqn.py`](dqn/dqn.py) - DQN Agent code\n* [`dqn/utils.py`](dqn/utils.py) - Utility functions\n* [`dqn/wumpus_dqn.py`](dqn/wumpus_dqn.py) - Training code for Wumpus world environment using DQN\n* [`dqn/test_wumpus_dqn.py`](dqn/test_wumpus_dqn.py) - Testing code for Wumpus world environment with DQN, generates GIFs using checkpoints\n\n#### Running code\n\n* Run [`dqn/wumpus_dqn.py`](dqn/wumpus_dqn.py) file, \n* Give `env_id` (example, `wumpus-v0`) to `setup.sh` file\n* Set `ENV_NAME` variable in `wumpus_dqn.py`\n\n```\ncd dqn\n./setup.sh wumpus-v0\npython wumpus_dqn.py\n```\n\nRun `clean.sh` to clear out the generated `logs`, `models`, and `tests`\n\n```sh\n./clean.sh\n```\n\n#### Hyperparameters\n\nChange the hyperparameters inside [`dqn/wumpus_dqn.py`](dqn/wumpus_dqn.py) file\n\n```python\n\n...\n\nEPISODES = 35000\n...\n\nagent = Agent(learning_rate=0.01, gamma=0.95,\n              state_shape=env.observation_space.shape, actions=7,\n              batch_size=64,\n              epsilon_initial=0.9, epsilon_decay=1e-6, epsilon_final=0.01,\n              replay_buffer_capacity=1000000,\n              ...)\n\n...\n\n```\n\n#### Testing an existing model\n\nTest any environment using the pretrained existing models. Run `python test_wumpus_dqn.py \u003cenv_name\u003e`. For example,\n\n```sh\npython test_wumpus.py wumpus-noise-v0\n```\n\n#### List of environments\n\n| Environment                  | Noise (%) | Modified reward function (?) | Grid # |\n|------------------------------|-----------|------------------------------|--------|\n| `wumpus-v0`                  | 0         | Yes                          | 1      |\n| `wumpus-nr-v0`               | 0         | No                           | 1      |\n| `wumpus-noise2-v0`           | 10        | Yes                          | 1      |\n| `wumpus-nr-noise2-v0`        | 10        | No                           | 1      |\n| `wumpus-noise-v0`            | 20        | Yes                          | 1      |\n| `wumpus-nr-noise-v0`         | 20        | No                           | 1      |\n| `wumpus-l4x4_1-v0`           | 0         | Yes                          | 2      |\n| `wumpus-l4x4_1-nr-v0`        | 0         | No                           | 2      |\n| `wumpus-l4x4_1-noise2-v0`    | 10        | Yes                          | 2      |\n| `wumpus-l4x4_1-nr-noise2-v0` | 10        | No                           | 2      |\n| `wumpus-l4x4_1-noise-v0`     | 20        | Yes                          | 2      |\n| `wumpus-l4x4_1-nr-noise-v0`  | 20        | No                           | 2      |\n| `wumpus-l4x4_2-v0`           | 0         | Yes                          | 3      |\n| `wumpus-l4x4_2-nr-v0`        | 0         | No                           | 3      |\n| `wumpus-l4x4_2-noise2-v0`    | 10        | Yes                          | 3      |\n| `wumpus-l4x4_2-nr-noise2-v0` | 10        | No                           | 3      |\n| `wumpus-l4x4_2-noise-v0`     | 20        | Yes                          | 3      |\n| `wumpus-l4x4_2-nr-noise-v0`  | 20        | No                           | 3      |\n| `wumpus-l5x5_1-v0`           | 0         | Yes                          | 4      |\n| `wumpus-l5x5_1-nr-v0`        | 0         | No                           | 4      |\n| `wumpus-l5x5_1-noise2-v0`    | 10        | Yes                          | 4      |\n| `wumpus-l5x5_1-nr-noise2-v0` | 10        | No                           | 4      |\n| `wumpus-l5x5_1-noise-v0`     | 20        | Yes                          | 4      |\n| `wumpus-l5x5_1-nr-noise-v0`  | 20        | No                           | 4      |\n\n```\n       GRID #1                            GRID #2\n  0   1   2   3   4   5            0   1   2   3   4   5\n|---|---|---|---|---|---|        |---|---|---|---|---|---|\n| # | # | # | # | # | # | 5      | # | # | # | # | # | # | 5\n|---|---|---|---|---|---|        |---|---|---|---|---|---|\n| # |   |   |   |   | # | 4      | # |   |   |   |   | # | 4\n|---|---|---|---|---|---|        |---|---|---|---|---|---|\n| # | W | G | P |   | # | 3      | # | W | G |   |   | # | 3\n|---|---|---|---|---|---|        |---|---|---|---|---|---|\n| # |   |   |   |   | # | 2      | # |   | P |   |   | # | 2\n|---|---|---|---|---|---|        |---|---|---|---|---|---|\n| # | ^ |   | P |   | # | 1      | # | ^ |   | P |   | # | 1\n|---|---|---|---|---|---|        |---|---|---|---|---|---|\n| # | # | # | # | # | # | 0      | # | # | # | # | # | # | 0\n|---|---|---|---|---|---|        |---|---|---|---|---|---|\n\n       GRID #3                           GRID #4\n\n  0   1   2   3   4   5          0   1   2   3   4   5   6\n|---|---|---|---|---|---|        |---|---|---|---|---|---|---|\n| # | # | # | # | # | # | 5      | # | # | # | # | # | # | # | 6  \n|---|---|---|---|---|---|        |---|---|---|---|---|---|---|\n| # |   |   | W | G | # | 4      | # |   | G |   |   |   | # | 5  \n|---|---|---|---|---|---|        |---|---|---|---|---|---|---|\n| # |   |   | P |   | # | 3      | # |   |   | W |   |   | # | 4\n|---|---|---|---|---|---|        |---|---|---|---|---|---|---|\n| # |   |   |   |   | # | 2      | # |   |   | P |   |   | # | 3  \n|---|---|---|---|---|---|        |---|---|---|---|---|---|---|\n| # | ^ |   |   | P | # | 1      | # |   |   |   |   |   | # | 2  \n|---|---|---|---|---|---|        |---|---|---|---|---|---|---|\n| # | # | # | # | # | # | 0      | # | ^ |   |   | P |   | # | 1  \n|---|---|---|---|---|---|        |---|---|---|---|---|---|---|\n                                 | # | # | # | # | # | # | # | 0\n                                 |---|---|---|---|---|---|---|  \n```\n\n#### Tensorboard\n\nDuring training, logs are generated in `dqn/logs/` folder. To view the Tensorboard, run the below command\n\n```sh\ntensorboard --logdir dqn/logs/\n```\n\n![Tensorboard screenshot 1](assets/tensorboard_screen_1.png)\n![Tensorboard screenshot 2](assets/tensorboard_screen_2.png)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnowke%2Fwumpus-rl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnowke%2Fwumpus-rl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnowke%2Fwumpus-rl/lists"}