{"id":17383813,"url":"https://github.com/numblr/drlnd-banananav","last_synced_at":"2026-02-18T14:34:15.279Z","repository":{"id":79291316,"uuid":"168344563","full_name":"numblr/drlnd-banananav","owner":"numblr","description":"DQN/DDQN Reinforcement Learning Agent for Navigation ","archived":false,"fork":false,"pushed_at":"2019-02-04T19:40:39.000Z","size":1409,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-10T00:10:50.170Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/numblr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-01-30T13:06:46.000Z","updated_at":"2019-02-04T19:40:41.000Z","dependencies_parsed_at":"2023-05-25T04:45:29.876Z","dependency_job_id":null,"html_url":"https://github.com/numblr/drlnd-banananav","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/numblr/drlnd-banananav","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/numblr%2Fdrlnd-banananav","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/numblr%2Fdrlnd-banananav/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/numblr%2Fdrlnd-banananav/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/numblr%2Fdrlnd-banananav/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/numblr","download_url":"https://codeload.github.com/numblr/drlnd-banananav/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/numblr%2Fdrlnd-banananav/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29582318,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-18T13:56:48.962Z","status":"ssl_error","status_checked_at":"2026-02-18T13:54:34.145Z","response_time":162,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-16T07:43:53.570Z","updated_at":"2026-02-18T14:34:10.270Z","avatar_url":"https://github.com/numblr.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"[//]: # (Image References)\n\n[image1]: https://user-images.githubusercontent.com/10624937/42135619-d90f2f28-7d12-11e8-8823-82b970a54d7e.gif \"Trained Agent\"\n\n# DQN/DDQN Reinforcement Learning Agent for Navigation\n\n### Introduction\n\nThis project is taken from the [Udacity DRLND nano degree program](https://github.com/udacity/deep-reinforcement-learning/tree/master/p1_navigation) and implements an agent to navigate (and collect bananas!) in a large, square world.  \n\n![Trained Agent][image1]\n([source](https://user-images.githubusercontent.com/10624937/42135619-d90f2f28-7d12-11e8-8823-82b970a54d7e.gif))\n\n#### General setting (see also the original [repo](https://github.com/udacity/deep-reinforcement-learning/tree/master/p1_navigation))\n\nThe agent navigates in a predefined [*Unity* environment](https://github.com/Unity-Technologies/ml-agents) that represents a square world were it has to collect bananas. A reward of +1 is provided for collecting a yellow banana, and a reward of -1 is provided for collecting a blue banana.  Thus, the goal of your agent is to collect as many yellow bananas as possible while avoiding blue bananas.  \n\nThe state space of the agent's environment is represented by a 37 dimensional vector or real numbers and contains the agent's velocity, along with ray-based perception of objects around agent's forward direction.  Given this information, the agent has to learn how to best select actions. Four discrete actions are available, corresponding to:\n- **`0`** - move forward.\n- **`1`** - move backward.\n- **`2`** - turn left.\n- **`3`** - turn right.\n\nThe task is episodic, and the environment is considered solved if the agent is able to achieve an average score of +13 over 100 consecutive episodes.\n\n#### Deep reinforcement learning agent\n\nThis repository provides an agent implementation that can be trained to solve the task described in [General settings]($general-settings) using deep reinforcement learning. In particular implementations of two variants of neural network based Q-learning algorithms (see e.g. [Q-learning](https://en.wikipedia.org/wiki/Q-learning)) are provided:\n- Q-learning with a DNN as approximator (DQN) for the Q-function using experience replay,\n- Double Q-learning with a DNN as approximator (DDQN)  using experience replay.\n\nIn addition to this algorithm variants two different architectures of DNNs as function approximators are implemented,\n- a simple variant with two hidden layers of 64 units each, and\n- a more advanced autoencoder-like architecture that is extended with two additional hidden layers with 16 and 8 units at the  input side of the DNN.\n\nThe implementation can execute learning and compare a selection of combinations of algorithm variant and network architecture. Once the agent is trained it can replay an episode of using the parameters stored after learning. A discussion of the results can be found in `Report.ipynb` (or `Report.html` respectively). For a description on how to run the program see the [Instructions](#instructions) section below.\n\n### Getting started (see also the original [repo](https://github.com/udacity/deep-reinforcement-learning/tree/master/p1_navigation))\n\nTo set up your python environment to run the code in this repository, follow this steps:\n\n1. Create (and activate) a new environment with Python 3.6.\n\n\t- __Linux__ or __Mac__:\n\t```bash\n\tconda create --name banananav python=3.6\n\tsource activate banananav\n\t```\n\t- __Windows__:\n\t```bash\n\tconda create --name banananav python=3.6\n\tactivate banananav\n\t```\n\n3. Then install the dependencies from the requirements.txt file in this repository:\n```bash\npip install .\n```\n\n4. Download the *Unity* environment from one of the links below. You must select **only** the environment that matches your operating system:\n    - Linux: [click here](https://s3-us-west-1.amazonaws.com/udacity-drlnd/P1/Banana/Banana_Linux.zip)\n    - Mac OSX: [click here](https://s3-us-west-1.amazonaws.com/udacity-drlnd/P1/Banana/Banana.app.zip)\n    - Windows (32-bit): [click here](https://s3-us-west-1.amazonaws.com/udacity-drlnd/P1/Banana/Banana_Windows_x86.zip)\n    - Windows (64-bit): [click here](https://s3-us-west-1.amazonaws.com/udacity-drlnd/P1/Banana/Banana_Windows_x86_64.zip)\n\n    (_For Windows users_) Check out [this link](https://support.microsoft.com/en-us/help/827218/how-to-determine-whether-a-computer-is-running-a-32-bit-version-or-64) if you need help with determining if your computer is running a 32-bit version or 64-bit version of the Windows operating system.\n\n5. Place the file in the `banananav/resources/` folder, and unzip (or decompress) the file.\n\n6. To be able to run the `Report.ipynb` notebook, create an [IPython kernel](http://ipython.readthedocs.io/en/stable/install/kernel_install.html) for the `banananav` environment.  \n```bash\npython -m ipykernel install --user --name banananav --display-name \"banananav\"\n```\n\n7. Before running code in the notebook, change the kernel to match the `banananav` environment by using the drop-down `Kernel` menu.\n\n### Instructions\n\nThe main script in the repository is `banananavigation.py`, which can be invoked to train the agent and store the result, as well as to replay the agent on a single episode from the stored learning results.\n\nAfter all dependencies are set up as described in the [Getting started](#getting-started) section, the script can be invoked from the root directory of the repository as e.g.\n```bash\n\u003e python banananavigation.py --settings DQN,DDQN,DQN_SIMPLE,DDQN_SIMPLE --episodes 1000\n```\nto execute learning and\n```bash\n\u003e python banananavigation.py --replay\n```\nWhen invoked without command line parameters, the script will execute learning for all algorithm + network architecture combinations on 2500 episodes each. For help on the possible command line options execute the script with the --help option.\n\n### Results\n\nResults of training the different algorithm and models can be found in the *results/* directory. In particular the *results/xxx_model.parameters* files contain the trained parameters of the different models.\n\nThe included results achieve the target of an average score of +13 in less than 600 episodes and reach a top average score of +17.4. The parameters for the model with the best result are also stored in the *ddqn_model.parameters.pt* file in the root of the repository.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnumblr%2Fdrlnd-banananav","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnumblr%2Fdrlnd-banananav","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnumblr%2Fdrlnd-banananav/lists"}