{"id":17383816,"url":"https://github.com/numblr/drlnd-coball","last_synced_at":"2026-01-28T19:02:08.070Z","repository":{"id":79291324,"uuid":"175368273","full_name":"numblr/drlnd-coball","owner":"numblr","description":"Multi-agent reinforcement learning with PPO","archived":false,"fork":false,"pushed_at":"2019-03-19T18:04:27.000Z","size":2951,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-05-31T19:31:56.537Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/numblr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-03-13T07:26:09.000Z","updated_at":"2019-11-23T09:37:07.000Z","dependencies_parsed_at":"2023-05-25T04:45:11.810Z","dependency_job_id":null,"html_url":"https://github.com/numblr/drlnd-coball","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/numblr/drlnd-coball","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/numblr%2Fdrlnd-coball","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/numblr%2Fdrlnd-coball/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/numblr%2Fdrlnd-coball/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/numblr%2Fdrlnd-coball/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/numblr","download_url":"https://codeload.github.com/numblr/drlnd-coball/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/numblr%2Fdrlnd-coball/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28849363,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-28T15:15:36.453Z","status":"ssl_error","status_checked_at":"2026-01-28T15:15:13.020Z","response_time":57,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-16T07:43:54.144Z","updated_at":"2026-01-28T19:02:08.054Z","avatar_url":"https://github.com/numblr.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Competition and collaboration in Reinforcement Learning\n\nThis repository is based on the competition and collaboration problem studied in the [Deep Reinforcement Learning Nanodegree program](https://github.com/udacity/deep-reinforcement-learning/tree/master/p1_navigation) and demonstrates training of a simple multi-agent system.\n\n### General Settings (see also the original [repo](https://github.com/udacity/deep-reinforcement-learning/tree/master/p3_collab-compet))\n\nFor this project, you will work with the [Tennis](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Learning-Environment-Examples.md#tennis) environment.\n\n![Trained Agent](resources/agent.gif)\n\nIn this environment, two agents control rackets to bounce a ball over a net. If an agent hits the ball over the net, it receives a reward of +0.1.  If an agent lets a ball hit the ground or hits the ball out of bounds, it receives a reward of -0.01.  Thus, the goal of each agent is to keep the ball in play.\n\nThe observation space consists of 8 variables corresponding to the position and velocity of the ball and racket. Each agent receives its own, local observation.  Two continuous actions are available, corresponding to movement toward (or away from) the net, and jumping.\n\nThe task is episodic, and in order to solve the environment, your agents must get an average score of +0.5 (over 100 consecutive episodes, after taking the maximum over both agents). Specifically,\n\n- After each episode, we add up the rewards that each agent received (without discounting), to get a score for each agent. This yields 2 (potentially different) scores. We then take the maximum of these 2 scores.\n- This yields a single **score** for each episode.\n\n### Solving the Environment\n\nThe environment is considered solved, when the average (over 100 episodes) of those **scores** is at least +0.5.\n\n### Getting started (see also the original [repo](https://github.com/udacity/deep-reinforcement-learning/tree/master/p3_collab-compet))\n\nTo set up your python environment to run the code in this repository, follow this steps:\n\n1. Create (and activate) a new environment with Python 3.6.\n\n\t- __Linux__ or __Mac__:\n\t```bash\n\tconda create --name coball python=3.6\n\tsource activate coball\n\t```\n\t- __Windows__:\n\t```bash\n\tconda create --name coball python=3.6\n\tactivate coball\n\t```\n\n3. Then install the dependencies from the requirements.txt file in this repository:\n```bash\npip install requirements.txt\n```\n\n4. Download the *Unity* environment from one of the links below. You must select **only** the environment that matches your operating system:\n  - Linux: [click here](https://s3-us-west-1.amazonaws.com/udacity-drlnd/P3/Tennis/Tennis_Linux.zip)\n  - Mac OSX: [click here](https://s3-us-west-1.amazonaws.com/udacity-drlnd/P3/Tennis/Tennis.app.zip)\n  - Windows (32-bit): [click here](https://s3-us-west-1.amazonaws.com/udacity-drlnd/P3/Tennis/Tennis_Windows_x86.zip)\n  - Windows (64-bit): [click here](https://s3-us-west-1.amazonaws.com/udacity-drlnd/P3/Tennis/Tennis_Windows_x86_64.zip)\n\n(_For Windows users_) Check out [this link](https://support.microsoft.com/en-us/help/827218/how-to-determine-whether-a-computer-is-running-a-32-bit-version-or-64) if you need help with determining if your computer is running a 32-bit version or 64-bit version of the Windows operating system.\n\n5. Place and unzip the downloaded file into one of the following locations corresponding to your operating system:\n  - **Mac**: `\"coball/resources/Reacher.app\"`\n  - **Windows** (x86): `\"coball/resources/Reacher_Windows_x86/Reacher.exe\"`\n  - **Windows** (x86_64): `\"coball/resources/Reacher_Windows_x86_64/Reacher.exe\"`\n  - **Linux** (x86): `\"coball/resources/Reacher_Linux/Reacher.x86\"`\n  - **Linux** (x86_64): `\"coball/resources/Reacher_Linux/Reacher.x86_64\"`\n  - **Linux** (x86, headless): `\"coball/resources/Reacher_Linux_NoVis/Reacher.x86\"`\n  - **Linux** (x86_64, headless): `\"coball/resources/Reacher_Linux_NoVis/Reacher.x86_64\"`\n\nIt is important that the *coball/resources* folder **_only contains the agent for your operating system_**!\n\n6. To be able to run the `Continuous_Control.ipynb` notebook, create an [IPython kernel](http://ipython.readthedocs.io/en/stable/install/kernel_install.html) for the `coball` environment.  \n```bash\npython -m ipykernel install --user --name coball --display-name \"coball\"\n```\n\n7. Before running code in the notebook, change the kernel to match the `coball` environment by using the drop-down `Kernel` menu.\n\n### Instructions\n\nThe main script in the repository is `coball.py`, which can be invoked to train the agent and store the result, as well as to replay the agent on a single episode from the stored learning results.\n\nAfter all dependencies are set up as described in the [Getting started](#getting-started) section, the script can be invoked from the root directory of the repository as e.g.\n```bash\n\u003e python coball.py --dummy\n```\nto show the untrained agent on a single episode,\n```bash\n\u003e python coball.py --epochs 250\n```\nto execute learning on 250 epochs (this may take a couple of hours),\n```bash\n\u003e python coball.py --show\n```\nto replay the trained agent on a single episode and\n```bash\n\u003e python coball.py --replay\n```\nto run the trained agent on 100 episodes and print the obtained average score (this may take a couple of minutes).\n\nWhen invoked without command line parameters, the script will execute learning on 250 epochs. For help on the possible command line options execute the script with the --help option.\n\n### Results\n\nThe parameters of the trained models can be found in the *actor_parameters.pt* and *critic_parameters.pt* files. The scores obtained during training are plotted in the *scores.png* file.\n\nThe included results achieve the target of an average score of +0.5 over 100 episodes in about than 25000 episodes during training using a noisy policy. The trained agent, however, achieves an average score around __2.5__ when executing the policy without exploration noise.\n\n![Scores](results/scores.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnumblr%2Fdrlnd-coball","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnumblr%2Fdrlnd-coball","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnumblr%2Fdrlnd-coball/lists"}