{"id":24030699,"url":"https://github.com/rvk007/multi-env-decision-making","last_synced_at":"2026-04-24T12:03:05.161Z","repository":{"id":140051517,"uuid":"429192977","full_name":"rvk007/Multi-Env-Decision-Making","owner":"rvk007","description":"Making a single agent navigate through traffic across different environments.","archived":false,"fork":false,"pushed_at":"2021-12-18T06:36:05.000Z","size":507,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-27T00:16:13.846Z","etag":null,"topics":["decision-making","deep-q-network","deep-reinforcement-learning","multi-task-learning","reinforcement-learning"],"latest_commit_sha":null,"homepage":"https://shantanuacharya.notion.site/shantanuacharya/Multi-Env-Decision-Making-d40e0ad783e64eebbb755756306e8ed9","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rvk007.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-17T20:35:01.000Z","updated_at":"2022-07-27T07:13:16.000Z","dependencies_parsed_at":null,"dependency_job_id":"f2048b22-1106-4457-9645-931acb28d120","html_url":"https://github.com/rvk007/Multi-Env-Decision-Making","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/rvk007/Multi-Env-Decision-Making","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rvk007%2FMulti-Env-Decision-Making","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rvk007%2FMulti-Env-Decision-Making/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rvk007%2FMulti-Env-Decision-Making/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rvk007%2FMulti-Env-Decision-Making/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rvk007","download_url":"https://codeload.github.com/rvk007/Multi-Env-Decision-Making/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rvk007%2FMulti-Env-Decision-Making/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32222483,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-24T10:26:35.452Z","status":"ssl_error","status_checked_at":"2026-04-24T10:25:27.643Z","response_time":64,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["decision-making","deep-q-network","deep-reinforcement-learning","multi-task-learning","reinforcement-learning"],"created_at":"2025-01-08T17:56:29.023Z","updated_at":"2026-04-24T12:03:05.129Z","avatar_url":"https://github.com/rvk007.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Multi-Env Decision Making\n\n[![Blog](https://img.shields.io/badge/Website-Blog-blue)](https://shantanuacharya.notion.site/Multi-Env-Decision-Making-d40e0ad783e64eebbb755756306e8ed9)\n\nOne of the current major areas of research in Reinforcement Learning is trying to make policies that are generalizable. Currently, in most of the tasks, a policy trained to perform well in an environment starts to struggle when even deployed in a slightly different environment.\n\nWe aim to tackle this issue by creating a **_multi-environment_ decision-making policy** that can perform well in different environment settings. We try to achieve this goal by using **Deep Q Network (DQN)** and **modifying its policy network**.\n\n## Environment\n\nWe use [Elurent’s Highway-env](https://github.com/eleurent/highway-env) project which offers a collection of environments for autonomous driving and tactical decision-making tasks. From this project, we used three environments named Highway, Merge, and Roundabout for our experiments. The goal of the agent (a.k.a the ego vehicle) is to drive at a high speed without colliding with neighboring vehicles.\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg\n    src=\"media/highway.gif\"\n    alt=\"highway\"\n  /\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \u003cp\u003eHighway\u003c/p\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg\n    src=\"media/merge.gif\"\n    alt=\"merge\"\n  /\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \u003cp\u003eMerge\u003c/p\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg\n    src=\"media/roundabout.gif\"\n    alt=\"roundabout\"\n    width=\"300\"\n  /\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \u003cp\u003eRoundabout\u003c/p\u003e\n\u003c/div\u003e\n\n## Policy\n\nWe chose **Deep Q Network** (DQN) to solve this task. The policy network is modified to use an encoder-decoder based Multi-Layer Feed Forward Policy Network (MlpPolicy). The encoder is shared by all the environments and hence learns features that are common to all environments. On the other hand, each environment has a separate decoder that learns the environment-specific features.\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg\n    src=\"media/policy_network.png\"\n    alt=\"policy network\"\n  /\u003e\n\u003c/div\u003e\n\nTo train the multi-env policy, the observations are rolled out randomly from all three environments. This enables the policy to learn all environments at once and ensure the presence of a variety of observations in the experience replay buffer.\n\n## Setup Instructions\n\nTo run the scripts, first install the necessary packages by running the command\n\n`$ pip install -r requirements.txt`\n\n## Usage\n\nTo train the policy\n\n`python run.py --config \u003cpath to config yaml\u003e`\n\nTo evaluate a policy (without rendering videos in real-time)\n\n`python run.py --config \u003cpath to config yaml\u003e -m test -p \u003cpath to policy weight\u003e`\n\nTo evaluate a policy (with video rendering in real-time)\n\n`python run.py --config \u003cpath to config yaml\u003e -m test -p \u003cpath to policy weight\u003e --render_video`\n\n## Contact/Getting Help\n\nTo know more about the experiments we conducted and their results, please refer to the [blog](https://shantanuacharya.notion.site/Multi-Env-Decision-Making-d40e0ad783e64eebbb755756306e8ed9). If you still have questions, feel free to raise an issue.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frvk007%2Fmulti-env-decision-making","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frvk007%2Fmulti-env-decision-making","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frvk007%2Fmulti-env-decision-making/lists"}