{"id":19233967,"url":"https://github.com/traffic-alpha/illm-tsc","last_synced_at":"2025-08-22T05:31:37.714Z","repository":{"id":195341829,"uuid":"692715388","full_name":"Traffic-Alpha/iLLM-TSC","owner":"Traffic-Alpha","description":"This repository contains the code for the paper“iLLM-TSC: Integration reinforcement learning and large language model for traffic signal control policy improvement”","archived":false,"fork":false,"pushed_at":"2024-11-05T09:35:37.000Z","size":24523,"stargazers_count":44,"open_issues_count":9,"forks_count":8,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-12-12T03:04:40.626Z","etag":null,"topics":["llm","reinforcement-learning","rl","tsc"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Traffic-Alpha.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-09-17T11:09:30.000Z","updated_at":"2024-12-04T16:37:15.000Z","dependencies_parsed_at":"2023-09-27T13:59:42.214Z","dependency_job_id":"c7406bb3-118c-4089-95f8-a9853012bd9f","html_url":"https://github.com/Traffic-Alpha/iLLM-TSC","commit_stats":null,"previous_names":["traffic-alpha/refinmentgpt","traffic-alpha/tsc-harla","traffic-alpha/illm-tsc"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Traffic-Alpha%2FiLLM-TSC","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Traffic-Alpha%2FiLLM-TSC/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Traffic-Alpha%2FiLLM-TSC/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Traffic-Alpha%2FiLLM-TSC/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Traffic-Alpha","download_url":"https://codeload.github.com/Traffic-Alpha/iLLM-TSC/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230561013,"owners_count":18245324,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["llm","reinforcement-learning","rl","tsc"],"created_at":"2024-11-09T16:12:27.153Z","updated_at":"2025-08-22T05:31:37.701Z","avatar_url":"https://github.com/Traffic-Alpha.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# iLLM-TSC: Integration reinforcement learning and large language model for traffic signal control policy improvement\n\n## [Paper](https://arxiv.org/abs/2407.06025) | [Simulation](https://github.com/Traffic-Alpha/TransSimHub) |\n## iLLM-TSC's powerful capabilities\n\nhttps://github.com/Traffic-Alpha/TSC-HARLA/assets/75999557/92d6ff7f-cc5b-42ba-9feb-046022e70ad9\n\n\n\n## Info\nWe propose a framework that utilizes LLM to support RL models. This framework refines RL decisions based on real-world contexts and provides reasonable actions when RL agents make erroneous decisions. \n\n\u003cdiv align=center\u003e\n\u003cimg width=\"90%\" src=\"./assets/iLLM-Framework.png\" /\u003e\n\nThe detailed structure of iLLM-TSC.\n\u003c/div\u003e\n\n\n## Typical Cases\n\n- Case1: LLM think that the action taken by the RL Agent was unreasonable and gave a reasonable explanation and recommended actions.\n\u003cdiv align=center\u003e\n\u003cimg width=\"90%\" src=\"./assets/Case1.png\" /\u003e\n\n\n\u003c/div\u003e\n\n- Case 2: LLM considers that the movement made by the RL Agent is not the movement with the highest current mean occupancy but it is reasonable, after which LLM gives an explanation and recommendation.\n\u003cdiv align=center\u003e\n\u003cimg width=\"90%\" src=\"./assets/Case2.png\" /\u003e\n\u003c/div\u003e\n\n- Case 3: An ambulance needs to pass through the intersection, but the RL Agent does not take into account that the ambulance needs to be prioritized. LLM modifies the RL Agent’s action to prioritize the ambulance to pass through the intersection.\n\u003cdiv align=center\u003e\n\u003cimg width=\"90%\" src=\"./assets/Case3.png\" /\u003e\n\u003c/div\u003e\n\n## Install\n\n### Install [TransSimHub](https://github.com/Traffic-Alpha/TransSimHub)\nThe simulation environment we used is TransSimHub, which is based on SUMO and can be used for TSC, V2X and UAM simulation. More information is available via [docs](https://transsimhub.readthedocs.io/en/latest/).\n\nYou can install TransSimHub by cloning the GitHub repository. Follow these steps:\n```bash\ngit clone https://github.com/Traffic-Alpha/TransSimHub.git\ncd TransSimHub\npip install -e .\n```\n\nAfter the installation is complete, you can use the following Python command to check if TransSimHub is installed and view its version:\n\n```bash\nimport tshub\nprint(tshub.__version__)\n```\n\n###  Install HARLA\nYou can install HARLA by cloning the GitHub repository. Follow these steps:\n```bash\ngit clone https://github.com/Traffic-Alpha/iLLM-TSC\ncd iLLM-TSC\npip install -r requirements.txt\n```\nAfter completing the above ``Install steps``, you can use this program locally. \n## Run HARLA locally\n### Train RL\nThe first thing you need to do is train a RL model. You can do it with the following code:\n```bash\ncd iLLM-TSC\npython sb3_ppo.py\n```\nThe training results are shown in the figure, and model weight has been uploaded in [models](./models/). \n\n\u003cdiv align=center\u003e\n\u003cimg width=\"70%\" src=\"./assets/train_result.png\" /\u003e\n\u003c/div\u003e\n\nThe effect of the RL model can be tested with the following code:\n```bash\npython eval_rl_agent.py\n```\n### Try RL+LLM\nBefore you can use LLM, you need to have your own KEY and fill it in the [``utils/config.yaml``](./utils/config.yaml). \n```bash\nOPENAI_PROXY: \nOPENAI_API_KEY:\n```\nThe entire framework can be used with the following code.\n```bash\npython rl_llm_tsc.py\n```\n\n\n**Evaluation Rule: To make fair evaluation and comparison among different models, make sure you use the same LLM evaluation model (we use GPT4) for all the models you want to evaluate. Using a different scoring model or API updating might lead to different results.**\n\n## License and Citation\nAll assets and code in this repository are under the [Apache 2.0 license](./LICENSE) unless specified otherwise. The language data is under [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/). Other datasets (including nuScenes) inherit their own distribution licenses. Please consider citing our project if it helps your research.\n```BibTeX\n@article{pang2024illm,\n  title={iLLM-TSC: Integration reinforcement learning and large language model for traffic signal control policy improvement},\n  author={Pang, Aoyu and Wang, Maonan and Pun, Man-On and Chen, Chung Shue and Xiong, Xi},\n  journal={arXiv preprint arXiv:2407.06025},\n  year={2024}\n}\n```\n```BibTeX\n@article{wang2024llm,\n  title={LLM-Assisted Light: Leveraging Large Language Model Capabilities for Human-Mimetic Traffic Signal Control in Complex Urban Environments},\n  author={Wang, Maonan and Pang, Aoyu and Kan, Yuheng and Pun, Man-On and Chen, Chung Shue and Huang, Bo},\n  journal={arXiv preprint arXiv:2403.08337},\n  year={2024}\n}\n```\n\n\n## More Projects\niLLM-TSC just explores the combination of RL and LLM, more work will be updated in [**TSC-LLM**](https://github.com/Traffic-Alpha/TSC-LLM), welcome to star!\n\n## Acknowledgment\n\n- **Yufei Teng**: Thanks for editing the video.\n- **Thank you to everyone who pays attention to our work. Hope our work can help you.**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftraffic-alpha%2Fillm-tsc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftraffic-alpha%2Fillm-tsc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftraffic-alpha%2Fillm-tsc/lists"}