{"id":28427535,"url":"https://github.com/wmn7/universal-light","last_synced_at":"2025-07-08T00:14:57.757Z","repository":{"id":109233666,"uuid":"606390205","full_name":"wmn7/Universal-Light","owner":"wmn7","description":"This repository contains the code for the paper \"UniTSA: A Universal Reinforcement Learning Framework for V2X Traffic Signal Control\".","archived":false,"fork":false,"pushed_at":"2024-11-24T19:46:03.000Z","size":10695,"stargazers_count":38,"open_issues_count":0,"forks_count":5,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-07-04T03:40:57.690Z","etag":null,"topics":["intelligent-transportation-systems","reinforcement-learning","traffic-signal-control","transportation","universal-model"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wmn7.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-25T10:49:42.000Z","updated_at":"2025-07-01T11:39:54.000Z","dependencies_parsed_at":null,"dependency_job_id":"39a82262-e4c7-42c1-90ee-9581f1459ca8","html_url":"https://github.com/wmn7/Universal-Light","commit_stats":null,"previous_names":["wmn7/universal-light"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/wmn7/Universal-Light","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wmn7%2FUniversal-Light","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wmn7%2FUniversal-Light/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wmn7%2FUniversal-Light/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wmn7%2FUniversal-Light/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wmn7","download_url":"https://codeload.github.com/wmn7/Universal-Light/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wmn7%2FUniversal-Light/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264167504,"owners_count":23567075,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["intelligent-transportation-systems","reinforcement-learning","traffic-signal-control","transportation","universal-model"],"created_at":"2025-06-05T12:11:14.915Z","updated_at":"2025-07-08T00:14:57.745Z","avatar_url":"https://github.com/wmn7.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!--\n * @Author: WANG Maonan\n * @Date: 2023-03-01 17:22:53\n * @Description: Universal Light Introduction\n * @LastEditTime: 2024-11-24 20:43:44\n--\u003e\n# UniTSA: A Universal Reinforcement Learning Framework for V2X Traffic Signal Control \n\n**[Paper-1, UniTSA](https://ieeexplore.ieee.org/abstract/document/10535743/) | [Paper-2, ADLight](https://arxiv.org/abs/2210.13378) | [Blog-1](https://mp.weixin.qq.com/s/RPYLqVYRyGci-r7HCsgWVw) | [Blog-2](https://github.com/Traffic-Alpha/awesome-V2X/blob/main/Paper/2024/Wang_2024_UniTSA.md) | [Model Checkpoint Download](./save_models/)**\n\n\nThis repository contains the code for the paper \"[UniTSA: A Universal Reinforcement Learning Framework for V2X Traffic Signal Control](https://ieeexplore.ieee.org/abstract/document/10535743/)\".\n\n## 🎉 News\n\n1. Congratulations! Our research has been accepted by IEEE Transactions on Vehicular Technology. [Read the paper here](https://ieeexplore.ieee.org/abstract/document/10535743/).\n2. We have transitioned the simulation platform in the project from Aiolos to [TransSimHub](https://github.com/Traffic-Alpha/TransSimHub) (TSHub). We extend our gratitude to our colleagues at SenseTime, **@KanYuheng (阚宇衡)**, **@MaZian (马子安)**, and **@XuChengcheng (徐承成)** (listed alphabetically) for their contributions. The development of TransSimHub (TSHub) is built upon the foundation of Aiolos.\n3. The weights of the model have been uploaded. The uploaded model weights can be found in [save_models](./save_models) and can be verified using `eval_model.py`.\n\n## 🔑 Key Points\n\n- **Universal Model for Different Intersections**: UniTSA uses a junction matrix to characterize different intersections, enabling the same model to be applied to various intersection designs.\n\n\u003cdiv align=center\u003e\n   \u003cimg src=\"./assets/junction_matrix.png\" width=\"85%\" \u003e\n\u003c/div\u003e\n\u003cp align=\"center\"\u003e(a) A 3-way intersection with (b) its junction matrix with zero padding.\u003c/p\u003e\n\n- **Performance Enhancement at Unseen Intersections**: UniTSA incorporates traffic state augmentation techniques that emphasize the relative positioning of vehicles, enhancing the model's adaptability to changing traffic conditions and unfamiliar scenarios.\n\n\u003cdiv align=center\u003e\n   \u003cimg src=\"./assets/state_augmentation_example.png\" width=\"85%\" \u003e\n\u003c/div\u003e\n\u003cp align=\"center\"\u003ellustration of three traffic state augmentation methods applied to both 4-way and 3-way intersections.\u003c/p\u003e\n\n\n- **Improved Results at Key Intersections**: UniTSA integrates the Low-Rank Adaptation (LoRA) method, allowing for efficient model customization to specific intersections with minimal additional training.\n\n\u003cdiv align=center\u003e\n   \u003cimg src=\"./assets/overall_framework.png\" width=\"85%\" \u003e\n\u003c/div\u003e\n\u003cp align=\"center\"\u003eThe overall framework of UniTSA, including: 1. RL Training and 2. Fine-tuning.\u003c/p\u003e\n\n## 📥 Installation\n\nBefore using, make sure [TSHub](https://github.com/Traffic-Alpha/TransSimHub/tree/main) is installed.\n\n```shell\ngit clone https://github.com/Traffic-Alpha/TransSimHub.git\ncd TransSimHub\npip install -e \".[rl]\"\n```\n\n## 🏃‍♂️ Training\n\nAfter installation, run `train_UniTSA.py` to train the model.\n\n```shell\npython train_UniTSA.py\n```\n\nThe trained reward curve is roughly as follows:\n\n\u003cdiv align=center\u003e\n   \u003cimg src=\"./assets/reward.png\" width=\"100%\" \u003e\n\u003c/div\u003e\n\nNote: There are large fluctuations here because the rewards vary greatly in different environments. You can rollout in advance to get the mean and variance for normalization.\n\nThe trained model will be stored in the [save_models](./save_models) directory. This model can be applied to intersections of varying structures. **For improved results under diverse traffic flow patterns, we recommend generating a wider variety of routes for training.** Please note that currently, only a subset of `rou` files have been uploaded.\n\n## 🧪 Testing\n\nTesting can be done later. If you want to test directly on the road network in this article, you can run:\n\n```shell\npython eval_model.py\n```\n\nIf you want to replace it with your own road network, you can put the road network in [sumo_datasets](./sumo_datasets/), and then modify [EVAL_CONFIG](./sumo_datasets/EVAL_CONFIG.py) to modify the corresponding road network. \n\nFor example, you can modify the `EVAL_CONFIG` as follows:\n\n```python\nEVAL_SUMO_CONFIG = dict(\n    # Four-way intersection, lane count (3,3,3,3)\n    test_four_34=dict(\n        tls_id = 'J1',\n        sumocfg = 'test_four_34.sumocfg',\n        nets = ['4phases.net.xml'],\n        routes = ['0.rou.xml', '1.rou.xml',],\n        start_time = 0,\n        edges = ['E0', '-E1', '-E3'],\n        connections = {\n            'WE-EW':['E0 E1', '-E1 -E0'],\n            'NS-SN':['-E3 E2', '-E2 E3']\n        }\n    ),\n)\n```\n\nHere:\n\n- `test_four_34`: Modify this to the name of your new folder.\n- `sumocfg`: This is the path to the sumocfg file, which needs to be in the `env` directory.\n- `nets`: This is the name of the SUMO net file. There can be multiple files.\n- `routes`: This is the file name of SUMO routes, which needs to be in the `routes` directory. There can be multiple files.\n\n## 📚 Citation\n\nIf you find this work useful, please cite our papers:\n\n```bibtex\n@article{wang2024unitsa,\n  author={Wang, Maonan and Xiong, Xi and Kan, Yuheng and Xu, Chengcheng and Pun, Man-On},\n  journal={IEEE Transactions on Vehicular Technology}, \n  title={{UniTSA}: A Universal Reinforcement Learning Framework for V2X Traffic Signal Control}, \n  year={2024},\n  volume={73},\n  number={10},\n  pages={14354-14369},\n  doi={10.1109/TVT.2024.3403879}\n}\n```\n\nThe paper below presents the preliminary concept of `UniTSA`, specifically introducing the `movement shuffle` concept:\n\n```bibtex\n@inproceedings{wang2022adlight,\n  title={{ADLight}: A Universal Approach of Traffic Signal Control with Augmented Data Using Reinforcement Learning},\n  author={Wang, Maonan and Xu, Yutong and Xiong, Xi and Kan, Yuheng and Xu, Chengcheng and Pun, Man-On},\n  booktitle={2023 Transportation Research Board Annual Meeting (102nd TRB)},\n  year={2023},\n}\n```\n\n\n## 📫 Contact\n\nIf you have any questions, please open an issue in this repository. We will respond as soon as possible.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwmn7%2Funiversal-light","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwmn7%2Funiversal-light","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwmn7%2Funiversal-light/lists"}