{"id":19526609,"url":"https://github.com/zju-fast-lab/radar-diffusion","last_synced_at":"2025-05-09T01:21:19.861Z","repository":{"id":226415995,"uuid":"768624037","full_name":"ZJU-FAST-Lab/Radar-Diffusion","owner":"ZJU-FAST-Lab","description":null,"archived":false,"fork":false,"pushed_at":"2025-04-16T14:16:34.000Z","size":1150,"stargazers_count":108,"open_issues_count":0,"forks_count":5,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-04-16T21:25:37.908Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ZJU-FAST-Lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-07T12:36:04.000Z","updated_at":"2025-04-16T14:16:38.000Z","dependencies_parsed_at":"2024-08-29T07:53:16.742Z","dependency_job_id":"b2d9ea6a-50cb-4cf9-962f-0a218fcb0526","html_url":"https://github.com/ZJU-FAST-Lab/Radar-Diffusion","commit_stats":null,"previous_names":["zju-fast-lab/radar-diffusion"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZJU-FAST-Lab%2FRadar-Diffusion","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZJU-FAST-Lab%2FRadar-Diffusion/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZJU-FAST-Lab%2FRadar-Diffusion/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZJU-FAST-Lab%2FRadar-Diffusion/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ZJU-FAST-Lab","download_url":"https://codeload.github.com/ZJU-FAST-Lab/Radar-Diffusion/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253172058,"owners_count":21865447,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T01:10:54.305Z","updated_at":"2025-05-09T01:21:19.846Z","avatar_url":"https://github.com/ZJU-FAST-Lab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Radar-Diffusion: Towards Dense and Accurate Radar Perception Via Efficient Cross-modal Diffusion Model\n# News\n- 25 June, 2024: Paper accepted by  _IEEE Robotics and Automation Letters (RA-L)_ !\n- 27 July, 2024: Code and pre-trained models released!\n- 29 August, 2024: Updating coloradar dataset [download link](http://zjufast.tpddns.cn:9110/share.cgi?ssid=f7f0dcf180c04488ba9219f6504bbb7b).\n- 18 October, 2024: Updating checkpoint [download link](http://zjufast.tpddns.cn:9110/share.cgi?ssid=a54ee5a706754b1c9cfb0e4a752180fc) in case that you fail to download the checkpoints uploaded to this git repo.  \n- 8 March, 2025: Updating evaluation scripts.\n- 16 April, 2025: Updating google drive link for downloading radar diffusion checkpoints [download link](https://drive.google.com/drive/folders/1xRnR3ED-hOHMdpfS8XIQYtJirp7ZizTE?usp=drive_link).\n  \n# TODO\n- [x] Release training and testing code for Radar-Diffusion.\n- [x] Release pre-trained models in **diffusion_consistency_radar/checkpoint**.\n- [x] Release user guide. \n- [x] Release data pre-processing code.\n- [x] Release performance evaluation code.\n\n# Introduction\n\nThis repository contains the source code and pre-trained models of **Radar-Diffusion** described in our paper \"Towards Dense and Accurate Radar Perception Via Efficient Cross-modal Diffusion Model.\" accepted by  _IEEE Robotics and Automation Letters (RA-L)_, 2024.\n\n__Authors__: [Ruibin Zhang](https://github.com/RoboticsZhang)\u003csup\u003e\\*\u003c/sup\u003e, [Donglai Xue](https://github.com/dungloi)\u003csup\u003e\\*\u003c/sup\u003e, [Yuhan Wang](https://github.com/johannwyh), [Ruixu Geng](https://github.com/ruixv), and [Fei Gao](http://zju-fast.com/fei-gao/) ( \u003csup\u003e\\*\u003c/sup\u003e equal contributors )\n\n__Paper__: [arXiv](https://arxiv.org/abs/2403.08460), [IEEE](https://ieeexplore.ieee.org/document/10592769)\n\n__Supplementary Video__: [YouTube](https://www.youtube.com/watch?v=Q3S-9w3dGV4\u0026t=13s), [Bilibili](https://www.bilibili.com/video/BV1eK421b76M/?spm_id_from=333.337.search-card.all.click).\n\n\u003ca href=\"https://www.youtube.com/watch?v=Q3S-9w3dGV4\u0026t=13s\" target=\"blank\"\u003e\n  \u003cp align=\"center\"\u003e\n    \u003cimg src=\"misc/topgraph.png\" width=\"1000\"/\u003e\n  \u003c/p\u003e\n\u003c/a\u003e\n\n__Abstract__: Millimeter wave (mmWave) radars have attracted significant attention from both academia and industry due to their capability to operate in extreme weather conditions. However, they face challenges in terms of sparsity and noise interference, which hinder their application in the field of\nmicro aerial vehicle (MAV) autonomous navigation. To this end, this paper proposes a novel approach to dense and accurate mmWave radar point cloud construction via cross-modal learning. Specifically, we introduce diffusion models, which possess state-of-the-art performance in generative modeling, to\npredict LiDAR-like point clouds from paired raw radar data. We also incorporate the most recent diffusion model inference accelerating techniques to ensure that the proposed method can be implemented on MAVs with limited computing resources. We validate the proposed method through extensive benchmark comparisons and real-world experiments, demonstrating its superior performance and generalization ability..\n\n\n# User Guide\n\n## Quick Start\n```sh\ngit clone https://github.com/ZJU-FAST-Lab/Radar-Diffusion.git\ncd diffusion_consistency_radar\npip install -e .\nsh launch/inference_cd_example_batch.sh\n```\nIn case of network issues, you can manually download the checkpoints in **diffusion_consistency_radar/checkpoint**.\n\nThe above script runs consistency inference in **only 1 step** using the pre-trained checkpoint. After that, you can find the predicted results and Ground-Truth LiDAR bev point clouds in **diffusion_consistency_radar/inference_results**.\n\n## Dataset Pre-processing\n1. First, download the [Coloradar dataset](https://arpg.github.io/coloradar/) (kitti format).\n   In case of network issues, we share a [download link](http://zjufast.tpddns.cn:9110/share.cgi?ssid=f7f0dcf180c04488ba9219f6504bbb7b) here. \n2. Unzip all the subsequences in a folder, then run: \n```\npython Coloradar_pre_processing/generate_coloradar_timestamp_index.py\n```\n3. Download [patchwork++](https://github.com/url-kaist/patchwork-plusplus.git) to **Coloradar_pre_processing/patchwork-plusplus**. Then install patchwork++ by running:\n```\ncd Coloradar_pre_processing/patchwork-plusplus\nmake pyinstall\n``` \n4. Generate pre-processed dataset by running:\n```\npython Coloradar_pre_processing/dataset_generation_coloradar.py\n``` \n\n## Train and test Radar-Diffusion\n1. Train a regular EDM model:\n```\nsh diffusion_consistency_radar/launch/train_edm.sh \n``` \n2. Distill a CD model from the above EDM model:\n```\nsh diffusion_consistency_radar/launch/train_cd.sh \n``` \n3. Inference from an EDM model:\n```\nsh diffusion_consistency_radar/launch/inference_edm.sh\n``` \n3. Inference from a CD model in one step:\n```\nsh diffusion_consistency_radar/launch/inference_cd.sh\n``` \n4. Evaluate the results:\n\nChange *BASE_PATH* and *SCENE_NAME*  in diffusion_consistency_radar/scripts/evaluate.py, Then:\n```\ncd diffusion_consistency_radar\npython scripts/evaluate.py\n``` \nNote that due to an unexpected server crash, the original checkpoints were lost. The checkpoints provided in this repo are the ones we retrained, so the quantitative results are different from those in the paper. When benchmarking against the proposed method, you can either use the results from the paper or the results based on the checkpoints provided in this repo.\n\n# Licence\nThe source code is released under [MIT](https://en.wikipedia.org/wiki/MIT_License) license.\n\n# Acknowledgments\n1. The diffusion-consistendy model code is heavily based on [consistency_models](https://github.com/openai/consistency_models.git).\n2. The radar pre-processing code is heavily based on [azinke/coloradar](https://github.com/azinke/coloradar).\n\n# Cite\nIf you find this method and/or code useful, please consider citing\n~~~\n@article{zhang2024towards,\n  title={Towards Dense and Accurate Radar Perception Via Efficient Cross-Modal Diffusion Model},\n  author={Zhang, Ruibin and Xue, Donglai and Wang, Yuhan and Geng, Ruixu and Gao, Fei},\n  journal={arXiv preprint arXiv:2403.08460},\n  year={2024}\n}\n~~~\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzju-fast-lab%2Fradar-diffusion","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzju-fast-lab%2Fradar-diffusion","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzju-fast-lab%2Fradar-diffusion/lists"}