{"id":15480997,"url":"https://github.com/happinesslz/LION","last_synced_at":"2025-10-12T01:31:03.460Z","repository":{"id":250199573,"uuid":"833109213","full_name":"happinesslz/LION","owner":"happinesslz","description":"[NeurIPS 2024] Official code of ”LION: Linear Group RNN for 3D Object Detection in Point Clouds“","archived":false,"fork":false,"pushed_at":"2025-06-17T09:26:17.000Z","size":1721,"stargazers_count":182,"open_issues_count":9,"forks_count":15,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-17T10:28:41.370Z","etag":null,"topics":["3d-object-detection","linear-attention","linear-rnn"],"latest_commit_sha":null,"homepage":"https://happinesslz.github.io/projects/LION/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/happinesslz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-24T11:21:50.000Z","updated_at":"2025-06-17T09:26:21.000Z","dependencies_parsed_at":"2025-01-29T07:32:19.625Z","dependency_job_id":"633d979e-464a-4773-98ab-9e22aadbbf38","html_url":"https://github.com/happinesslz/LION","commit_stats":null,"previous_names":["happinesslz/lion"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/happinesslz/LION","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/happinesslz%2FLION","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/happinesslz%2FLION/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/happinesslz%2FLION/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/happinesslz%2FLION/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/happinesslz","download_url":"https://codeload.github.com/happinesslz/LION/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/happinesslz%2FLION/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279009769,"owners_count":26084648,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-object-detection","linear-attention","linear-rnn"],"created_at":"2024-10-02T05:01:00.018Z","updated_at":"2025-10-12T01:31:03.448Z","avatar_url":"https://github.com/happinesslz.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/lion-linear-group-rnn-for-3d-object-detection/3d-object-detection-on-waymo-open-dataset)](https://paperswithcode.com/sota/3d-object-detection-on-waymo-open-dataset?p=lion-linear-group-rnn-for-3d-object-detection)\r\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/lion-linear-group-rnn-for-3d-object-detection/3d-object-detection-on-nuscenes-lidar-only)](https://paperswithcode.com/sota/3d-object-detection-on-nuscenes-lidar-only?p=lion-linear-group-rnn-for-3d-object-detection)\r\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/lion-linear-group-rnn-for-3d-object-detection/3d-object-detection-on-argoverse2)](https://paperswithcode.com/sota/3d-object-detection-on-argoverse2?p=lion-linear-group-rnn-for-3d-object-detection)\r\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/lion-linear-group-rnn-for-3d-object-detection/3d-object-detection-on-once)](https://paperswithcode.com/sota/3d-object-detection-on-once?p=lion-linear-group-rnn-for-3d-object-detection)\r\n\r\n\u003cdiv align=\"center\"\u003e\r\n\r\n### \u003cimg src=\"./assets/lion.jpg\" alt=\"Image 2\" width=\"4%\" style=\"margin: 0 auto;\" \u003e [LION: Linear Group RNN for 3D Object Detection in Point Clouds](https://arxiv.org/abs/2407.18232)\r\n\r\n[Zhe Liu](https://happinesslz.github.io) \u003csup\u003e1,* \u003c/sup\u003e,\r\n[Jinghua Hou](https://github.com/AlmoonYsl) \u003csup\u003e1,* \u003c/sup\u003e,\r\n[Xinyu Wang](https://github.com/deepinact) \u003csup\u003e1,* \u003c/sup\u003e,\r\n[Xiaoqing Ye](https://shuluoshu.github.io)  \u003csup\u003e3\u003c/sup\u003e,\r\n[Jingdong Wang](https://jingdongwang2017.github.io) \u003csup\u003e3\u003c/sup\u003e,\r\n[Hengshuang Zhao](https://hszhao.github.io) \u003csup\u003e2\u003c/sup\u003e,\r\n[Xiang Bai](https://xbai.vlrlab.net) \u003csup\u003e1,✉\u003c/sup\u003e\r\n\u003cbr\u003e\r\n\u003csup\u003e1\u003c/sup\u003e Huazhong University of Science and Technology,\r\n\u003csup\u003e2\u003c/sup\u003e The University of Hong Kong,\r\n\u003csup\u003e3\u003c/sup\u003e Baidu Inc.\r\n\u003cbr\u003e\r\n\\* Equal contribution, ✉ Corresponding author.\r\n\u003cbr\u003e\r\n\r\n[**Project Page**](https://happinesslz.github.io/projects/LION) | [**NeurIPS 2024**](https://arxiv.org/abs/2407.18232)\r\n\r\n\u003cimg src=\"./assets/all.jpg\" alt=\"Image 2\" width=\"60%\" style=\"margin: 0 auto;\" \u003e\r\n\r\n\u003c/div\u003e\r\n\r\n## 🔥 Highlights\r\n\r\n* **Strong performance**. LION achieves state-of-the-art performance on Waymo, nuScenes, Argoverse V2, and ONCE datasets. 💪\r\n\r\n* **Strong generalization**. LION can support almost all linear RNN operators including Mamba, RWKV, RetNet, xLSTM, and TTT. Anyone is welcome to verify more linear RNN operators. 😀\r\n\r\n* **More friendly**. LION can train all models on less 24G GPU memory (i.e., RTX 3090, RTX4090, V100 and A100 are enough to train our LION). 😀\r\n\r\n## News\r\n* **2025.06.16**: Our new work about Transformer-Mamba architecture [HybridTM](https://github.com/deepinact/HybridTM) have been accepted by IROS 2025 as **Oral** presentation. 🎉\r\n* **2024.09.26**: LION has been accepted by NeurIPS 2024. 🎉\r\n* **2024.07.25**: LION paper released. 🔥\r\n* **2024.07.02**: Our new works [OPEN](https://github.com/AlmoonYsl/OPEN) and [SEED](https://github.com/happinesslz/SEED) have been accepted by ECCV 2024. 🎉\r\n\r\n## Results\r\n* **Waymo Val Set~(100%)**\r\n\r\n| Model        | mAP/mAPH_L1 | mAP/mAPH_L2 |  Vec_L1   |  Vec_L2   |  Ped_L1   |  Ped_L2   |  Cyc_L1   |  Cyc_L2   | Config |\r\n|--------------|:-----------:|:-----------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:------:|\r\n| LION-RetNet  |  80.9/78.8  |  74.6/72.7  | 79.0/78.5 | 70.6/70.2 | 84.6/80.0 | 77.2/72.8 | 79.0/78.0 | 76.1/75.1 |[config](tools/cfgs/lion_models/lion_retnet_waymo_8x_1f_1x_one_stride_64dim.yaml)|\r\n| LION-RWKV    |  81.0/79.0  |  74.7/72.8  | 79.7/79.3 | 71.3/71.0 | 84.6/80.0 | 77.1/72.7 | 78.7/77.7 | 75.8/74.8 |[config](tools/cfgs/lion_models/lion_rwkv_waymo_8x_1f_1x_one_stride_64dim.yaml)|\r\n| LION-Mamba   |  81.4/79.4  |  75.1/73.2  | 79.5/79.1 | 71.1/70.7 | 84.9/80.4 | 77.5/73.2 | 79.7/78.7 | 76.7/75.8 |[config](tools/cfgs/lion_models/lion_mamba_waymo_8x_1f_1x_one_stride_64dim.yaml)|\r\n| LION-Mamba-L |  82.1/80.1  |  75.9/74.0  | 80.3/79.9 | 72.0/71.6 | 85.8/81.4 | 78.5/74.3 | 80.1/79.0 | 77.2/76.2 |[config](tools/cfgs/lion_models/lion_mamba_waymo_8x_1f_1x_one_stride_128dim.yaml)|\r\n\r\nNote: You could reduce the training epochs from 24 to 12~(the performance gap is within 1 mAP/mAPH) or reduce the 100% training to 20% training sets.\r\n\r\n* **nuScenes**\r\n\r\n|    Model    | Split | Epoch | CBGS| NDS  | mAP  | Config | Download (Baidu Pan) |                                                           Download (Google Drive)                                                           |\r\n|:-----------:|:-----:|:----:|:----:|:----:|:----:|:------:|:--------------------:|:-------------------------------------------------------------------------------------------------------------------------------------------:|\r\n| LION-RetNet |  Val  | 36 | False | 71.9 | 67.3 |[config](tools/cfgs/lion_models/lion_retnet_nusc_8x_1f_1x_one_stride_128dim.yaml)|       [nus_retnet.pth]( https://pan.baidu.com/s/11m4Tka9VRoLM7dO_-CQfng)  (ksmp)       |          [nus_retnet.pth](https://drive.google.com/file/d/1LDM6Iq91muZhzFMMa0WqKtjzJwYVadF6/view?usp=sharing)           |\r\n|  LION-RWKV  |  Val  | 36 | False | 71.7 | 66.8 |[config](tools/cfgs/lion_models/lion_rwkv_nusc_8x_1f_1x_one_stride_128dim.yaml)|                      |                                                                                                                                             |\r\n| LION-Mamba  |  Val  | 36 | False | 72.1 | 68.0 |[config](tools/cfgs/lion_models/lion_mamba_nusc_8x_1f_1x_one_stride_128dim.yaml)|       [nus_mamba.pth](https://pan.baidu.com/s/1fkGHULD3XVwN6qgjqahWIA)  (2tvc)       |                                                    [nus_mamba.pth](https://drive.google.com/file/d/1bBzZcZUukt7B5aD3bmKlM2mT7SL8q3Fv/view?usp=sharing)                                                    |\r\n| LION-Mamba  |  Val  | 48 | False | 72.3 | 68.2 |[config](tools/cfgs/lion_models/lion_mamba_nusc_8x_1f_1x_one_stride_128dim_ep48.yaml)|                      |                                                                                                                                             |\r\n| LION-Mamba  | Test  | 36 | False | 73.9 | 69.8 |        |                      |                                                                                                                                             |\r\n\r\nNote: Our model on nuScenes does not use CBGS for training more time and without any test-time augmentation or model ensembling!\r\nFor obtaining more stable and better performance, you could try to train more time~(e.g., 48 epochs)\r\n\r\n* **Argoverse V2 Val Set**\r\n\r\n|    Model    | mAP  | Config |                                       Download  (Baidu Pan)                                        | Download  (Google Drive) |\r\n|:-----------:|:----:|:------:|:--------------------------------------------------------------------------------------------------:|:------------------------:|\r\n| LION-RetNet | 40.7 |[config](tools/cfgs/lion_models/lion_retnet_1f_1x_argo_128dim_sparse_v2.yaml)|            [argov2_retnet.pth]( https://pan.baidu.com/s/1Te8HhIVJpxarFLvr1GMtWQ) (yghm)            |           [argov2_retnet.pth](https://drive.google.com/file/d/17ZTdThUvK2tnsz0ZhVUrCuJf3bvxQDTl/view?usp=sharing)           |\r\n|  LION-RWKV  | 41.1 |[config](tools/cfgs/lion_models/lion_rwkv_1f_1x_argo_128dim_sparse_v2.yaml)|            [argov2_rwkv.pth](https://pan.baidu.com/s/14r1OKM5Eh4JMURIvZ5i2hg)    (cr4e)            |           [argov2_rwkv.pth](https://drive.google.com/file/d/15zX5OFvd0WkdE8jsqRXMJ9JdM16gfGkM/view?usp=sharing)           |\r\n| LION-Mamba  | 41.5 |[config](tools/cfgs/lion_models/lion_mamba_1f_1x_argo_128dim_sparse_v2.yaml)|  [argov2_mamba.pth](https://pan.baidu.com/s/1IPMhAgW4Oq14-pkfEPkrSA)   (k63i)  |           [argov2_mamba.pth](https://drive.google.com/file/d/1NfgKZZZJDodhWnp-Yi3Bi-yXHWM9mW1U/view?usp=sharing)           |\r\n\r\n* **ONCE Val Set**\r\n\r\n|    Model    | Vehicle | Pedestrian | Cyclist | mAP  |  Config   | Download |\r\n|:-----------:|:-------:|:----------:|:-------:|:----:|:---------:|:--------:|\r\n| LION-RetNet |  78.1   |    52.4    |  68.3   | 66.3 |[config](tools/cfgs/once_models/centerpoint_with_lion_with_128dim_retnet.yaml)|          |\r\n|  LION-RWKV  |  78.3   |    50.6    |  68.4   | 65.8 |[config](tools/cfgs/once_models/centerpoint_with_lion_with_128dim_rwkv.yaml)|          |\r\n| LION-Mamba  |  78.2   |    53.2    |  68.5   | 66.6 |[config](tools/cfgs/once_models/centerpoint_with_lion_with_128dim_mamba.yaml)|          |\r\n\r\n\r\n## Quick Validation\r\n* We provide some examples of LION models on KITTI dataset for quick validation of any Linear RNN operators.\r\n* Here, we provide the results of moderate difficulty for LION with RetNet, RWKV, Mamba, xLSTM, and TTT.\r\n* Anyone is welcome to verify more linear RNN operators. 😀\r\n\r\n\r\n|    Model    | Car  | Pedestrian | Cyclist |  Config  | Download |\r\n|:-----------:|:----:|:----------:|:-------:|:--------:|:--------:|\r\n|  LION-TTT   | 78.0 |    58.6    |  69.6   |[config](tools/cfgs/kitti_models/second_with_lion_TTT_64dim.yaml)|          |\r\n| LION-xLSTM  | 77.9 |    59.3    |  67.4   |[config](tools/cfgs/kitti_models/second_with_lion_xLSTM_64dim.yaml)|        |\r\n| LION-RetNet | 77.9 |    60.2    |  69.6   |[config](tools/cfgs/kitti_models/second_with_lion_retnet_64dim.yaml)|       |\r\n| LION-Mamba  | 78.3 |    60.2    |  68.6   |[config](tools/cfgs/kitti_models/second_with_lion_mamba_64dim.yaml)|        |\r\n|  LION-RWKV  | 78.3 |    62.2    |  71.2   |[config](tools/cfgs/kitti_models/second_with_lion_rwkv_64dim.yaml)|         |\r\n\r\n\r\n## Installation\r\nPlease refer to [INSTALL.md](docs/INSTALL.md) for the installation of LION codebase.\r\n\r\n\r\n## Getting Started\r\nWe provide all training\u0026evaluation scripts for training our LION, please refer to [tools/](tools/)\r\n* Train all models of LION on nuScenes\r\n```shell script\r\nbash run_train_lion_for_nus.sh\r\n```\r\n\r\n* Train all models of LION on Waymo\r\n```shell script\r\nbash run_train_lion_for_waymo.sh\r\n```\r\n\r\n* Train all models of LION on Argoverse V2\r\n```shell script\r\nbash run_train_lion_for_argov2.sh\r\n```\r\n\r\n* Train all models of LION on ONCE\r\n```shell script\r\nbash run_train_lion_for_once.sh\r\n```\r\n\r\n* Train all models of LION on KITTI\r\n```shell script\r\nbash run_train_lion_for_kitti.sh\r\n```\r\n\r\nFor more details about LION, please refer to [GETTING_STARTED.md](docs/GETTING_STARTED.md) to learn more usage about LION.\r\n\r\n## TODO\r\n- [x] Release the paper.\r\n- [x] Release the code of LION on the Waymo.\r\n- [x] Release the code of LION on the nuScenes.\r\n- [x] Release the code of LION on the Argoverse V2.\r\n- [x] Release the code of LION on the ONCE.\r\n- [x] Release the code of LION on the KITTI.\r\n- [x] Release some important checkpoints of LION (nuScenes and Argoverse v2).\r\n- [ ] Support more linear RNNs.\r\n\r\n## Citation\r\n```\r\n@article{liu2024lion,\r\n  title={LION: Linear Group RNN for 3D Object Detection in Point Clouds},\r\n  author={Zhe Liu, Jinghua Hou, Xingyu Wang, Xiaoqing Ye, Jingdong Wang, Hengshuang Zhao, Xiang Bai},\r\n  journal={Advances in Neural Information Processing Systems},\r\n  year={2024}\r\n  }\r\n```\r\n\r\n## Acknowledgements\r\nWe thank these great works and open-source repositories:\r\n[OpenPCDet](https://github.com/open-mmlab/OpenPCDet), [DSVT](https://github.com/Haiyang-W/DSVT), [FlatFormer](https://github.com/mit-han-lab/flatformer), [HEDNet](https://github.com/zhanggang001/HEDNet), [Mamba](https://github.com/state-spaces/mamba), [RWKV](https://github.com/BlinkDL/RWKV-LM), [Vision-RWKV](https://github.com/OpenGVLab/Vision-RWKV), [RMT](https://github.com/qhfan/RMT), [xLSTM](https://github.com/NX-AI/xlstm),  [TTT](https://github.com/test-time-training/ttt-lm-pytorch), and [flash-linear-attention](https://github.com/sustcsonglin/flash-linear-attention).\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhappinesslz%2FLION","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhappinesslz%2FLION","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhappinesslz%2FLION/lists"}