{"id":13440715,"url":"https://github.com/OpenDriveLab/End-to-end-Autonomous-Driving","last_synced_at":"2025-03-20T10:32:06.061Z","repository":{"id":175331456,"uuid":"623482435","full_name":"OpenDriveLab/End-to-end-Autonomous-Driving","owner":"OpenDriveLab","description":"[IEEE T-PAMI 2024] All you need for End-to-end Autonomous Driving","archived":false,"fork":false,"pushed_at":"2024-12-17T15:44:07.000Z","size":5113,"stargazers_count":2878,"open_issues_count":1,"forks_count":270,"subscribers_count":67,"default_branch":"main","last_synced_at":"2025-03-19T19:08:14.850Z","etag":null,"topics":["autonomous-driving","end-to-end-autonomous-driving","policy-learning","simulation"],"latest_commit_sha":null,"homepage":"https://doi.org/10.1109/TPAMI.2024.3435937","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OpenDriveLab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["OpenDriveLab"],"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"lfx_crowdfunding":null,"custom":null}},"created_at":"2023-04-04T13:10:58.000Z","updated_at":"2025-03-19T07:47:16.000Z","dependencies_parsed_at":null,"dependency_job_id":"da944653-1cfd-40dc-ae7a-08d01f575334","html_url":"https://github.com/OpenDriveLab/End-to-end-Autonomous-Driving","commit_stats":null,"previous_names":["opendrivelab/end-to-end-autonomous-driving"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenDriveLab%2FEnd-to-end-Autonomous-Driving","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenDriveLab%2FEnd-to-end-Autonomous-Driving/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenDriveLab%2FEnd-to-end-Autonomous-Driving/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenDriveLab%2FEnd-to-end-Autonomous-Driving/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OpenDriveLab","download_url":"https://codeload.github.com/OpenDriveLab/End-to-end-Autonomous-Driving/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244595192,"owners_count":20478424,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autonomous-driving","end-to-end-autonomous-driving","policy-learning","simulation"],"created_at":"2024-07-31T03:01:25.439Z","updated_at":"2025-03-20T10:32:06.053Z","avatar_url":"https://github.com/OpenDriveLab.png","language":null,"funding_links":["https://github.com/sponsors/OpenDriveLab"],"categories":["Others"],"sub_categories":[],"readme":"\u003cdiv id=\"top\"\u003e\n\n# End-to-end Autonomous Driving\n\n\u003e **This repo is all you need for end-to-end autonomous driving research.** We present awesome talks, comprehensive paper collections, benchmarks, and challenges.\n\n\u003c!-- ![](https://img.shields.io/badge/Record-137-673ab7.svg)\n![](https://img.shields.io/badge/License-MIT-lightgrey.svg) --\u003e\n\n## Table of Contents\n\n- [At a Glance](#at-a-glance)\n- [Learning Materials for Beginners](#learning-materials-for-beginners)\n- [Workshops and Talks](#workshops-and-talks)\n- [Paper Collection](#paper-collection)\n- [Benchmarks and Datasets](#benchmarks-and-datasets)\n- [Competitions / Challenges](#competitions--challenges) \n- [Contributing](#contributing)\n- [License](#license)\n- [Citation](#citation)\n- [Contact](#contact)\n\n## At a Glance\n\nThe autonomous driving community has witnessed a rapid growth in approaches that embrace an end-to-end algorithm framework, utilizing raw sensor input to generate vehicle motion plans, instead of concentrating on individual tasks such as detection and motion prediction. In this survey, we provide a comprehensive analysis of more than 270 papers on the motivation, roadmap, methodology, challenges, and future trends in end-to-end autonomous driving. More details can be found in our survey paper.\n\n\u003e [**End-to-end Autonomous Driving: Challenges and Frontiers**](https://arxiv.org/abs/2306.16927)\n\u003e\n\u003e [Li Chen](https://scholar.google.com/citations?user=ulZxvY0AAAAJ\u0026hl=en\u0026authuser=1)\u003csup\u003e1,2\u003c/sup\u003e, [Penghao Wu](https://penghao-wu.github.io)\u003csup\u003e1\u003c/sup\u003e, [Kashyap Chitta](https://kashyap7x.github.io/)\u003csup\u003e3,4\u003c/sup\u003e, [Bernhard Jaeger](https://kait0.github.io/)\u003csup\u003e3,4\u003c/sup\u003e, [Andreas Geiger](https://www.cvlibs.net/)\u003csup\u003e3,4\u003c/sup\u003e, and [Hongyang Li](https://lihongyang.info/)\u003csup\u003e1,2\u003c/sup\u003e\n\u003e \n\u003e \u003csup\u003e1\u003c/sup\u003e OpenDriveLab, Shanghai AI Lab, \u003csup\u003e2\u003c/sup\u003e University of Hong Kong, \u003csup\u003e3\u003c/sup\u003e University of Tübingen, \u003csup\u003e4\u003c/sup\u003e Tübingen AI Center\n\u003e\n\n\u003cbr/\u003e\n\n![](assets/overview.jpg)\n\n\u003cbr/\u003e\n\n``\nIf you find some useful related materials, shoot us an email or simply open a PR!\n``\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#top\"\u003eback to top\u003c/a\u003e)\u003c/p\u003e\n\n\n## Learning Materials for Beginners\n  \n**Online Courses**\n- [Lecture: Self-Driving Cars](https://uni-tuebingen.de/en/fakultaeten/mathematisch-naturwissenschaftliche-fakultaet/fachbereiche/informatik/lehrstuehle/autonomous-vision/lectures/self-driving-cars/), Andreas Geiger, University of Tübingen, Germany\n- [Self-Driving Cars Specialization](https://www.coursera.org/specializations/self-driving-cars), University of Toronto, Coursera\n- [The Complete Self-Driving Car Course - Applied Deep Learning](https://www.udemy.com/course/applied-deep-learningtm-the-complete-self-driving-car-course/), Udemy\n- [Self-Driving Car Engineer Nanodegree Program](https://www.udacity.com/course/self-driving-car-engineer-nanodegree--nd0013), Udacity\n\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#top\"\u003eback to top\u003c/a\u003e)\u003c/p\u003e\n  \n## Workshops and Talks\n\n**Workshops (recent years)**\n- [CVPR 2024] [Foundation Models for Autonomous Systems](https://opendrivelab.com/cvpr2024/workshop/)\n- [CVPR 2024] [Tutorial: End-to-End Autonomy: A New Era of Self-Driving](https://wayve.ai/cvpr-e2ead-tutorial/)\n- [CVPR 2024] [Tutorial: Towards Building AGI in Autonomy and Robotics](https://opendrivelab.com/cvpr2024/tutorial/)\n- [CVPR 2023] [Workshop on End-to-end Autonomous Driving](https://opendrivelab.com/e2ead/cvpr23.html)\n- [CVPR 2023] [End-to-End Autonomous Driving: Perception, Prediction, Planning and Simulation](https://e2ead.github.io/2023.html)\n- [ICRA 2023] [Scalable Autonomous Driving](https://sites.google.com/view/icra2023av/home?authuser=0)\n\n**Workshops (previous years)**\n\u003cdetails\u003e\n\n  - [NeurIPS 2022] [Machine Learning for Autonomous Driving](https://ml4ad.github.io/)\n  - [IROS 2022] [Behavior-driven Autonomous Driving in Unstructured Environments](https://gamma.umd.edu/workshops/badue22/)\n  - [ICRA 2022] [Fresh Perspectives on the Future of Autonomous Driving Workshop](https://www.self-driving-cars.org/)\n  - [NeurIPS 2021] [Machine Learning for Autonomous Driving](https://ml4ad.github.io/2021/)\n  - [NeurIPS 2020] [Machine Learning for Autonomous Driving](https://ml4ad.github.io/2020/)\n  - [CVPR 2020] [Workshop on Scalability in Autonomous Driving](https://sites.google.com/view/cvpr20-scalability)\n\n\u003c/details\u003e\n\u003c/br\u003e\n\n**Talks**\n\u003cdetails\u003e\n  \u003csummary\u003eRelevant talks from other workshops\u003c/summary\u003e\n  \n  - [Common Misconceptions in Autonomous Driving](https://www.youtube.com/watch?v=x_42Fji1Z2M) - Andreas Geiger, Workshop on Autonomous Driving, CVPR 2023\n  - [Learning Robust Policies for Self-Driving](https://www.youtube.com/watch?v=rm-1sPQV4zg) - Andreas Geiger, AVVision: Autonomous Vehicle Vision Workshop, ECCV 2022\n  - [Autonomous Driving: The Way Forward](https://www.youtube.com/watch?v=XmtTjqimW3g) -  Vladlen Koltun, Workshop on AI for Autonomous Driving, ICML 2020\n  - [Feedback in Imitation Learning: Confusion on Causality and Covariate Shift](https://www.youtube.com/watch?v=4VAwdCIBTG8) -  Sanjiban Choudhury and Arun Venkatraman, Workshop on AI for Autonomous Driving, ICML 2020\n  \n\u003c/details\u003e\n  \n\u003cp align=\"right\"\u003e(\u003ca href=\"#top\"\u003eback to top\u003c/a\u003e)\u003c/p\u003e\n\n## Paper Collection\nWe list key challenges from a wide span of candidate concerns, as well as trending methodologies. Please refer to [this page](./papers.md) for the full list, and the [survey paper](https://arxiv.org/abs/2306.16927) for detailed discussions.\n\n- [Survey](./papers.md#survey)\n- [Language / VLM for Driving](./papers.md#language--vlm-for-driving)\n  - [Review for VLM in Driving](./papers.md#review-for-vlm-in-driving)\n  - [Papers for VLM in Driving](./papers.md#papers-for-vlm-in-driving)\n- [World Model \u0026 Model-based RL](./papers.md#world-model--model-based-rl)\n- [Multi-sensor Fusion](./papers.md#multi-sensor-fusion)\n- [Multi-task Learning](./papers.md#multi-task-learning)\n- [Interpretability](./papers.md#interpretability)\n  - [Review for Interpretability](./papers.md#review-for-interpretability)\n  - [Attention Visualization](./papers.md#attention-visualization)\n  - [Interpretable Tasks](./papers.md#interpretable-tasks)\n  - [Cost Learning](./papers.md#cost-learning)\n  - [Linguistic Explainability](./papers.md#linguistic-explainability)\n  - [Uncertainty Modeling](./papers.md#uncertainty-modeling)\n  - [Counterfactual Explanations and Causal Inference](./papers.md#counterfactual-explanations-and-causal-inference)\n- [Visual Abstraction / Representation Learning](./papers.md#visual-abstraction--representation-learning)\n- [Policy Distillation](./papers.md#policy-distillation)\n- [Causal Confusion](./papers.md#causal-confusion)\n- [Robustness](./papers.md#robustness)\n  - [Long-tailed Distribution](./papers.md#long-tailed-distribution)\n  - [Covariate Shift](./papers.md#covariate-shift)\n  - [Domain Adaptation](./papers.md#domain-adaptation)\n- [Affordance Learning](./papers.md#affordance-learning)\n- [BEV](./papers.md#bev)\n- [Transformer](./papers.md#transformer)\n- [V2V Cooperative](./papers.md#v2v-cooperative)\n- [Distributed RL](./papers.md#distributed-rl)\n- [Data-driven Simulation](./papers.md#data-driven-simulation)\n  - [Parameter Initialization](./papers.md#parameter-initialization)\n  - [Traffic Simulation](./papers.md#traffic-simulation)\n  - [Sensor Simulation](./papers.md#sensor-simulation)\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#top\"\u003eback to top\u003c/a\u003e)\u003c/p\u003e\n\n## Benchmarks and Datasets\n\n\u003e Real-world deployment is the final benchmark for autonomous driving.\n\u003e However, testing in the real world is expensive. For academic benchmarking, we recommend you read this write-up from Jaeger et al. 2024: [Common mistakes in benchmarking](https://github.com/autonomousvision/carla_garage/blob/leaderboard_2/docs/common_mistakes_in_benchmarking_ad.md).\n\n**Closed-loop**\n- [CARLA](https://leaderboard.carla.org/leaderboard/)\n  - [Leaderboard 1.0](https://leaderboard.carla.org/get_started_v1/)\n  - [Leaderboard 2.0](https://leaderboard.carla.org/get_started/)\n- [nuPlan](https://www.nuscenes.org/nuplan)\n  - [Leaderboard](https://eval.ai/web/challenges/challenge-page/1856/overview) (inactive after the CVPR 2023 challege)\n  - [NAVSIM](https://github.com/autonomousvision/navsim)\n\n\u003cdetails\u003e\n  \u003csummary\u003eOpen-loop\u003c/summary\u003e\n  \n- [nuScenes](https://www.nuscenes.org/nuscenes)\n- [nuPlan](https://www.nuscenes.org/nuplan)\n- [Argoverse](https://www.argoverse.org/av2.html)\n- [Waymo Open Dataset](https://waymo.com/open/)\n  \n\u003c/details\u003e\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#top\"\u003eback to top\u003c/a\u003e)\u003c/p\u003e\n\n## Competitions / Challenges\n\n- [End-to-End Driving at Scale](https://opendrivelab.com/challenge2024/#end_to_end_driving_at_scale), Foundation Models for Autonomous Systems, CVPR 2024\n- [CARLA Autonomous Driving Challenge](https://opendrivelab.com/challenge2024/#carla), Foundation Models for Autonomous Systems, CVPR 2024\n- [nuPlan planning](https://opendrivelab.com/AD23Challenge.html#nuplan_planning), Workshop on End-to-end Autonomous Driving, CVPR 2023\n- [CARLA Autonomous Driving Challenge 2022](https://ml4ad.github.io/#challenge), Machine Learning for Autonomous Driving, NeurIPS 2022\n- [CARLA Autonomous Driving Challenge 2021](https://ml4ad.github.io/2021/#challenge), Machine Learning for Autonomous Driving, NeurIPS 2021\n- [CARLA Autonomous Driving Challenge 2020](https://ml4ad.github.io/2020/#challenge), Machine Learning for Autonomous Driving, NeurIPS 2020\n- [Learn-to-Race Autonomous Racing Virtual Challenge](https://www.aicrowd.com/challenges/learn-to-race-autonomous-racing-virtual-challenge), 2022\n- [INDY Autonomous Challenge](https://www.indyautonomouschallenge.com/)\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#top\"\u003eback to top\u003c/a\u003e)\u003c/p\u003e\n  \n## Contributing\nThank you for all your contributions. Please make sure to read the [contributing guide](./CONTRIBUTING.md) before you make a pull request.\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#top\"\u003eback to top\u003c/a\u003e)\u003c/p\u003e\n\n## License\nEnd-to-end Autonomous Driving is released under the [MIT license](./LICENSE).\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#top\"\u003eback to top\u003c/a\u003e)\u003c/p\u003e\n\n## Citation\nIf you find this project useful in your research, please consider citing:\n```BibTeX\n@article{chen2023e2esurvey,\n  title={End-to-end Autonomous Driving: Challenges and Frontiers},\n  author={Chen, Li and Wu, Penghao and Chitta, Kashyap and Jaeger, Bernhard and Geiger, Andreas and Li, Hongyang},\n  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},\n  year={2024}\n}\n```\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#top\"\u003eback to top\u003c/a\u003e)\u003c/p\u003e\n\n## Contact\nPrimary contact: `hy@opendrivelab.com`. You can also contact: `lichen@opendrivelab.com`.\n\nJoin [OpenDriveLab Slack](https://opendrivelab.slack.com/join/shared_invite/zt-2ft3dfjoz-6XErfBts4s_8Fen88wO4Jg#/shared-invite/email) to chat with the commuty! Slack channel: `#e2ead`.\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#top\"\u003eback to top\u003c/a\u003e)\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FOpenDriveLab%2FEnd-to-end-Autonomous-Driving","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FOpenDriveLab%2FEnd-to-end-Autonomous-Driving","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FOpenDriveLab%2FEnd-to-end-Autonomous-Driving/lists"}