awesome-knowledge-driven-AD
A curated list of awesome knowledge-driven autonomous driving (continually updated)
https://github.com/PJLab-ADG/awesome-knowledge-driven-AD
Last synced: 2 days ago
JSON representation
-
:books: Papers
-
Environment
- Generalized Predictive Model for Autonomous Driving
- DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation
- TCLC-GS: Tightly Coupled LiDAR-Camera Gaussian Splatting for Surrounding Autonomous Driving Scenes
- Urban Architect: Steerable 3D Urban Scene Generation with Layout Prior
- UniSim: A Neural Closed-Loop Sensor Simulator
- NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles - devkit)]
- DrivingDiffusion: Layout-Guided multi-view driving scene video generation with latent diffusion model
- OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving
- ADriver-I: A General World Model for Autonomous Driving
- Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving - wm.github.io/), [Github](https://github.com/BraveGroup/Drive-WM)]
- WoVoGen: World Volume-aware Diffusion for Controllable Multi-camera Driving Scene Generation - zvg/WoVoGen)]
- DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving
- MagicDrive: Street View Generation with Diverse 3D Geometry Control
- Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research - research/waymax)]
- MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representations
- Natural-language-driven Simulation Benchmark and Copilot for Efficient Production of Object Interactions in Virtual Road Scenes
- LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programs
- DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes
- OccNeRF: Self-Supervised Multi-Camera Occupancy Prediction with Neural Radiance Fields
- Neural Lighting Simulation for Urban Scenes
- Street Gaussians for Modeling Dynamic Urban Scenes
- Panacea: Panoramic and Controllable Video Generation for Autonomous Driving - ad.github.io/)]
- TCLC-GS: Tightly Coupled LiDAR-Camera Gaussian Splatting for Surrounding Autonomous Driving Scenes
- Urban Architect: Steerable 3D Urban Scene Generation with Layout Prior
- DriveWorld: 4D Pre-trained Scene Understanding via World Models for Autonomous Driving
- CarDreamer: Open-Source Learning Platform for World Model based Autonomous Driving
- Probing Multimodal LLMs as World Models for Driving
- SubjectDrive: Scaling Generative Data in Autonomous Driving via Subject Control
- GAIA-1: A Generative World Model for Autonomous Driving
- CarDreamer: Open-Source Learning Platform for World Model based Autonomous Driving
- DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving
- Neural Rendering based Urban Scene Reconstruction for Autonomous Driving
- OASim: an Open and Adaptive Simulator based on Neural Rendering for Autonomous Driving - ADG/OASim), [Project](https://pjlab-adg.github.io/OASim/)]
- LimSim++: A Closed-Loop Platform for Deploying Multimodal LLMs in Autonomous Driving - adg.github.io/limsim_plus/), [Project](https://pjlab-adg.github.io/limsim_plus/)]
- ChatSim: Editable Scene Simulation for Autonomous Driving via LLM-Agent Collaboration
- LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programs
- ADriver-I: A General World Model for Autonomous Driving
- MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representations
- Natural-language-driven Simulation Benchmark and Copilot for Efficient Production of Object Interactions in Virtual Road Scenes
- DrivingDiffusion: Layout-Guided multi-view driving scene video generation with latent diffusion model
- OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Driving
- Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability
- MagicDrive3D: Controllable 3D Generation for Any-View Rendering in Street Scenes
- Unleashing Generalization of End-to-End Autonomous Driving with Controllable Long Video Generation - autolab.github.io/delphi.github.io/,)]
-
Driver Agent
- Prompting Multi-Modal Tokens to Enhance End-to-End Autonomous Driving Imitation Learning with LLMs
- AgentsCoDriver: Large Language Model Empowered Collaborative Driving with Lifelong Learning
- Grounding human-to-vehicle advice for self-driving vehicles
- Talk to the Vehicle: Language Conditioned Autonomous Navigation of Self Driving Cars
- Drive Like a Human: Rethinking Autonomous Driving with Large Language Models - ADG/DriveLikeAHuman)]
- DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model
- DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models - ADG/DiLu)]
- GPT-Driver: Learning to Drive with GPT - Driver)]
- Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving - with-LLMs)]
- LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving - mpc)]
- Receive, Reason, and React: Drive as You Say with Large Language Models in Autonomous Vehicles
- Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehicles
- SurrealDriver: Designing Generative Driver Agent Simulation Framework in Urban Contexts based on Large Language Model
- Language-Guided Traffic Simulation via Scene-Level Diffusion
- Language Prompt for Autonomous Driving
- Talk2BEV: Language-Enhanced Bird's Eye View (BEV) Maps
- BEVGPT: Generative Pre-trained Large Model for Autonomous Driving Prediction, Decision-Making, and Planning
- HiLM-D: Towards High-Resolution Understanding in Multimodal Large Language Models for Autonomous Driving
- Can you text what is happening? Integrating pre-trained language encoders into trajectory prediction models for autonomous driving
- OpenAnnotate3D: Open-Vocabulary Auto-Labeling System for Multi-modal 3D Data - ProjectTitan/OpenAnnotate3D)]
- LangProp: A Code Optimization Framework Using Language Models Applied to Driving - iclr24/LangProp)]
- Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion
- Planning with an Ensemble of World Models
- Large Language Models Can Design Game-Theoretic Objectives for Multi-Agent Planning
- TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction
- BEV-CLIP: Multi-Modal BEV Retrieval Methodology for Complex Scene in Autonomous Driving
- Large Language Models Can Design Game-theoretic Objectives for Multi-Agent Planning
- Semantic Anomaly Detection with Large Language Models
- Driving through the Concept Gridlock: Unraveling Explainability Bottlenecks in Automated Driving
- Drama: Joint risk localization and captioning in driving
- SwapTransformer: Highway Overtaking Tactical Planner Model via Imitation Learning on OSHA Dataset
- NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenario - QA)]
- Drive Anywhere: Generalizable End-to-end Autonomous Driving with Multi-modal Foundation Models
- Addressing Limitations of State-Aware Imitation Learning for Autonomous Driving
- On the Road with GPT-4V (ision): Early Explorations of Visual-Language Model on Autonomous Driving
- Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Driving - zvg/Reason2Drive)]
- GPT-4 Enhanced Multimodal Grounding for Autonomous Driving: Leveraging Cross-Modal Attention with Large Language Models
- ChatGPT as Your Vehicle Co-Pilot: An Initial Attempt
- NuScenes-MQA: Integrated Evaluation of Captions and QA for Autonomous Driving Datasets using Markup Annotations - MQA)]
- Evaluation of Large Language Models for Decision Making in Autonomous Driving
- DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving
- Large Language Models for Autonomous Driving: Real-World Experiments
- LingoQA: Video Question Answering for Autonomous Driving
- DriveLM: Driving with Graph Visual Question Answering
- LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoning
- Holistic Autonomous Driving Understanding by Bird’s-Eye-View Injected Multi-Modal Large Models - lab/NuInstruct)]
- BEV-CLIP: Multi-modal BEV Retrieval Methodology for Complex Scene in Autonomous Driving
- DME-Driver: Integrating Human Decision Logic and 3D Scene Perception in Autonomous Driving
- OmniDrive: A Holistic LLM-Agent Framework for Autonomous Driving with 3D Perception, Reasoning and Planning
- Co-driver: VLM-based Autonomous Driving Assistant with Human-like Behavior and Understanding for Complex Road Scenes
- VLP: Vision Language Planning for Autonomous Driving
- LORD: Large Models based Opposite Reward Design for Autonomous Driving
- RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Model - Driver), [Project](https://yuanjianhao508.github.io/RAG-Driver/)]
- DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models - mars-lab.github.io/DriveVLM/)]
- Embodied Understanding of Driving Scenarios
- Human-Centric Autonomous Systems With LLMs for User Command Reasoning
- LMDrive: Closed-Loop End-to-End Driving with Large Language Models
- LangProp: A Code Optimization Framework Using Language Models Applied to Driving - iclr24/LangProp)]
- Driving Everywhere with Large Language Model Policy Adaptation
- Driving Style Alignment for LLM-powered Driver Agent
- Large Language Models Powered Context-aware Motion Prediction
- DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models - ADG/DiLu)]
- NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenario - QA)]
- A Language Agent for Autonomous Driving
- HiLM-D: Towards High-Resolution Understanding in Multimodal Large Language Models for Autonomous Driving
- Can you text what is happening? Integrating pre-trained language encoders into trajectory prediction models for autonomous driving
- OpenAnnotate3D: Open-Vocabulary Auto-Labeling System for Multi-modal 3D Data - ProjectTitan/OpenAnnotate3D)]
- Semantic Anomaly Detection with Large Language Models
- Large Language Models for Autonomous Driving: Real-World Experiments
- LingoQA: Video Question Answering for Autonomous Driving
- AD-H: Autonomous Driving with Hierarchical Agents
- DriVLMe: Enhancing LLM-based Autonomous Driving Agents with Embodied and Social Experiences
- PlanAgent: A Multi-modal Large Language Agent for Closed-loop Vehicle Motion Planning
- REvolve: Reward Evolution with Large Language Models for Autonomous Driving
- Continuously Learning, Adapting, and Improving: A Dual-Process Approach to Autonomous Driving - ADG/LeapAD), [Project](https://leapad-2024.github.io/LeapAD/)]
- LMDrive: Closed-Loop End-to-End Driving with Large Language Models
- Addressing Limitations of State-Aware Imitation Learning for Autonomous Driving
-
Dataset \& Benchmark
- Cityscapes-Ref
- HAD
- Talk2Car
- HDBD
- DRAMA
- Rank2Tell - 4, METEOR, ROUGE, CIDER |
- BDD-X - 4, METEOR, CIDEr-D |
- DADA-2000
- Refer-KITTI
- DriveLM - Score |
- NuScenes-QA
- Reason2Drive - 4, METEOR, ROUGE, CIDER|
- NuScenes-MQA - 4, METEOR, ROUGE|
- LangAuto
- DriveMLM - 4, METEOR, CIDER|
- NuInstruct - , Frame-, Ego-, Instance Information, Question Answering | Question Answering, Scene Captioning | MAE, Accuracy, BLEU-4, mAP |
- DR(eye)VE
- DESIGN - 4, METEOR, ROUGE, L2 Error, Collision Rate |
-
-
:clipboard: Survey
-
Driver Agent
- Applications of Large Scale Foundation Models for Autonomous Driving
- A Survey of Large Language Models for Autonomous Driving
- Vision Language Models in Autonomous Driving and Intelligent Transportation Systems
- Choose Your Simulator Wisely: A Review on Open-source Simulators for Autonomous Driving
- Towards Knowledge-driven Autonomous Driving
- Prospective Role of Foundation Models in Advancing Autonomous Vehicles
- Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunities
- World Models for Autonomous Driving: An Initial Survey
- A Survey on Multimodal Large Language Models for Autonomous Driving
- Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectives
- A Survey for Foundation Models in Autonomous Driving
-
-
:mortar_board: Tutorial
-
Driver Agent
-
Programming Languages
Sub Categories