{"id":50572720,"url":"https://github.com/stepankonev/StandardE2E","last_synced_at":"2026-06-21T14:00:54.320Z","repository":{"id":331230848,"uuid":"1123845903","full_name":"stepankonev/StandardE2E","owner":"stepankonev","description":"A framework for unified end-to-end autonomous driving datasets processing","archived":false,"fork":false,"pushed_at":"2026-06-15T12:49:49.000Z","size":2196,"stargazers_count":22,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-15T13:24:18.126Z","etag":null,"topics":["agroverse","argoverse2","autonomous-driving","autonomous-vehicles","comma2k19","datasets","end-to-end-autonomous-driving","multi-modal-sensor-data","navsim","nuscenes","pytorch","self-driving-car","truckdrive","view-of-delft","waymo","waymo-open-dataset","wayve","world-model"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2606.04271","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stepankonev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-27T18:48:33.000Z","updated_at":"2026-06-15T12:49:21.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/stepankonev/StandardE2E","commit_stats":null,"previous_names":["stepankonev/standarde2e"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/stepankonev/StandardE2E","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stepankonev%2FStandardE2E","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stepankonev%2FStandardE2E/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stepankonev%2FStandardE2E/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stepankonev%2FStandardE2E/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stepankonev","download_url":"https://codeload.github.com/stepankonev/StandardE2E/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stepankonev%2FStandardE2E/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34610832,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-21T02:00:05.568Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agroverse","argoverse2","autonomous-driving","autonomous-vehicles","comma2k19","datasets","end-to-end-autonomous-driving","multi-modal-sensor-data","navsim","nuscenes","pytorch","self-driving-car","truckdrive","view-of-delft","waymo","waymo-open-dataset","wayve","world-model"],"created_at":"2026-06-04T20:00:18.001Z","updated_at":"2026-06-21T14:00:54.313Z","avatar_url":"https://github.com/stepankonev.png","language":"Jupyter Notebook","funding_links":[],"categories":["Papers"],"sub_categories":["Others"],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/stepankonev/StandardE2E/main/assets/standard_e2e_logo_contrast.png\" alt=\"StandardE2E Logo\" width=\"400\"/\u003e\n  \n  \u003cp\u003e\u003cem\u003eA framework for unified end-to-end autonomous driving datasets processing\u003c/em\u003e\u003c/p\u003e\n\n  ![Python versions](https://img.shields.io/badge/Python-3.12-informational?logo=python\u0026logoColor=white)\n  ![PyTorch](https://img.shields.io/badge/PyTorch-ee4c2c?logo=pytorch\u0026logoColor=white)\n  ![License](https://img.shields.io/badge/license-MIT-green.svg)\n  [![Tests](https://github.com/stepankonev/StandardE2E/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/stepankonev/StandardE2E/actions/workflows/tests.yml)\n  [![codecov](https://codecov.io/github/stepankonev/StandardE2E/graph/badge.svg?token=3MWJNB10OO)](https://codecov.io/github/stepankonev/StandardE2E)\n  [![Code Style](https://img.shields.io/badge/code%20style-black-black.svg)](https://github.com/psf/black)\n  [![mypy](https://img.shields.io/badge/mypy-checked-2A6DB2?logo=mypy\u0026logoColor=white)](http://mypy-lang.org/)\n  [![StandardE2E](https://raw.githubusercontent.com/stepankonev/StandardE2E/main/assets/StandardE2E_gh_badge_dark.svg)](https://github.com/stepankonev/StandardE2E)\n  \n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n  [![Docs](https://readthedocs.org/projects/standarde2e/badge/?version=latest)](https://standarde2e.readthedocs.io/en/latest/)\n  [![Discord](https://img.shields.io/badge/Discord-Join%20Server-5865F2?style=flat\u0026logo=discord\u0026logoColor=white)](https://discord.gg/vJnQNcQGQ8)\n\n\u003c/div\u003e\n\nStandardE2E provides a consistent interface for preprocessing, loading, and training with multimodal data from various end-to-end autonomous driving datasets. It standardizes the complex process of working with different dataset formats, allowing researchers to focus on model development rather than data engineering.\n\n\n\n\n![StandardE2E Architecture](https://raw.githubusercontent.com/stepankonev/StandardE2E/main/assets/standard_e2e_scheme.png)\n\n---\n\n## 📖 Documentation\n\n- Latest docs: https://standarde2e.readthedocs.io/en/latest/\n\n## 📦 Installation\n\n### Option 1: From PyPI (Recommended for Users)\n```bash\npip install standard-e2e\n```\n\n### Option 2: Development with uv (recommended)\n```bash\n# Install uv: https://docs.astral.sh/uv/\ngit clone https://github.com/stepankonev/StandardE2E.git\ncd StandardE2E\nuv sync --all-extras   # installs deps and dev deps from uv.lock\nuv run pytest tests/   # run tests\n```\n\n### Option 3: Manual development (pip/conda)\n```bash\nconda create -n standard_e2e python=3.12\nconda activate standard_e2e\npip install -e \".[dev]\"\n```\n\n## Plan for E2E Autonomous Driving Datasets Support\n\n\n| Dataset | Cameras | Lidar | HD Map | Detections | Driving Command | Preference Trajectories |\n|---------|---------|-------|--------|------------|-----------------|-------------------------|\n| [Waymo End-to-end](https://waymo.com/open/data/e2e/) ![](https://img.shields.io/badge/supported-darkgreen) | ![](https://img.shields.io/badge/circle-darkgreen) | ❌ | ❌ | ❌ | ✅ | ✅ |\n| [Waymo Perception](https://waymo.com/open/data/perception/) ![](https://img.shields.io/badge/supported-darkgreen) | ![](https://img.shields.io/badge/semicircle-orange) | ✅ | ✅ | ✅ | ❌ | ❌ |\n| [Navsim](https://github.com/autonomousvision/navsim/blob/main/docs/splits.md) ![](https://img.shields.io/badge/supported-darkgreen) | ![](https://img.shields.io/badge/circle-darkgreen) | ✅ | ✅ | ✅ | ✅ | ❌ |\n| [WayveScenes101](https://wayve.ai/science/wayvescenes101) ![](https://img.shields.io/badge/supported-darkgreen) | ![](https://img.shields.io/badge/semicircle-orange) | ![](https://img.shields.io/badge/SfM-orange)¹ | ❌ | ❌ | ❌ | ❌ |\n| [Argoverse 2 Sensor](https://www.argoverse.org/av2.html#sensor-link) ![](https://img.shields.io/badge/supported-darkgreen) | ![](https://img.shields.io/badge/circle-darkgreen) | ✅ | ✅ | ✅ | ❌ | ❌ |\n| [Argoverse 2 Lidar](https://www.argoverse.org/av2.html#lidar-link) ![](https://img.shields.io/badge/supported-darkgreen) | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |\n| [comma2k19](https://github.com/commaai/comma2k19) ![](https://img.shields.io/badge/supported-darkgreen) | ![](https://img.shields.io/badge/front-darkred)² | ❌ | ❌ | ❌ | ❌ | ❌ |\n| [TruckDrive](https://torc-ai.github.io/TruckDrive/) ![](https://img.shields.io/badge/supported-darkgreen) | ![](https://img.shields.io/badge/circle-darkgreen)³ | ✅ | ❌ | ✅ | ❌ | ❌ |\n| [View-of-Delft](https://intelligent-vehicles.org/datasets/view-of-delft/) ![](https://img.shields.io/badge/supported-darkgreen) | ![](https://img.shields.io/badge/front-darkred)⁴ | ✅ | ❌ | ✅ | ❌ | ❌ |\n| [nuScenes](https://www.nuscenes.org/nuscenes) ![](https://img.shields.io/badge/supported-darkgreen) | ![](https://img.shields.io/badge/circle-darkgreen)⁵ | ✅ | ✅ | ✅ | ❌ | ❌ |\n| [Argoverse 2 Map Change](https://www.argoverse.org/av2.html#mapchange-link) ![](https://img.shields.io/badge/TBD-gray) | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |\n| [KITTI](https://www.cvlibs.net/datasets/kitti/) ![](https://img.shields.io/badge/TBD-gray) | ![](https://img.shields.io/badge/front-darkred) | ✅ | ❓ | ❓ | ❓ | ❓ |\n| [KITTI-360](https://www.cvlibs.net/datasets/kitti-360/) ![](https://img.shields.io/badge/TBD-gray) | ![](https://img.shields.io/badge/front-darkred) + 2 x ![](https://img.shields.io/badge/side%20fisheye-blue) | ✅ | ❓ | ❓ | ❓ | ❓ |\n| [Waymo Motion Prediction](https://waymo.com/open/data/motion/) ![](https://img.shields.io/badge/TBD-gray) | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ |\n| [Argoverse 2 Motion Forecasting [?]](https://www.argoverse.org/av2.html#forecasting-link) ![](https://img.shields.io/badge/TBD-gray) | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ |\n\n¹ WayveScenes101 ships no sensor lidar; the `lidar_pc` modality is populated from the per-scene **COLMAP SfM** point cloud, transformed into each frame's ego frame and range-clipped so it flows through the lidar adapters. It is photogrammetric (sparse, up-to-scale), not a sensor measurement. Cameras are 5 fisheye views (forward + side arc); the ego past/future trajectory is derived from the COLMAP poses. Scenes must be extracted from the distributed `scene_\u003cNNN\u003e.zip` archives before processing.\n\n² comma2k19 is a single forward-facing 20 Hz camera (comma EON, 1164×874, pinhole) with a fused GNSS/IMU ego pose; the ego past/future trajectory is derived from the global ECEF poses and `global_position` additionally carries the ego **speed**. It ships no lidar, HD map, 3D boxes, or driving command. Segments must be extracted from the distributed `Chunk_*.zip` archives before processing (as with WayveScenes101); each `video.hevc` is then decoded frame-by-frame, forward-only. `global_position` is expressed in a per-segment local frame (ECEF-axis-aligned, origin at the segment start), so absolute X/Y/Z and heading are segment-relative; the ego-relative past/future trajectories are unaffected. Native rate is 20 Hz — use `--frame_stride` to subsample and bound output volume.\n\n³ TruckDrive is a long-range highway **heavy-truck** dataset (Torc Robotics / Princeton, CVPR 2026). Its 8 MP surround rig has **11 cameras** — more than the eight canonical `CameraDirection` slots — so each camera is mapped to the canonical surround member matching its facing (`FRONT`, `FRONT_LEFT`, `SIDE_LEFT`, `REAR_LEFT`, …) wherever one fits, with dedicated members added only for the genuinely-extra views the eight slots can't name (the forward telephoto pair `FRONT_{LEFT,RIGHT}_NARROW` and the rear-facing side pair `SIDE_{LEFT,RIGHT}_BACK`). `lidar_pc` is the seven-sensor **Aeva FMCW** joint cloud (xyz kept, in the ego frame); `detections_3d` are the tracked 3D boxes in the ego frame, with the ego vehicle's own cab/trailer and `DontCare` groups excluded per the paper taxonomy. The ego pose (PPK + LiDAR-SLAM) drives the past/future trajectory. Short-range Ouster lidar, 4D radar, accumulated GT depth and lane lines have no StandardE2E target yet and are not ingested. Frames are matched across sensors by their synchronization key. **The dataset ships as per-scene, per-modality zips and must be extracted first** — use [`scripts/extract_truckdrive.sh`](scripts/extract_truckdrive.sh), or [`scripts/prepare_dataset_truckdrive.sh`](scripts/prepare_dataset_truckdrive.sh) to extract and preprocess in one step.\n\n⁴ View-of-Delft (VoD) (TU Delft, IEEE RA-L 2022) is a compact **urban** dataset whose distinctive sensor is a **3+1D radar**. StandardE2E ingests its single **front camera** (`CameraDirection.FRONT`, 1936×1216 pinhole), the 64-layer **Velodyne** `lidar_pc` (xyz in the ego/`velodyne` frame; per-point reflectance dropped) and KITTI-format `detections_3d` mapped from camera coordinates into the ego frame — each of VoD's 13 classes folded into the coarse `DetectionType` taxonomy (the two-wheeler family `bicycle`/`Cyclist`/`rider`/`moped_scooter`/`motor` → `BICYCLE`; `Car`/`truck`/`vehicle_other` → `VEHICLE`; static/ambiguous boxes → `UNKNOWN`; `DontCare` dropped). Box yaw is VoD's KITTI rotation about the LiDAR **−Z** axis (camera-x zero-reference), so the ego-frame heading is `−(rotation + π/2)`. One **frame = one keyframe**; one **segment = one recording scene** (`delft_*`), grouped via the official scene table so the per-segment past/future ego trajectory (from the per-frame `mapToCamera` pose) never spans two recordings. The **3+1D radar** (`radar` / `radar_3frames` / `radar_5frames`) has no StandardE2E modality yet and is not ingested; per-frame **timestamps are synthesised** at the 10 Hz LiDAR-lead rate (the detection release ships none); the **test** split has sensor data but no labels (so no detections). **The dataset ships as zips and must be extracted first** — use [`scripts/extract_vod.sh`](scripts/extract_vod.sh) (lidar tree only, with an optional track-id overlay), or [`scripts/prepare_dataset_vod.sh`](scripts/prepare_dataset_vod.sh) to extract and preprocess in one step.\n\n⁵ nuScenes (Motional, CVPR 2020) is the de-facto surround-view E2E / BEV benchmark: 1000 ~20 s scenes, a **6-camera** surround rig (1600×900), a 32-beam `LIDAR_TOP` and densely annotated 3D boxes at 2 Hz keyframes (one **frame = one keyframe sample**, one **segment = one scene**). The six `CAM_*` channels map onto the canonical `CameraDirection` members; `lidar_pc` is the `LIDAR_TOP` cloud (xyz, in the ego frame); `detections_3d` are the `sample_annotation` boxes transformed from the global frame into the ego frame, each `category_name` folded into the coarse `DetectionType` taxonomy (nuScenes doesn't annotate the ego vehicle, so nothing is excluded). The ego pose drives the past/future trajectory. **HD map**: the vector **map-expansion** (lane centers recovered from the arcline paths, lane/road dividers, crossings, walkways, stop lines, drivable area, intersections) is translated to the unified `MapElementType` taxonomy in the ego frame and rasterised to BEV by `HDMapBEVAdapter`, **when** the separate `nuScenes-map-expansion-v1.3` pack is unzipped into `\u003cdataroot\u003e/maps/` (absent from the base download; the HD map is skipped if it isn't there). `--split` is an official nuScenes label that also selects the metadata version (`mini_train`/`mini_val` → `v1.0-mini`, `train`/`val` → `v1.0-trainval`, `test` → `v1.0-test`); the **test** split ships no annotations. nuScenes is read **directly from the JSON tables** — the `nuscenes-devkit` is *not* a runtime dependency (it pins `numpy\u003c2`, conflicting with this project's numpy 2.x), so the split scene-lists and the lane-arcline discretization are vendored from it (Apache-2.0). The 5 radars have no StandardE2E target yet. A **partially-downloaded** trainval converts cleanly — scenes whose sensor blob isn't on disk yet are skipped. The release ships as `.tgz` archives and **must be extracted first** — use [`scripts/extract_nuscenes.sh`](scripts/extract_nuscenes.sh), or [`scripts/prepare_dataset_nuscenes.sh`](scripts/prepare_dataset_nuscenes.sh) to extract and preprocess in one step.\n\n## 🚀 Key Features\n\n- **Unified Dataset Interface**: Work with multiple datasets through a single API\n- **Multimodal Support**: Cameras, LiDAR (point cloud + BEV histogram), HD maps (BEV raster), trajectories, detections and more\n- **Flexible Preprocessing**: Configurable pipelines with standardization and augmentation\n- **Lazy modality loading**: Configured adapters declare what they consume; no decoding work is done for modalities no adapter reads\n- **Trajectory Management**: Advanced handling of time-series vehicle data\n- **PyTorch Integration**: Ready-to-use datasets and dataloaders\n\n## 📝 Quick Start \u0026 Examples\n### Notebooks\n\n- [intro_tutorial.ipynb](notebooks/intro_tutorial.ipynb) - Introduction to StandardE2E framework\n- [containers.ipynb](notebooks/containers.ipynb) - Working with data containers\n- [multi_dataset_training_and_filtering.ipynb](notebooks/multi_dataset_training_and_filtering.ipynb) - Multi-dataset training and filtering\n- [creating_custom_adapter.ipynb](notebooks/creating_custom_adapter.ipynb) - Creating custom dataset adapters\n\n### Code Examples\n\nRun from the project root so `uv run` uses the project environment. If you use pip/conda instead, activate your env and use `python` in place of `uv run python`.\n\n1. **Preprocess Waymo End-to-end dataset** - Convert raw dataset to standardized format ([`dataset_preprocessing.py`](examples/dataset_preprocessing.py))\n    ```bash\n    uv run python examples/dataset_preprocessing.py \\\n      --e2e_dataset_path E2E_DATASET_PATH \\\n      --split {training,val,test} \\\n      --processed_data_path PROCESSED_DATA_PATH\n    ```\n2. **Train your model** - End-to-end training with multimodal data ([`very_simple_training.py`](examples/very_simple_training.py)). This example illustrates iteration over the preprocessed dataset. Also, in this example for validation we use 2 DataLoaders - full validation split and filtered validation split that only contains samples with preferred trajectories.\n    ```bash\n    uv run python examples/very_simple_training.py --processed_data_path PROCESSED_DATA_PATH\n    ```\n3. **Create a unified DataLoader**: This example shows how to process 2 different datasets within same DataLoader. First, please do preprocessing for `Waymo E2E` and `Waymo Perception` datasets in order to utilize them in the DataLoader with the script ([`prepare_datasets_waymo_e2e_perception.sh`](scripts/prepare_datasets_waymo_e2e_perception.sh)).\n\n    The script [`creating_unified_dataloader.py`](examples/creating_unified_dataloader.py) created a unified dataloader that iterates over both `Waymo E2E` and `Waymo Perception` in one epoch providing consistent data structure.\n\n    ```bash\n    uv run python examples/creating_unified_dataloader.py --processed_data_path PROCESSED_DATA_PATH\n    ```\n4. **Add a new dataset adapter** - Guide for adding support for new datasets ([`adding_new_dataset.md`](standard_e2e/caching/src_datasets/adding_new_dataset.md))\n\n\n## 📄 License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## 📚 Citation\n\nIf you find this project useful, you can support it by giving it a ⭐, or by contributing with your PRs / issues / feature requests. Also, if you use this project, you can greatly support it by citing our paper ([arXiv:2606.04271](https://arxiv.org/abs/2606.04271)):\n```bibtex\n@misc{konev2026standarde2e,\n  title={StandardE2E: A Unified Framework for End-to-End Autonomous Driving Datasets},\n  author={Stepan Konev},\n  year={2026},\n  eprint={2606.04271},\n  archivePrefix={arXiv},\n  primaryClass={cs.CV},\n  url={https://arxiv.org/abs/2606.04271}\n}\n```\nand using the badge [![StandardE2E](https://raw.githubusercontent.com/stepankonev/StandardE2E/main/assets/StandardE2E_gh_badge_dark.svg)](https://github.com/stepankonev/StandardE2E)\n\nMarkdown\n\n```markdown\n[![StandardE2E](https://raw.githubusercontent.com/stepankonev/StandardE2E/refs/heads/main/assets/StandardE2E_gh_badge_dark.svg)](https://github.com/stepankonev/StandardE2E)\n```\n\nHTML\n\n```html\n\u003ca href=\"https://github.com/stepankonev/StandardE2E\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/stepankonev/StandardE2E/refs/heads/main/assets/StandardE2E_gh_badge_dark.svg\" alt=\"StandardE2E\"/\u003e\n\u003c/a\u003e\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstepankonev%2FStandardE2E","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstepankonev%2FStandardE2E","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstepankonev%2FStandardE2E/lists"}