{"id":25365815,"url":"https://github.com/willguimont/torch_waymo","last_synced_at":"2026-04-06T03:34:26.130Z","repository":{"id":65613728,"uuid":"595324616","full_name":"willGuimont/torch_waymo","owner":"willGuimont","description":"PyTorch dataloader for Waymo Open Dataset","archived":false,"fork":false,"pushed_at":"2023-12-20T00:11:51.000Z","size":20,"stargazers_count":14,"open_issues_count":1,"forks_count":3,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-08-16T22:25:04.411Z","etag":null,"topics":["dataloader","dataset","point-cloud","pytorch","torch","waymo"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/willGuimont.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-01-30T21:12:04.000Z","updated_at":"2025-05-06T20:38:11.000Z","dependencies_parsed_at":"2023-11-17T01:09:33.931Z","dependency_job_id":"3b14f6ce-7cb4-48ee-a865-3ad8f11071fc","html_url":"https://github.com/willGuimont/torch_waymo","commit_stats":{"total_commits":11,"total_committers":3,"mean_commits":"3.6666666666666665","dds":0.2727272727272727,"last_synced_commit":"aef821810d345b9315a4eb7a2cc1fe65247baeba"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/willGuimont/torch_waymo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/willGuimont%2Ftorch_waymo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/willGuimont%2Ftorch_waymo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/willGuimont%2Ftorch_waymo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/willGuimont%2Ftorch_waymo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/willGuimont","download_url":"https://codeload.github.com/willGuimont/torch_waymo/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/willGuimont%2Ftorch_waymo/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273082421,"owners_count":25042282,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-01T02:00:09.058Z","response_time":120,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dataloader","dataset","point-cloud","pytorch","torch","waymo"],"created_at":"2025-02-14T23:50:41.281Z","updated_at":"2025-10-08T19:01:50.523Z","avatar_url":"https://github.com/willGuimont.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# torch_waymo\n\nLoad Waymo Open Dataset in PyTorch\n\nCite this repository:\n```\n@software{Guimont-Martin_A_PyTorch_dataloader_2023,\n    author = {Guimont-Martin, William},\n    month = {1},\n    title = {{A PyTorch dataloader for Waymo Open Dataset}},\n    version = {0.1.1},\n    year = {2023}\n}\n```\n\n## Usage\n\nRequires:\n- Python \u003c 3.10\n\n### Download the dataset\n\n```shell\n# Login to gcloud\ngcloud auth login\n\n# Download the full dataset\ncd \u003cpath/to/waymo\u003e\ngsutil -m cp -r \\\n  \"gs://waymo_open_dataset_v_1_4_1/individual_files/training\" \\\n  \"gs://waymo_open_dataset_v_1_4_1/individual_files/validation\" \\\n  .\n```\n\n### Convert it\n\n```shell\n# Make a tf venv with Python \u003c 3.10\npython3.9 -m venv venv_tf\nsource venv_tf/bin/activate\n\n# We recommend using uv for faster installs\npip install uv\nuv pip install \"torch_waymo[waymo]\"\n\n# Convert all splits (FULL frames with images \u0026 lasers) -\u003e writes to \u003cpath\u003e/converted\ntorch-waymo-convert --dataset \u003cpath/to/waymo\u003e\n\n# Convert only training split (FULL frames)\ntorch-waymo-convert --dataset \u003cpath/to/waymo\u003e --splits training\n\n# Convert multiple splits (FULL frames)\ntorch-waymo-convert --dataset \u003cpath/to/waymo\u003e --splits training validation\n\n# (NEW) Convert to SIMPLIFIED frames (no camera images stored, point cloud + labels only)\n# Writes to \u003cpath\u003e/converted_simplified\ntorch-waymo-convert --dataset \u003cpath/to/waymo\u003e --simplified\n\n# Simplified + specific splits\ntorch-waymo-convert --dataset \u003cpath/to/waymo\u003e --simplified --splits training validation\n```\n\n### Load it in your project\n\nNow that the dataset is converted, you don't have to depend on `waymo-open-dataset-tf-2-11-0` in your downstream project.\nYou can simply install `torch_waymo` in your *runtime* environment.\n\n```shell\npip install torch_waymo\n```\nExample usage:\ntrain_dataset = WaymoDataset('~/Datasets/Waymo/converted_simplified', 'training')\nExample usage (Full conversion):\n```python\nfrom torch_waymo import WaymoDataset\n\n# Simplified frames (no images, only point clouds + labels)\ntrain_dataset = WaymoDataset('~/Datasets/Waymo/converted_simplified', 'training')\nfor i in range(10):\n    # frame is of type SimplifiedFrame\n    frame = train_dataset[i]\n    print(frame.timestamp_micros)\n    print(frame.timestamp_micros, len(frame.lasers))\n\n# Full frames (with images)\ntrain_dataset = WaymoDataset('~/Datasets/Waymo/converted', 'training')\nfor i in range(10):\n    # frame is of type Frame\n    frame = train_dataset[i]\n    print(frame.timestamp_micros)\n    print(frame.timestamp_micros, len(frame.images))\n```\n\nNotes:\n- Paths with `~` are supported; they will expand to your home directory.\n- `len.pkl` inside each split directory stores cumulative frame counts for indexing.\n- If you re-run conversion, existing frames are skipped (idempotent per frame file).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwillguimont%2Ftorch_waymo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwillguimont%2Ftorch_waymo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwillguimont%2Ftorch_waymo/lists"}