{"id":20048203,"url":"https://github.com/tri-ml/dgp","last_synced_at":"2025-04-04T16:15:33.381Z","repository":{"id":38298897,"uuid":"263108125","full_name":"TRI-ML/dgp","owner":"TRI-ML","description":"ML Dataset Governance Policy for Autonomous Vehicle Datasets","archived":false,"fork":false,"pushed_at":"2025-01-04T01:55:43.000Z","size":95886,"stargazers_count":98,"open_issues_count":11,"forks_count":63,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-28T15:09:14.547Z","etag":null,"topics":["autonomous-driving","data-management","deep-learning","machine-learning","pytorch"],"latest_commit_sha":null,"homepage":"https://tri-ml.github.io/dgp/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TRI-ML.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"docs/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-05-11T17:14:43.000Z","updated_at":"2025-02-14T19:01:54.000Z","dependencies_parsed_at":"2025-02-23T22:10:55.284Z","dependency_job_id":"5ad6c06c-1607-4485-ac25-165de7a7f4a1","html_url":"https://github.com/TRI-ML/dgp","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TRI-ML%2Fdgp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TRI-ML%2Fdgp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TRI-ML%2Fdgp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TRI-ML%2Fdgp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TRI-ML","download_url":"https://codeload.github.com/TRI-ML/dgp/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247208175,"owners_count":20901570,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autonomous-driving","data-management","deep-learning","machine-learning","pytorch"],"created_at":"2024-11-13T11:41:13.069Z","updated_at":"2025-04-04T16:15:33.357Z","avatar_url":"https://github.com/TRI-ML.png","language":"Python","readme":"\u003c!-- markdownlint-disable-next-line --\u003e\n\n[\u003cimg src=\"docs/tri-logo.png\" width=\"40%\"\u003e](https://www.tri.global/)\n\n# Dataset Governance Policy (DGP)\n\n[![build-docker](https://github.com/TRI-ML/dgp/actions/workflows/build-docker.yml/badge.svg)](https://github.com/TRI-ML/dgp/actions/workflows/build-docker.yml)\n[![license](https://img.shields.io/github/license/TRI-ML/dgp.svg)](https://github.com/TRI-ML/dgp/blob/master/LICENSE)\n[![open-issues](https://img.shields.io/github/issues/TRI-ML/dgp.svg)](https://github.com/TRI-ML/dgp/issues)\n![coverage badge](./docs/coverage.svg)\n[![docs](https://img.shields.io/badge/documentation-beta-red)](https://tri-ml.github.io/dgp/)\n\nTo ensure the traceability, reproducibility and standardization for all ML\ndatasets and models generated and consumed within Toyota Research Institute\n(TRI), we developed the Dataset-Governance-Policy (DGP) that codifies the schema\nand maintenance of all TRI's Autonomous Vehicle (AV) datasets.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/3d-viz-proj.gif\" alt=\"3d-viz-proj\"/\u003e\n\u003c/p\u003e\n\n## Components\n\n- [Schema](dgp/proto/README.md):\n  [Protobuf](https://developers.google.com/protocol-buffers)-based schemas for\n  raw data, annotations and dataset management.\n- [DataLoaders](dgp/datasets): Universal PyTorch DatasetClass to load all\n  DGP-compliant datasets.\n- [CLI](dgp/README.md): Main CLI for handling DGP datasets and the entrypoint of\n  visulization tools.\n\n## Getting Started\n\nPlease see [Getting Started](docs/GETTING_STARTED.md) for environment setup.\n\nGetting started is as simple as initializing a dataset-class with the relevant\ndataset JSON, raw data sensor names, annotation types, and split information.\nBelow, we show a few examples of initializing a Pytorch dataset for multi-modal\nlearning from 2D bounding boxes, and 3D bounding boxes.\n\n```python\nfrom dgp.datasets import SynchronizedSceneDataset\n\n# Load synchronized pairs of camera and lidar frames, with 2d and 3d\n# bounding box annotations.\ndataset = SynchronizedSceneDataset('\u003cdataset_name\u003e_v0.0.json',\n    datum_names=('camera_01', 'lidar'),\n    requested_annotations=('bounding_box_2d', 'bounding_box_3d'),\n    split='train')\n```\n\n## Examples\n\nA list of starter scripts are provided in the [examples](examples/) directory.\n\n- [examples/load_dataset.py](examples/load_dataset.py): Simple example script to\n  load a multi-modal dataset based on the **Getting Started** section above.\n\n## Build and run tests\n\nYou can build the base docker image and run the tests within\n[docker container](docs/GETTING_STARTED.md#markdown-header-develop-within-docker)\nvia:\n\n```sh\nmake docker-build\nmake docker-run-tests\n```\n\n## Contributing\n\nWe appreciate all contributions to DGP! To learn more about making a\ncontribution to DGP, please see [Contribution Guidelines](docs/CONTRIBUTING.md).\n\n## CI Ecosystem\n\n| Job          | CI                                                                                                                                                              | Notes                                                                                           |\n| ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- |\n| docker-build | [![Build Status](https://github.com/TRI-ML/dgp/actions/workflows/build-docker.yml/badge.svg)](https://github.com/TRI-ML/dgp/actions/workflows/build-docker.yml) | Docker build and push to [container registry](https://github.com/TRI-ML/dgp/pkgs/container/dgp) |\n| pre-merge    | [![Build Status](https://github.com/TRI-ML/dgp/actions/workflows/pre-merge.yml/badge.svg)](https://github.com/TRI-ML/dgp/actions/workflows/pre-merge.yml)       | Pre-merge testing                                                                               |\n| doc-gen      | [![Build Status](https://github.com/TRI-ML/dgp/actions/workflows/doc-gen.yml/badge.svg)](https://github.com/TRI-ML/dgp/actions/workflows/doc-gen.yml)           | [GitHub Pages](https://tri-ml.github.io/dgp/) doc generation                                    |\n| coverage     | [![Build Status](https://github.com/TRI-ML/dgp/actions/workflows/coverage.yml/badge.svg)](https://github.com/TRI-ML/dgp/actions/workflows/coverage.yml)         | Code coverage metrics and badge generation                                                      |\n\n## 💬 Where to file bug reports\n\n| Type                    | Platforms                                                    |\n| ----------------------- | ------------------------------------------------------------ |\n| 🚨 **Bug Reports**      | [GitHub Issue Tracker](https://github.com/TRI-ML/dgp/issues) |\n| 🎁 **Feature Requests** | [GitHub Issue Tracker](https://github.com/TRI-ML/dgp/issues) |\n\n## 👩‍💻 The Team 👨‍💻\n\nDGP is developed and currently maintained by _Quincy Chen, Arjun Bhargava, Chao\nFang, Chris Ochoa and Kuan-Hui Lee_ from ML-Engineering team at\n[Toyota Research Institute (TRI)](https://www.tri.global/), with contributions\ncoming from ML-Research team at TRI,\n[Woven Planet](https://www.woven-planet.global/en) and\n[Parallel Domain](https://paralleldomain.com/).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftri-ml%2Fdgp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftri-ml%2Fdgp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftri-ml%2Fdgp/lists"}