{"id":18478764,"url":"https://github.com/facebookresearch/ego4d","last_synced_at":"2025-05-15T01:07:49.803Z","repository":{"id":37009632,"uuid":"431206320","full_name":"facebookresearch/Ego4d","owner":"facebookresearch","description":"Ego4d dataset repository. Download the dataset, visualize, extract features \u0026 example usage of the dataset","archived":false,"fork":false,"pushed_at":"2025-01-10T22:16:27.000Z","size":49434,"stargazers_count":426,"open_issues_count":32,"forks_count":51,"subscribers_count":25,"default_branch":"main","last_synced_at":"2025-05-15T01:07:42.012Z","etag":null,"topics":["computer-vision","dataset","feature-extraction","video","visuzalization"],"latest_commit_sha":null,"homepage":"https://ego4d-data.org/docs/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/facebookresearch.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-23T18:10:47.000Z","updated_at":"2025-05-14T08:03:46.000Z","dependencies_parsed_at":"2023-02-16T15:46:24.007Z","dependency_job_id":"75d328cf-a67f-49c6-80e0-58038a752942","html_url":"https://github.com/facebookresearch/Ego4d","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2FEgo4d","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2FEgo4d/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2FEgo4d/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2FEgo4d/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/facebookresearch","download_url":"https://codeload.github.com/facebookresearch/Ego4d/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254254041,"owners_count":22039792,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","dataset","feature-extraction","video","visuzalization"],"created_at":"2024-11-06T12:11:43.473Z","updated_at":"2025-05-15T01:07:44.793Z","avatar_url":"https://github.com/facebookresearch.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003e [!IMPORTANT]\n\u003e **EGO-EXO4D DATASET ANNOUNCEMENT:** *Ego-Exo4D* **V2** is now *available to\n\u003e the public*. V2 contains **1286.30 video hours** (**221.26 ego-hours**) across 5035\n\u003e takes with **more annotations** Please refer to the\n\u003e [changelog](https://docs.ego-exo4d-data.org/changelog) for details on what\n\u003e has changed.\n\u003e\n\u003e **EGO4D UPDATE:** *V2.1* has been released due to the addition of the\n\u003e [Goal-Step](https://openreview.net/pdf?id=3BxYAaovKr) annotations and\n\u003e accompanying \"grouped videos\". Please refer to the\n\u003e [documentation](https://ego4d-data.org/docs/updates/#ego4d-goal-step--grouped-videos) for more information.\n\n# Ego4D \u0026 Ego-Exo4D\n\n**Ego-Exo4D** is a large-scale multi-modal multi-view video dataset (including 3D) and benchmark challenge. The dataset consists of time-synchronized videos of participants recorded with at least one first-person (egocentric Aria glasses) and third-person (exocentric GoPro cameras) perspective cameras. \n- Please refer to the [website](https://ego-exo4d-data.org/),\n  [documentation](https://docs.ego-exo4d-data.org/),\n  [paper](https://arxiv.org/abs/2311.18259), [blog\n  post](https://ai.meta.com/blog/ego-exo4d-video-learning-perception/) and\n  [video introduction](https://www.youtube.com/watch?v=GdooXEBAnI8).\n\n**Ego4D** is the world's largest egocentric (first person) video ML dataset and benchmark suite, including over 3700 hours of annotated first-person video data. \n- Please refer to the [website](https://ego4d-data.org/),\n  [documentation](https://ego4d-data.org/docs/) or\n  [paper](https://arxiv.org/abs/2110.07058) for more information.\n\n## Getting Started\n- To **access** the data, please refer to the Documentation:\n    - For Ego-Exo4D: refer to the [Getting Started](https://docs.ego-exo4d-data.org/getting-started/) page.\n    - For Ego4D: refer to the [Start Here](https://ego4d-data.org/docs/start-here/) page.\n- To **download** the data, refer to:\n    - For Ego-Exo4D: [Ego-Exo4D's Downloader CLI README](ego4d/egoexo/download/README.md)\n    - For Ego4D: [Ego4D's CLI README](ego4d/cli/README.md)\n- **Explore** Ego4D or Ego-Exo4D here (you'll need a license): [Ego4D Visualizer](https://visualize.ego4d-data.org/)\n- Read the [Summary](#summary) below for details about this repository.\n\n## Summary\n\nThe Ego4d repository (`ego4d` python module) includes: \n- [Ego-Exo4D Downloader CLI](ego4d/egoexo/download/README.md) for the Ego-Ego4D dataset (available as the command `egoexo`)\n- [Ego4D Downloader CLI](ego4d/cli/README.md) for the Ego4D dataset (available as the command `ego4d`)\n- A simple API abstracting common video reading libraries\n([TorchAudio](https://github.com/facebookresearch/Ego4d/blob/main/ego4d/research/readers.py#L69),\n[PyAV](https://github.com/facebookresearch/Ego4d/blob/main/ego4d/research/readers.py#L136)),\n- An API for [feature\nextraction](https://github.com/facebookresearch/Ego4d/blob/main/ego4d/features/README.md#as-an-api), which includes [lightweight wrappers for common models](https://github.com/facebookresearch/Ego4d/tree/main/ego4d/features/models), such as: [Omnivore](https://github.com/facebookresearch/Ego4d/blob/main/ego4d/features/models/omnivore.py) and [SlowFast](https://github.com/facebookresearch/Ego4d/blob/main/ego4d/features/models/slowfast.py)\n- Notebooks (for [Ego4D](https://github.com/facebookresearch/Ego4d/tree/main/notebooks) and [Ego-Exo4D]()) serving as examples/tutorials to analyze \u0026 use the dataset\n    - Colab notebooks for Ego4D serve as additional examples for the benchmarks (VQ, NLQ and STA) can be found on: https://ego4d-data.org/docs/challenge/\n- Research code to train models on the dataset, e.g. [clep](https://github.com/facebookresearch/Ego4d/tree/main/ego4d/research/clep)\n    - **NOTE:** baseline code for Ego-Exo4D is coming soon!\n    - **NOTE:** baseline code for the Ego4D benchmarks exists on separate GitHub repositories, see the [EGO4D organization](https://github.com/EGO4D/) and [docs](https://ego4d-data.org/docs/benchmarks/overview/)\n\n\nPlease see [**Structure of the Repository**](#structure-of-the-repository) below for details.\n\n## Setup\n\nPlease follow the below instructions to setup the downloader CLI for Ego4d and\nto install the `ego4d` python module. \n\n### Option 1: From the PyPi package\n\n\n\u003e[!TIP]\n\u003ePlease ensure you have a conda or pyenv environment created \u0026 activated. If you're unsure\n\u003eon how to do so, you can follow [Option 2: Step 1](step-1-create-or-use-an-environment).\n\n```\npip install ego4d --upgrade\n```\n\n**NOTE:** Please ensure you are on at least Python 3.10\n\n### Option 2: Clone/Download the Code\n\nEnsure you have cloned or downloaded the code to your local disk. All\ninstructions assume you are the **root of the repository**.\n\n#### Step 1: Create or Use an Environment\n\nCreate a conda environment to enable pip installation:\n```\nconda create -n ego4d python=3.11 -y\nconda activate ego4d\n```\n\nIf you are using an existing conda (or pyenv) environment: please ensure you\nhave installed *at least* Python 3.10.\n\n#### Step 2: \n\n```\npip install .  # run from the root of Ego4d\n```\n\nNow you should be able to import ego4d:\n\n```\npython3 -c 'import ego4d; print(ego4d)'\n```\n\nYou can check that the ego4d module links to the correct file on your file system from the output of the above command.\n\n## Structure of the Repository\nThe repository contains multiple directories covering a specific theme. Each\ntheme contains an associated `README.md` file, please refer to them.\n\n\nAll python code is located in the `ego4d` and associated subdirectories. The\ngoal for each subdirectory is to cover one specific theme. \n\n- `ego4d`: the `ego4d` *python* module exists\n    - [`cli`](ego4d/cli/README.md): The Ego4D CLI for downloading the dataset\n    - [`features`](ego4d/features/README.md): Feature extraction across the dataset\n    - [`research`](ego4d/research/README.md): Everything related to research and\n      usage of the dataset (dataloaders, etc).\n        - [`research/clep`](ego4d/research/clep/README.md): Contrastive Language Ego-centric video Pre-training\n- [`viz`](viz/narrations/README.md): visualization engine\n\n## Visualization and Demo\n- For a demo notebook: [Annotation Notebook](notebooks/annotation_visualization.ipynb)\n- For the visualization engine: [Viz README](viz/narrations/README.md)\n\n# License\n\nEgo4D is released under the [MIT License](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffacebookresearch%2Fego4d","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffacebookresearch%2Fego4d","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffacebookresearch%2Fego4d/lists"}