{"id":13684795,"url":"https://github.com/chengaopro/Awesome-EmbodiedAI","last_synced_at":"2025-05-01T00:33:02.763Z","repository":{"id":217137633,"uuid":"284692928","full_name":"chengaopro/Awesome-EmbodiedAI","owner":"chengaopro","description":"A curated list about Awesome Embodied AI works and is still in construct. Now it contains a list of Simulators, Tasks and Datasets.","archived":false,"fork":false,"pushed_at":"2020-08-19T13:13:21.000Z","size":5,"stargazers_count":31,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-24T03:01:53.451Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chengaopro.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-08-03T12:22:49.000Z","updated_at":"2025-03-03T02:47:50.000Z","dependencies_parsed_at":"2024-01-17T07:59:46.778Z","dependency_job_id":"9372d53f-508c-4eda-95d0-315e62061498","html_url":"https://github.com/chengaopro/Awesome-EmbodiedAI","commit_stats":null,"previous_names":["chengaopro/awesome-embodiedai"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chengaopro%2FAwesome-EmbodiedAI","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chengaopro%2FAwesome-EmbodiedAI/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chengaopro%2FAwesome-EmbodiedAI/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chengaopro%2FAwesome-EmbodiedAI/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chengaopro","download_url":"https://codeload.github.com/chengaopro/Awesome-EmbodiedAI/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251686741,"owners_count":21627474,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T14:00:38.862Z","updated_at":"2025-05-01T00:33:02.686Z","avatar_url":"https://github.com/chengaopro.png","language":null,"funding_links":[],"categories":["Other awesome AI lists","Other Lists"],"sub_categories":["TeX Lists"],"readme":"# Awesome EmbodiedAI (still in construct) [![Awesome](https://awesome.re/badge.svg)](https://awesome.re)\r\n\r\nWe maintain a curated list of Awesome Embodied AI works. Currently, we include simulators, tasks and datasets in Embodied AI field. \r\n- Simulators help render images and simulate the behavior of agents, as if they are situated in an real world environment. \r\n- Datasets provide training data (e.g. navigation instructions) and ground truths (e.g. navigation trajectories). \r\n\r\n(Some simulator comes along with a dataset with the same name, so there might be duplicated names in different sections.)\r\n\r\nPlease feel free to pull requests or open an issue to add papers.\r\n\r\n## Awesome companies\r\n\r\n- [Covariant](https://covariant.ai/)\r\n- [Mujin](https://www.mujin.co.jp/en/)\r\n- [Righthand](https://www.righthandrobotics.com/)\r\n- [Boston Dynamics](https://www.bostondynamics.com/)\r\n\r\n## Awesome Labs\r\n\r\n- [SVL](http://svl.stanford.edu/)\r\n\r\n## Simulator\r\n\r\n_Platform to simulate real world environments._\r\n\r\n- Habitat-Simulator\r\n  - Venue/Year: ICCV 2019 | [[paper]](https://arxiv.org/abs/1904.01201) [[code]](https://github.com/facebookresearch/habitat-sim) [[homepage]](https://aihabitat.org/)\r\n  - Visual Content: Matterport3D, House3D, AI2-THOR, etc. (_partially realistic_)\r\n  - Action Space: continuous\r\n- AI2-THOR  \r\n  - Venue/Year: Arxiv 2019 | [[paper]](https://arxiv.org/abs/1712.05474) [[code]](https://github.com/allenai/ai2thor) [[homepage]](https://ai2thor.allenai.org/)\r\n  - Visual Content: AI2-THOR\r\n  - Action Space: continuous\r\n  - Interactive: Yes\r\n- CHALET \r\n  - Venue/Year: Arxiv 2019 | [[paper]](https://arxiv.org/abs/1801.07357) [[code]](https://github.com/lil-lab/chalet)\r\n  - Visual Content: CHALET\r\n  - Action Space: continuous\r\n  - Interactive: Yes\r\n- Matterport3D \r\n  - Venue/Year: 3DV 2017 | [[paper]](https://arxiv.org/abs/1709.06158) [[code]](https://github.com/niessner/Matterport) [[homepage]](https://niessner.github.io/Matterport/) \r\n  - Visual Content: Matterport3D (realistic)\r\n  - Action Space: graph based\r\n- MINOS \r\n  - Venue/Year: CVPR 2017 | [[paper]](https://arxiv.org/abs/1712.03931) [[code]](https://github.com/minosworld/minos) [[homepage]](https://minosworld.github.io/)\r\n  - Visual Content: SUNCG+Matterport3D (partially realistic)\r\n  - Action Space: continuous\r\n- Gibson \r\n  - Venue/Year: CVPR 2018 | [[paper]](https://arxiv.org/abs/1808.10654) [[code]](https://github.com/StanfordVL/GibsonEnv) [[homepage]](http://gibsonenv.stanford.edu/)\r\n  - Visual Content: Gibson+2D3DS+Matterport3D (realistic)\r\n  - Action Space: continuous\r\n  - Interactive: Yes\r\n- House3D \r\n  - Venue/Year: Arxiv 2018 | [[paper]](https://arxiv.org/abs/1801.02209) [[code]](https://github.com/facebookresearch/House3D)\r\n  - Visual Content: SUNCG\r\n  - Action Space: continuous\r\n- SUNCG \r\n  - Venue/Year: CVPR 2017 | [[paper]](https://arxiv.org/abs/1611.08974) \r\n  - Visual Content: SUNCG\r\n- HoME  \r\n  - Venue/Year: NIPS 2017 | [[paper]](https://arxiv.org/abs/1711.11017) [[code]](https://github.com/ml-lab/home-platform)\r\n  - Visual Content: SUNCG\r\n  - language content: description of objects\r\n  - Action Space: continuous\r\n- VirtualHome  \r\n  - Venue/Year: CVPR 2018 | [[paper]](https://arxiv.org/abs/1806.07011) [[code]](https://github.com/xavierpuigf/virtualhome) [homepage](http://virtual-home.org/) \r\n  - Visual Content: VirtualHome\r\n  - Action Space: continuous\r\n  - Interactive: Yes\r\n- SceneNet RGB-D \r\n  - Venue/Year: ICCV 2017 | [[paper]](http://www.imperial.ac.uk/media/imperial-college/research-centres-and-groups/dyson-robotics-lab/jmccormac_etal_iccv2017.pdf) [[code]](https://github.com/jmccormac/pySceneNetRGBD) [[homepage]](https://robotvault.bitbucket.io/scenenet-rgbd.html)\r\n  - Visual Content: SceneNet RGB-D\r\n  - Action Space: continuous\r\n  - Interactive: Yes\r\n\r\n## Tasks\r\n\r\n_Embodied task definitions._\r\n\r\n[REVERIE](https://arxiv.org/abs/1904.10151) - requires an intelligent agent to correctly localize a remote target object (can not be observed at starting location) specified by a concise high-level natural language instruction.\r\n\r\n[VLN](https://arxiv.org/abs/1711.07280) - requires an embodied agent to follow natural language instructions to navigate from a starting pose to a goal location.\r\n\r\n[VNLA](https://arxiv.org/abs/1812.04155) - a grounded vision-language task where an agent with visual perception is guided via language to find objects in photorealistic indoor environments.\r\n\r\n[EQA](https://arxiv.org/abs/1711.11543) - an agent is spawned at a random location in a 3D environment and asked a question. The agent must first intelligently navigate to explore the environment, gather necessary visual information through first-person (egocentric) vision, and then answer the question.\r\n\r\n[IQA](https://arxiv.org/abs/1712.03316) - requires an agent to navigate around the scene, acquire visual understanding of scene elements, interact with objects (e.g. open refrigerators) and plan for a series of actions conditioned on the question.\r\n\r\n[TOUCHDOWN](https://arxiv.org/abs/1811.12354) - requires an agent to first follow navigation instructions in a real-life visual urban environment, and then identify a location described in natural language to find a hidden object at the goal position.\r\n\r\n## Dataset\r\n\r\n_Embodied datasets built upon simulators._\r\n\r\n- REVERIE _CVPR 2020_ based on Matterport3D [paper](https://arxiv.org/abs/1904.10151) [code](https://github.com/YuankaiQi/REVERIE) \r\n  - language content: navigation instructions\r\n  - applicable tasks: REVERIE, VLN, referring expression\r\n- R2R _CVPR 2018_ based on Matterport3D [paper](https://arxiv.org/abs/1711.07280) [homepage](https://bringmeaspoon.org/)\r\n  - language content: navigation instructions\r\n  - applicable tasks: VLN\r\n- VNLA _CVPR 2019_ based on Matterport3D [paper](https://arxiv.org/abs/1812.04155) [code](https://github.com/debadeepta/vnla) \r\n  - language content: navigation instructions and assistance\r\n  - applicable tasks: VNLA, VLN, referring expression\r\n- HANNA _EMNLP 2019_ based on Matterport3D [paper](https://arxiv.org/abs/1909.01871) [code](https://github.com/khanhptnk/hanna)\r\n  - language content: navigation instructions and assistance\r\n  - applicable tasks: VNLA, VLN, referring expression\r\n- CVDN _CoRL 2019_ based on Matterport3D [paper](https://arxiv.org/abs/1907.04957) [code](https://github.com/mmurray/cvdn/) [homepage](https://cvdn.dev/) \r\n  - language content: dialogues\r\n  - applicable tasks: VNLA, VLN\r\n- EQA _CVPR 2018_ based on House3D [paper](https://arxiv.org/abs/1711.11543) [code](https://github.com/facebookresearch/EmbodiedQA) [homepage](https://embodiedqa.org/)\r\n  - language content: question-answer pairs\r\n  - applicable tasks: EQA, VLN\r\n- IQUADv1 _CVPR 2018_ based on AI2-THOR [paper](https://arxiv.org/abs/1712.03316) [code](https://github.com/danielgordon10/thor-iqa-cvpr-2018)\r\n  - language content: question-answer pairs\r\n  - applicable tasks: IQA, EQA, VLN\r\n- TOUCHDOWN _CVPR 2019_ based on Google Street View [paper](https://arxiv.org/abs/1811.12354) [code](https://github.com/lil-lab/touchdown)\r\n  - language content: navigation instructions\r\n  - applicable tasks: TOUCHDOWN, VLN, referring expression\r\n- Talk The Way _2018_ [paper](https://arxiv.org/abs/1807.03367) [code](https://github.com/facebookresearch/talkthewalk/)\r\n  - visual content: manually captured neighborhoods of New York City\r\n  - language content: navigation dialogues\r\n  - applicable tasks: VNLA, VLN\r\n- LANI \u0026 CHAI _2019_ based on CHALET [paper](https://arxiv.org/abs/1801.07357) [code](https://github.com/lil-lab/chalet)\r\n  - language content: navigation instructions\r\n  - applicable tasks: VLN\r\n- Activity \u0026 ActivityPrograms _CVPR 2018_ [paper](https://arxiv.org/abs/1806.07011) [code](https://github.com/xavierpuigf/virtualhome) [homepage](http://virtual-home.org/) \r\n  - language content: task descriptions\r\n  - applicable tasks: VLN\r\n- Habitat _ICCV 2019_ [paper](https://arxiv.org/abs/1904.01201) [code](https://github.com/facebookresearch/habitat-sim) [homepage](https://aihabitat.org/) \r\n  - language content: navigation instructions, task descriptions, etc.\r\n  - applicable tasks: IQA, EQA, VLN, language grounding, etc.\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchengaopro%2FAwesome-EmbodiedAI","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchengaopro%2FAwesome-EmbodiedAI","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchengaopro%2FAwesome-EmbodiedAI/lists"}