https://github.com/facebookresearch/house3d

a Realistic and Rich 3D Environment
https://github.com/facebookresearch/house3d

Last synced: 3 months ago
JSON representation

a Realistic and Rich 3D Environment

• Host: GitHub
• URL: https://github.com/facebookresearch/house3d
• Owner: facebookresearch
• License: apache-2.0
• Archived: true
• Created: 2017-11-29T18:41:51.000Z (about 7 years ago)
• Default Branch: master
• Last Pushed: 2020-07-06T04:41:49.000Z (over 4 years ago)
• Last Synced: 2024-09-25T21:43:48.047Z (3 months ago)
• Language: C++
• Size: 661 KB
• Stars: 1,189
• Watchers: 68
• Forks: 187
• Open Issues: 10
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md

Awesome Lists containing this project

• StarryDivineSky - facebookresearch/house3d

README

        
# House3D: A Rich and Realistic 3D Environment

#### [Yi Wu](https://jxwuyi.weebly.com/), [Yuxin Wu](https://github.com/ppwwyyxx), [Georgia Gkioxari](https://gkioxari.github.io/) and [Yuandong Tian](http://yuandong-tian.com/)

#



House3D is a virtual 3D environment which consists of thousands of indoor scenes equipped with

a diverse set of scene types, layouts and objects sourced from the [SUNCG dataset](https://arxiv.org/abs/1611.08974).

It consists of over 45k indoor 3D scenes, ranging from studios to two-storied houses

with swimming pools and fitness rooms. All 3D objects are fully annotated with category labels.

Agents in the environment have access to observations of multiple modalities, including RGB images,

depth, segmentation masks and top-down 2D map views. The renderer runs at

thousands frames per second, making it suitable for large-scale RL training.

Usage instructions can be found at [INSTRUCTION.md](INSTRUCTION.md)

## Existing Research Projects with House3D

### A. RoomNav ([paper](https://arxiv.org/abs/1801.02209))

*Yi Wu, Yuxin Wu, Georgia Gkioxari, Yuandong Tian*

In this work we introduce a concept learning task, RoomNav, where an agent is asked to navigate to a destination specified by a high-level concept, e.g. `dining room`.

We demonstrated two neural models: a gated-CNN and a gated-LSTM, which effectively improve the agent's sensitivity to different concepts.

For evaluation, we emphasize on generalization ability and show that our agent can __generalize across environments__

due to the diverse and large-scale dataset.







### B. Embodied QA ([project page](http://embodiedqa.org/) | [EQA paper](https://arxiv.org/abs/1711.11543) | [NMC paper](https://arxiv.org/abs/1810.11181))

*Abhishek Das, Samyak Datta, Georgia Gkioxari, Stefan Lee, Devi Parikh, Dhruv Batra*

Embodied Question Answering is a new AI task where an agent is spawned at a random location in a 3D environment and asked a natural language question ("What color is the car?").

In order to answer, the agent must first intelligently navigate to explore the environment, gather information through first-person (egocentric) vision, and then answer the question ("orange").







## Citation

If you use our platform in your research, you can cite us with:

```

@article{wu2018building,

  title={Building generalizable agents with a realistic and rich 3D environment},

  author={Wu, Yi and Wu, Yuxin and Gkioxari, Georgia and Tian, Yuandong},

  journal={arXiv preprint arXiv:1801.02209},

  year={2018}

}

```

## License

House3D is released under the [Apache 2.0 license](LICENSE).