{"id":13600944,"url":"https://github.com/UT-Austin-RPL/HouseDitto","last_synced_at":"2025-04-11T01:30:35.121Z","repository":{"id":191239242,"uuid":"537335279","full_name":"UT-Austin-RPL/HouseDitto","owner":"UT-Austin-RPL","description":"Code for Ditto in the House: Building Articulation Models of Indoor Scenes through Interactive Perception","archived":false,"fork":false,"pushed_at":"2023-08-25T17:13:30.000Z","size":110615,"stargazers_count":11,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-11-07T03:42:27.054Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://ut-austin-rpl.github.io/HouseDitto/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/UT-Austin-RPL.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-09-16T06:34:56.000Z","updated_at":"2024-08-31T11:37:44.000Z","dependencies_parsed_at":"2023-08-29T00:38:05.569Z","dependency_job_id":null,"html_url":"https://github.com/UT-Austin-RPL/HouseDitto","commit_stats":null,"previous_names":["ut-austin-rpl/houseditto"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UT-Austin-RPL%2FHouseDitto","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UT-Austin-RPL%2FHouseDitto/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UT-Austin-RPL%2FHouseDitto/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UT-Austin-RPL%2FHouseDitto/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/UT-Austin-RPL","download_url":"https://codeload.github.com/UT-Austin-RPL/HouseDitto/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248324968,"owners_count":21084841,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T18:00:51.361Z","updated_at":"2025-04-11T01:30:32.463Z","avatar_url":"https://github.com/UT-Austin-RPL.png","language":"Python","funding_links":[],"categories":["Reconstruction"],"sub_categories":[],"readme":"# Ditto in the House: Building Articulation Models of Indoor Scenes through Interactive Perception\n\n\n[Cheng-Chun Hsu](https://chengchunhsu.github.io/), [Zhenyu Jiang](http://zhenyujiang.me), [Yuke Zhu](https://www.cs.utexas.edu/~yukez/)\n\nICRA 2023\n\n[Project](https://ut-austin-rpl.github.io/HouseDitto/) | [arxiv](https://arxiv.org/abs/2302.01295)\n\n![teaser](assets/teaser.png)\n\n## Introduction\nOur approach, named *Ditto in the House*, discovers possible articulated objects through affordance prediction, interacts with these objects to produce articulated motions, and infers the articulation properties from the visual observations before and after each interaction. The approach consists of two stages — affordance prediction and articulation inference. During affordance prediction, we pass the static scene point cloud into the affordance network and predict the scene-level affordance map. Then, the robot interacts with the object based on those contact points. During articulation inference, we feed the point cloud observations before and after each interaction into the articulation model network to obtain articulation estimation. By aggregating the estimated articulation models, we build the articulation models of the entire scene.\n\n![intro](assets/framework.png)\n\nIf you find our work useful in your research, please consider [citing](#citing).\n\n## Installation\nThe codebase consists of three modules:\n\n- `DITH-igibson`: interaction and observation collection in iGibson simulator\n- `DITH-pointnet`: affordance prediction\n- `DITH-ditto`: articulation inference\n\nCreate conda environments and install required packages by running\n\n```bash\ncd DITH-igibson\nconda env create -f conda_env.yaml -n DITH-igibson\n\ncd ../DITH-pointnet\nconda env create -f conda_env.yaml -n DITH-pointnet\n\ncd ../DITH-ditto\nconda env create -f conda_env.yaml -n DITH-ditto\n```\n\nBuild Ditto's dependents by running\n\n```bash\ncd DITH-ditto \u0026\u0026 conda activate DITH-ditto\npython scripts/convonet_setup.py build_ext --inplace\n```\n\n## Data Collection\n\n1. Run `cd DITH-igibson \u0026\u0026 conda activate DITH-igibson`\n2. Follow [these instructions](https://github.com/StanfordVL/iGibson/tree/master/igibson/utils/data_utils/ext_scene) to import CubiCasa5k scenes into iGibon simulator.\n3. Generate training and testing data by running\n\n```bash\npython room_dataset_generate.py\npython room_dataset_split.py\npython room_dataset_preprocess.py\n```\n\nThe generated data can be found under `dataset/cubicasa5k_rooms_processed`.\n\n## Affordance Prediction\n\n1. Run `cd DITH-pointnet \u0026\u0026 conda activate DITH-pointnet`\n2. Set `datadir` in `configs/train_pointnet2.yaml` and `configs/test_pointnet2.yaml`.\n\n3. Train the model\n\n```bash\npython train.py\n```\n\n1. Set `ckpt_path` in `configs/test_pointnet2.yaml`\n\n2. Test the model\n\n```bash\npython test.py\n```\n\n## Interaction\n\n1. Run `cd DITH-igibson \u0026\u0026 conda activate DITH-igibson`\n2. Interact with the scene and save the results\n\n```bash\npython affordance_prediction_generate.py\n```\n\n3. Collect novel scene observations\n\n```bash\n# generate articulation observation for training\npython object_dataset_generate_train_set.py\n# for testing\npython object_dataset_generate_test_set.py\n```\n\n4. Preprocess for further training\n\n```bash\npython object_dataset_preprocess.py\n```\n\nThe generated data can be found under `dataset/cubicasa5k_objects_processed`.\n\n## Articulation Inference\n\n1. Run `cd DITH-ditto \u0026\u0026 conda activate DITH-ditto`\n2. Set `data_dir` in `configs/config.yaml`.\n\n3. Train the model\n\n```bash\npython run.py experiment=Ditto\n```\n\n4. Set `resume_from_checkpoint` in `configs/experiment/Ditto_test.yaml`\n\n\n5. Test the model\n\n```bash\npython run_test.py experiment=Ditto_test\n```\n\n## Related Repositories\n\n1. The codebase is based on the amazing [Lightning-Hydra-Template](https://github.com/ashleve/lightning-hydra-template).\n\n2. We use [Ditto](https://github.com/UT-Austin-RPL/Ditto) and [PointNet++](https://github.com/erikwijmans/Pointnet2_PyTorch/tree/master) as our backbone.\n\n## Citing\n\n```\n@inproceedings{Hsu2023DittoITH,\n  title={Ditto in the House: Building Articulation Models of Indoor Scenes through Interactive Perception},\n  author={Cheng-Chun Hsu and Zhenyu Jiang and Yuke Zhu},\n  booktitle={IEEE International Conference on Robotics and Automation (ICRA)},\n  year={2023}\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FUT-Austin-RPL%2FHouseDitto","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FUT-Austin-RPL%2FHouseDitto","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FUT-Austin-RPL%2FHouseDitto/lists"}