{"id":28516760,"url":"https://github.com/mit-spark/hydra-gnn","last_synced_at":"2025-07-16T21:08:41.924Z","repository":{"id":277584478,"uuid":"820572406","full_name":"MIT-SPARK/Hydra-GNN","owner":"MIT-SPARK","description":"Room classification network training and inference code","archived":false,"fork":false,"pushed_at":"2024-06-26T18:34:25.000Z","size":9021,"stargazers_count":14,"open_issues_count":2,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-07-05T23:35:24.784Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MIT-SPARK.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-26T18:34:00.000Z","updated_at":"2025-07-01T18:09:25.000Z","dependencies_parsed_at":"2025-02-14T18:48:53.692Z","dependency_job_id":null,"html_url":"https://github.com/MIT-SPARK/Hydra-GNN","commit_stats":null,"previous_names":["mit-spark/hydra-gnn"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/MIT-SPARK/Hydra-GNN","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-SPARK%2FHydra-GNN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-SPARK%2FHydra-GNN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-SPARK%2FHydra-GNN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-SPARK%2FHydra-GNN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MIT-SPARK","download_url":"https://codeload.github.com/MIT-SPARK/Hydra-GNN/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-SPARK%2FHydra-GNN/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265540620,"owners_count":23784945,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-09T04:12:44.174Z","updated_at":"2025-07-16T21:08:41.919Z","avatar_url":"https://github.com/MIT-SPARK.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"This repository contains code to train room-classification networks using 3D scene graphs as input.\nIt is based on the papers:\n  - [\"Neural Trees for Learning on Graphs\"](https://proceedings.neurips.cc/paper/2021/file/ddf88ea64eaed0f3de5531ac964a0a1a-Paper.pdf)\n  - [\"Foundations of Spatial Perception for Robotics: Hierarchical Representations and Real-time Systems\"](https://journals.sagepub.com/doi/10.1177/02783649241229725)\n\nIf you find this code relevant for your work, please consider citing one or both of these papers. Bibtex entries are provided below:\n\n```bibtex\n@inproceedings{talak2021neuraltree,\n               author = {Talak, Rajat and Hu, Siyi and Peng, Lisa and Carlone, Luca},\n               booktitle = {Advances in Neural Information Processing Systems},\n               title = {Neural Trees for Learning on Graphs},\n               year = {2021}\n}\n\n@article{hughes2024foundations,\n    title={Foundations of Spatial Perception for Robotics: Hierarchical Representations and Real-time Systems},\n    fullauthor={Nathan Hughes and Yun Chang and Siyi Hu and Rajat Talak and Rumaisa Abdulhai and Jared Strader and Luca Carlone},\n    author={N. Hughes and Y. Chang and S. Hu and R. Talak and R. Abdulhai and J. Strader and L. Carlone},\n    journal={The International Journal of Robotics Research},\n    doi={10.1177/02783649241229725},\n    url={https://doi.org/10.1177/02783649241229725},\n    year={2024},\n}\n```\n\n## Installation\n\nMake a virtual environment:\n```\n# if you don't have virtualenv already\n# pip3 install --user virtualenv\ncd path/to/env\npython3 -m virtualenv --download -p $(which python3) hydra_gnn\n```\n\nActivate the virtual environment and install:\n```\ncd path/to/installation\ngit clone git@github.com:MIT-SPARK/Hydra-GNN.git\ncd Hydra-GNN\nsource path/to/env/hydra_gnn/bin/activate\npip install -e .\n```\n\nThe training code primarily relies on\n  - [PyTorch](https://pytorch.org/get-started/locally/),\n  - [PyTorch Geometric](https://pytorch-geometric.readthedocs.io/en/latest/install/installation.html)\n  - [pyg_lib](https://github.com/pyg-team/pyg-lib) for Heterogeneous GNN operators\n\nWhile a default install **should** provide everything necessary, you may need to make sure the versions align correctly for these packages and are compatible with your CUDA version. You can either specify the desired versions in `setup.cfg` or manually install these libraries.\n\nThis code has been tested with:\n  - PyTorch 2.0.1, PyTorch Geometric 2.3.1, and Cuda 11.7\n  - PyTorch 1.12.1, PyTorch Geometric 2.2.0, and Cuda 11.3\n  - PyTorch 1.8.1, PyTorch Geometric 2.0.4, and Cuda 10.2\n\n## Dataset Organization\n\nAll datasets and resoruces (such as the pre-trained word2vec model) live in the `./data` folder. It is organized as follows:\n\n- data\n  - [GoogleNews-vectors-negative300.bin](https://www.kaggle.com/datasets/leadbest/googlenewsvectorsnegative300)\n  - Stanford3DSceneGraph\n    - [tiny](https://github.com/StanfordVL/3DSceneGraph)\n  - house_files (can be obtained from the habitat mp3d dataset following the download instructions [here](https://github.com/facebookresearch/habitat-sim/blob/main/DATASETS.md#matterport3d-mp3d-dataset))\n  - mp3d_benchmark\n  - mpcat40.tsv\n  - tro_graphs_2022_09_24\n\nSteps to get started for training:\n1) Obtain the word2vec model from [here](https://www.kaggle.com/datasets/leadbest/googlenewsvectorsnegative300)\n2) Obtain the house files for each mp3d scene from the MP3D dataset and extract them to the house_files directory\n3) Obtain the Hydra-produced scene graphs from [here](https://drive.google.com/drive/folders/1OgQOLYKUg5nRdZnfWQsFspBd7HEV5ZyW?usp=sharing)\n4) (optional) Obtain the Stanford3D tiny split from [here](https://github.com/StanfordVL/3DSceneGraph)\n\n## Training\n\nBefore training, you must construct the relevant pytorch-geometric dataset. For Stanford3D, you can do that via\n```\npython scripts/prepare_Stanford3DSG_for_training.py\n```\nand for MP3D you can do that via\n```\npython scripts/prepare_hydra_mp3d_training.py --repartition_rooms \n```\nTo train with Neural Tree algorithm, you need to decompose the input graphs to H-trees with `--save_htree`. \nFor MP3D, `--repartition_rooms` replaces room nodes from Hydra's room segmentation algorithm by ground truth rooms, \nand establish room-object connectivity using detected object geometry. \nYou can view other possible arguments in both scripts with `--help`.\n\nTraining for a specific dataset can be run via\n```\npython scripts/train_Stanford.py\n```\nor\n```\npython scripts/train_mp3d.py\n```\n\n## Running with Hydra\n\nWe provide pre-trained models [here](https://drive.google.com/drive/folders/1OgQOLYKUg5nRdZnfWQsFspBd7HEV5ZyW?usp=sharing)\n\nFirst, start the GNN model via\n```\n./bin/room_classification_server server path/to/pretrained/model path/to/hydra/label/space\n```\n\nFor the uhumans2 office, this would look like\n```\n./bin/room_classification_server server path/to/pretrained/model path/to/hydra/config/uhumans2/uhumans2_office_typology.yaml\n```\n\nThen, start Hydra with the `use_zmq_interface:=true` argument. For the uhumans2 office scene, this would look like:\n```\nroslaunch hydra_ros uhumans2.launch use_zmq_interface:=true\n```\n\n## Notebooks\n\nThere are several development notebooks available under the directory `notebooks`. These require [jupytext](https://jupytext.readthedocs.io/en/latest/) to view and use.\nYou can do\n```\npip3 install jupytext\n```\nto view and use them.\n\n## Authorship\n\n  - Primary author is Siyi Hu\n\n  - H-tree Construction was written by Rajat Talak\n\n  - Example inference server for Hydra was written by Nathan Hughes\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmit-spark%2Fhydra-gnn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmit-spark%2Fhydra-gnn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmit-spark%2Fhydra-gnn/lists"}