{"id":13535564,"url":"https://github.com/Uehwan/3-D-Scene-Graph","last_synced_at":"2025-04-02T01:31:15.564Z","repository":{"id":117474221,"uuid":"156166243","full_name":"Uehwan/3-D-Scene-Graph","owner":"Uehwan","description":"3D scene graph generator implemented in Pytorch.","archived":false,"fork":false,"pushed_at":"2019-08-11T09:35:22.000Z","size":73790,"stargazers_count":76,"open_issues_count":8,"forks_count":11,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-08-11T17:08:20.416Z","etag":null,"topics":["3d-models","3d-scene-graph","computer-vision","deep-learning","deeplearning","environmental-modelling","intelligence","pytorch","robot","robotics","scene-graph","scenegraph"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Uehwan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-11-05T05:40:04.000Z","updated_at":"2024-08-06T04:46:07.000Z","dependencies_parsed_at":null,"dependency_job_id":"961082cb-5de0-4ff3-858c-41838e79e561","html_url":"https://github.com/Uehwan/3-D-Scene-Graph","commit_stats":{"total_commits":102,"total_committers":4,"mean_commits":25.5,"dds":0.3137254901960784,"last_synced_commit":"0deb403f77f43c48fd730091fb8cd065a3dc0a59"},"previous_names":["uehwan/3d-scene-graph"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Uehwan%2F3-D-Scene-Graph","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Uehwan%2F3-D-Scene-Graph/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Uehwan%2F3-D-Scene-Graph/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Uehwan%2F3-D-Scene-Graph/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Uehwan","download_url":"https://codeload.github.com/Uehwan/3-D-Scene-Graph/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":222788514,"owners_count":17037777,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-models","3d-scene-graph","computer-vision","deep-learning","deeplearning","environmental-modelling","intelligence","pytorch","robot","robotics","scene-graph","scenegraph"],"created_at":"2024-08-01T08:00:58.479Z","updated_at":"2024-11-02T23:31:35.466Z","avatar_url":"https://github.com/Uehwan.png","language":"Python","readme":"# 3D-Scene-Graph: *A Sparse and Semantic Representation of Physical Environments for Intelligent Agents*\nThis work is based on our paper (IEEE Transactions on Cybernetics 2019, accepted). We proposed a new concept called 3D scene graph and its construction framework. Our work is based on [FactorizableNet](https://github.com/yikang-li/FactorizableNet), implemented in Pytorch.\n\n## 3D Scene Graph Construction Framework\nThe  proposed  3D  scene  graph  construction  framework  extracts  relevant  semantics  within  environments such  as  object categories  and  relations  between  objects  as  well  as  physical attributes \nsuch as 3D positions and major colors in the process of generating 3D scene graphs for the given environments. The framework receives a sequence of observations regarding the environments in the form of RGB-D image frames. \nFor robust performance,  the  framework  filters  out  unstable  observations(i.e., blurry images) \nusing the proposed adaptive blurry image detection algorithm. \nThen, the framework factors out keyframe groups  to  avoid  redundant  processing  of  the  same  information. \nKeyframe groups contain reasonably-overlapping frames. Next, the framework extracts semantics and physical attributes \nwithin the environments through recognition modules. \nDuring the  recognition  processes,  spurious  detections  get  rejected and  missing  entities  are  supplemented.  \nFinally,  the  gathered information gets fused into 3D scene graph and the graph gets updated upon new observations.\n\n\n\u003cimg src=\"vis_result/fig/3dsg_readme_fig.png\" width=\"600\"\u003e\n\n## Requirements\n* Ubuntu 16.04+\n* Python 2.7\n* [Pytorch 0.3.1](https://pytorch.org/get-started/previous-versions/) \n* torchtext 0.2.3\n* [FactorizableNet](https://github.com/yikang-li/FactorizableNet)\n* [ScanNet](http://www.scan-net.org) dataset (optional, used for test. an RGBD video from ScanNet is enough.)\n\n## Installation\n\nInstall [Pytorch 0.3.1](https://pytorch.org/get-started/previous-versions/). The code has been tested only with Python 2.7, CUDA 9.0 on Ubuntu 16.04. \nYou will need to modify a significant amount of code if you want to run in a different environment (Python 3+ or Pytorch 0.4+).\n\n\n1. Download 3D-Scene-Graph repository \n\n```\ngit clone --recurse-submodules https://github.com/Uehwan/3D-Scene-Graph.git\n```\n2. Install FactorizableNet\n```\ncd 3D-Scene-Graph/FactorizableNet\n```\nPlease follow the installation instructions in [FactorizableNet](https://github.com/yikang-li/FactorizableNet) repository.\nFollow steps 1 through 6. You can skip step 7. Download VG-DR-Net in step 8. You do not need to download other models.\n\n3. Install 3D-Scene-Graph\n```\ncd 3D-Scene-Graph\ntouch FactorizableNet/__init__.py\nln -s ./FactorizableNet/options/ options\nmkdir data\nln -s ./FactorizableNet/data/svg data/svg\nln -s ./FactorizableNet/data/visual_genome data/visual_genome\n   \npip install torchtext==0.2.3\npip install setuptools pyyaml graphviz webcolors pandas matplotlib \npip install git+https://github.com/chickenbestlover/ColorHistogram.git\n```\n\nAn Alternative: use installation script\n```\n   ./build.sh\n```\n\n4. Download [ScanNet](http://www.scan-net.org) dataset\n\nIn order to use ScanNet dataset, you need to fill out an agreement to toe ScanNet Terms of Use and send it to the ScanNet team at scannet@googlegroups.com.\nIf the process was successful, they will send you a script downloading ScanNet dataset.\n\nTo download a specific scan (e.g. `scene0000_00`) using the script (the script only runs on Python 2.7):\n```\ndownload-scannet.py -o [directory in which to download] --id scene0000_00\n(then press Enter twice)\n```\nAfter the download is finished, the scan is located in a new folder `scene0000_00`.\nIn the folder, `*.sens` file contains the RGBD Video with camera pose. \nTo extract them, we use SensReader, an extraction tool provided by [ScanNet git repo](https://github.com/ScanNet/ScanNet).\n\n```\ngit clone https://github.com/ScanNet/ScanNet.git\ncd ScanNet/SensReader/python/\npython reader.py \\\n   --filename [your .sens filepath]  \\\n   --output_path [ROOT of 3D-Scene-Graph]/data/scene0000_00/ \\\n   --export_depth_images \\\n   --export_color_images \\\n   --export_poses \\\n   --export_intrinsics\n    \n```\n\n\n## Example of usage\n\n```\npython scene_graph_tuning.py \\\n  --scannet_path data/scene0000_00/\\\n  --obj_thres 0.23\\\n  --thres_key 0.2\\\n  --thres_anchor 0.68 \\\n  --visualize \\\n  --frame_start 800 \\\n  --plot_graph \\\n  --disable_spurious \\\n  --gain 10 \\\n  --detect_cnt_thres 2 \\\n  --triplet_thres 0.065\n```\n\n### Core hyper-parameters #\n\nData settings:\n* *--dataset \u003cdataset\u003e* : choose dataset, default='scannet'.\n* *--scannet_path \u003cscannetpath\u003e* : scannet scan filepath , default='./data/scene0507/'.\n* *--frame_start \u003cscannetpath\u003e* : idx of frame to start , default=0.\n* *--frame_end \u003cscannetpath\u003e* : idx of frame to end , default=5000.\n\nFactorizableNet Output Filtering Settings:\n* *--obj_thres \u003cobjthres\u003e* : object recognition threshold score , default=0.25.\n* *--triplet_thres \u003ctrithres\u003e* : triplet recognition threshold score , default=0.08.\n* *--nms \u003ctrithres\u003e* : NMS threshold for post object NMS (negative means not NMS) , default=0.2.\n* *--triplet_nms \u003ctrithres\u003e* : Triplet NMS threshold for post object NMS (negative means not NMS) , default=0.4.\n\n\nKey-frame Extraction Settings:\n* *--thres_key \u003ckey\u003e* : keyframe threshold score , default=0.1.\n* *--thres_anchor \u003canchor\u003e* : achorframe threshold score , default=0.65.\n* *--alpha \u003canchor\u003e* : weight for Exponentially Weighted Summation , default=0.4.\n* *--gain \u003canchor\u003e* : gain for adaptive thresholding in blurry image detection , default=25.\n* *--offset \u003canchor\u003e* : offset for adaptive thresholding in blurry image detection , default=1.\n\nVisualization Settings:\n* *--pause_time \u003ckey\u003e* : a pause interval (sec) for every detection , default=1.\n* *--plot_graph \u003ckey\u003e* : plot 3D Scene Graph if true.\n* *--visualize \u003ckey\u003e* : enable visualization if ture.\n* *--format \u003ckey\u003e* : resulting image format, pdf or png, default='png'.\n* *--draw_color \u003ckey\u003e* : draw color node in 3D scene graph if true.\n* *--save_image \u003ckey\u003e* : save detection result image if true.\n\n\n\n\n## Result\n\n![scores1](./vis_result/fig/result.png)\n\n\n## Demo Video\n\n[![Video Label](http://img.youtube.com/vi/RF4bf7ZlAX4/0.jpg)](https://www.youtube.com/watch?v=RF4bf7ZlAX4)\n\n## Citations\n\nPlease consider citing this project in your publications if it helps your research. \nThe following is a BibTeX reference. \n\n```\n@article{kim2019graph3d,\n  title={3D-Scene-Graph: A Sparse and Semantic Representation of Physical Environments for Intelligent Agents},\n  author={Kim, Ue-Hwan and Park, Jin-Man and Song, Taek-Jin and Kim, Jong-Hwan},\n  journal={IEEE Cybernetics},\n  year={2019}\n}\n```\n\n## Acknowledgement\nThis work was supported by the ICT R\u0026D program\nof MSIP/IITP. [2016-0-00563, Research on Adaptive Machine\nLearning Technology Development for Intelligent Autonomous\nDigital Companion]\n","funding_links":[],"categories":["Scene Graph Generation:"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FUehwan%2F3-D-Scene-Graph","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FUehwan%2F3-D-Scene-Graph","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FUehwan%2F3-D-Scene-Graph/lists"}