{"id":21269136,"url":"https://github.com/yanx27/clevr3d","last_synced_at":"2025-07-20T04:33:58.266Z","repository":{"id":168494935,"uuid":"643782635","full_name":"yanx27/CLEVR3D","owner":"yanx27","description":"CLEVR3D Dataset: Comprehensive Visual Question Answering on Point Clouds through Compositional Scene Manipulation","archived":false,"fork":false,"pushed_at":"2024-02-02T08:20:03.000Z","size":5472,"stargazers_count":17,"open_issues_count":4,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-06T00:05:05.410Z","etag":null,"topics":["point-cloud","scene-graph","scene-understanding","vqa-3d","vqa-dataset"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yanx27.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-05-22T06:36:30.000Z","updated_at":"2025-03-03T10:02:21.000Z","dependencies_parsed_at":null,"dependency_job_id":"c1c919aa-f408-4392-9dcd-47bab8c15dba","html_url":"https://github.com/yanx27/CLEVR3D","commit_stats":{"total_commits":13,"total_committers":3,"mean_commits":4.333333333333333,"dds":"0.23076923076923073","last_synced_commit":"6db9c03f7a3bed15a440ed3f68312a49f70b4cf2"},"previous_names":["yanx27/clevr3d"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/yanx27/CLEVR3D","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yanx27%2FCLEVR3D","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yanx27%2FCLEVR3D/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yanx27%2FCLEVR3D/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yanx27%2FCLEVR3D/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yanx27","download_url":"https://codeload.github.com/yanx27/CLEVR3D/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yanx27%2FCLEVR3D/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266067319,"owners_count":23871332,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["point-cloud","scene-graph","scene-understanding","vqa-3d","vqa-dataset"],"created_at":"2024-11-21T08:07:28.163Z","updated_at":"2025-07-20T04:33:58.216Z","avatar_url":"https://github.com/yanx27.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CLEVR3D\n\n**Xu Yan***, **Zhihao Yuan***, Yuhao Du, Yinghong Liao, Yao Guo, Shuguang Cui, and Zhen Li \n\"Comprehensive Visual Question Answering on Point Clouds through Compositional Scene Manipulation\n\" [[arxiv]](https://arxiv.org/pdf/2112.11691.pdf).\n\n\u003e Our paper is accepted by TVCG (IEEE Transactions on Visualization and Computer Graphics)\n\n ![image](img/fig1.png)\n \n \nIf you find our work useful in your research, please consider citing:\n```latex\n@article{yan2023comprehensive,\n  title={Comprehensive Visual Question Answering on Point Clouds through Compositional Scene Manipulation},\n  author={Yan, Xu and Yuan, Zhihao and Du, Yuhao and Liao, Yinghong and Guo, Yao and Cui, Shuguang and Li, Zhen},\n  journal={IEEE Transactions on Visualization \\\u0026 Computer Graphics},\n  number={01},\n  pages={1--13},\n  year={2023},\n  publisher={IEEE Computer Society}\n}\n```\n\n\n## Installation\n\n### Requirements\n- pytorch \u003e= 1.8 \n- transformers\n- [PyTorch Lightning](https://lightning.ai/docs/pytorch/stable/)\n\n## Data Preparation\nThe VQA3D data can be found in `data/CLEVR3D/CLEVR3D-REAL.json`. The data has the following structure:\n```\n{\n\"question\":[\n{\n    \"scan\": \"f62fd5fd-9a3f-2f44-883a-1e5cf819608e\",\n    \"image_index\": 0,\n    \"question\": \"Are there the same number of sofas and wide sinks?\",\n    \"answer\": \"no\",\n    \"template_filename\": \"compare_integer.json\",\n    \"question_family_index\": 0,\n    \"question_type\": \"equal_integer\"\n},\n...\n]}\n```\nThe scan number is the same as [3RScan](https://github.com/WaldJohannaU/3RScan).\nPlease download the preprocessed 3RScan data from [Baidu Netdisk](https://pan.baidu.com/s/1q-K79cEeHzUaBJ1ZjkNxvw) (**ifei**). And modify the data path in `lib/config.py`.\n\n## Training\n```shell\ncd \u003croot dir of this repo\u003e\npython main.py --log_dir {LOGNAME} --use_scene_graph --preloading\n```\n\n\n## Evaluation\nYou cna download our weights from [OneDrive](https://cuhko365-my.sharepoint.com/:u:/g/personal/221019046_link_cuhk_edu_cn/EUZZSwJPTD9Btep3Z2lYa10BqxXJ4ecJydWa_pX5YQk9DQ?e=SkznPm)\n```shell\npython main.py --test --ckpt_path \u003cdir for the pytorch checkpoint\u003e --use_scene_graph --preloading\n```\n\n## Question Generation\n\nThe dataset is semi-automatic generated, where an initiating dataset is generated automatically, and some manual modification is applied.\n\nAll the files needed for question generation is in the directory of ```data_generation```.\n\nWe will generate questions, functional programs, and answers for the scenes. This step takes as input the single JSON file ``` 3dssg_scenes.json``` containing all ground-truth scene information and outputs a JSON file ``` questions.json``` containing questions, answers, and functional programs for the questions.\n\nYou can generate initiating questions like this:\n\n```\ncd question_generation\npython generate_questions.py\n```\n\nBy default, ``` generate_questions.py``` will generate questions for all scenes in the input file. However, you can generate questions by using other flags like ```--scene_start_idx```.\n\nYou can find more details about question generation [here](data_generation/README.md).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyanx27%2Fclevr3d","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyanx27%2Fclevr3d","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyanx27%2Fclevr3d/lists"}