{"id":21714648,"url":"https://github.com/trickygo/pano2room","last_synced_at":"2025-04-12T13:42:05.767Z","repository":{"id":251862698,"uuid":"838666499","full_name":"TrickyGo/Pano2Room","owner":"TrickyGo","description":"Pytorch implementation of Pano2Room (SIGGRAPH Asia 2024)","archived":false,"fork":false,"pushed_at":"2024-08-23T04:35:59.000Z","size":16841,"stargazers_count":100,"open_issues_count":10,"forks_count":7,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-26T08:22:26.102Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TrickyGo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-06T05:40:03.000Z","updated_at":"2025-03-13T08:43:24.000Z","dependencies_parsed_at":"2025-01-15T01:19:51.007Z","dependency_job_id":"ca4ff1cd-ac67-4bf0-88bc-24a55aa7fceb","html_url":"https://github.com/TrickyGo/Pano2Room","commit_stats":null,"previous_names":["trickygo/pano2room"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TrickyGo%2FPano2Room","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TrickyGo%2FPano2Room/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TrickyGo%2FPano2Room/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TrickyGo%2FPano2Room/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TrickyGo","download_url":"https://codeload.github.com/TrickyGo/Pano2Room/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248574551,"owners_count":21127036,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-26T00:37:16.419Z","updated_at":"2025-04-12T13:42:05.747Z","avatar_url":"https://github.com/TrickyGo.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Welcome to Pano2Room!\n\n[Pano2Room: Novel View Synthesis from a Single Indoor Panorama (SIGGRAPH Asia 2024)](https://arxiv.org/abs/2408.11413).\n\n## Overview\n#### In short, Pano2Room converts an input panorama into 3DGS. \n\u003cimg src=\"demo/Teaser.png\" width=\"100%\" \u003e\n\u003cimg src=\"demo/Overview.png\" width=\"100%\" \u003e\n\n\n\n## Demo\nIn this demo, specify input panorama as:\n\u003cimg src='demo/input_panorama.png' width=\"100%\" \u003e\n\nThen, Pano2Room generates the corresponding 3DGS and renders novel views:\n\n\u003cimg src=\"demo/GS_render_video.gif\" width=\"40%\" \u003e\n\nAnd the corresponding rendered depth:\n\n\u003cimg src=\"demo/GS_depth_video.gif\" width=\"40%\" \u003e\n\n## Quick Run\n### 0. Setup the environment\n(1) Create a new conda environment and install [Pytorch3D](https://github.com/facebookresearch/pytorch3d) (for mesh rendering) and [diff-gaussian-rasterization-w-depth](https://github.com/JonathonLuiten/diff-gaussian-rasterization-w-depth) (for 3DGS rendering with depth) accordingly. Other requirements are specified in \\\u003crequirements.txt\\\u003e. \n\n(2) Download pretrained weights in \\\u003ccheckpoints\\\u003e (for image inpainting and depth estimation). See \\\u003ccheckpoints/README.md\\\u003e for instructions.\n\n### 1. Run Demo\n```\nsh scripts/run_Pano2Room.sh\n```\nThis demo converts \\\u003cinput/input_panorama.png\\\u003e to 3DGS and renders novel views as in \\\u003coutput/Pano2Room-results\\\u003e.\n\n### (Optional) 0.5. Fine-tune Inpainter (SDFT)\n\nBefore running step 1., you can also fine-tune SD Inpainter model for better inpainting performance for a specific panorama. To do this:\n\n(1) Create self-supervised training pairs:\n```\nsh scripts/create_SDFT_pairs.sh\n```\nThe pairs are then stored at \\\u003coutput/SDFT_pseudo_pairs\\\u003e.\n\n(2) Training:\n```\nsh scripts/train_SDFT.sh\n```\nThe SDFT weights are then stored at \\\u003coutput/SDFT_weights\\\u003e.\n\nThen by running step 1., the SDFT weights will be automatically loaded.\n\nNotice this step needs to be performed for each new panorama. If you don't want to train SDFT for a new panorama, delete previous \\\u003coutput/SDFT_weights\\\u003e if exists.\n\n## Try on your own panorama\n\nSimply replace \\\u003cinput/input_panorama.png\\\u003e with your own panorama and run the previous steps!\n\n#### Camera Trajectory\nWe provide a camera trajectory at \\\u003cinput/Camera_Trajectory\\\u003e as in the above demo. Each file consists of [R|T] 4*4 matrix of a frame. Feel free to use more camera trajectories.\n\n\n## Cite our paper\n\nIf you find our work helpful, please cite our paper. Thank you!\n\nACM Reference Format:\n```\nGuo Pu, Yiming Zhao, and Zhouhui Lian. 2024. Pano2Room: Novel View Synthesis from a Single Indoor Panorama. In SIGGRAPH Asia 2024 Conference Papers (SA Conference Papers '24), December 3--6, 2024, Tokyo, Japan. ACM, New York, NY, USA, 10 pages.\nhttps://doi.org/10.1145/3680528.3687616\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftrickygo%2Fpano2room","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftrickygo%2Fpano2room","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftrickygo%2Fpano2room/lists"}