{"id":21714658,"url":"https://github.com/trickygo/sinmpi","last_synced_at":"2025-06-16T20:37:42.792Z","repository":{"id":186238919,"uuid":"674870455","full_name":"TrickyGo/SinMPI","owner":"TrickyGo","description":"Pytorch implementation of SinMPI (SIGGRAPH Asia 2023)","archived":false,"fork":false,"pushed_at":"2024-08-23T03:05:47.000Z","size":24520,"stargazers_count":56,"open_issues_count":2,"forks_count":4,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-05-30T12:56:12.453Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TrickyGo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-08-05T02:53:28.000Z","updated_at":"2025-05-17T16:32:16.000Z","dependencies_parsed_at":null,"dependency_job_id":"0d64e1fa-37dd-4c7e-932c-4f5cbcf97e02","html_url":"https://github.com/TrickyGo/SinMPI","commit_stats":null,"previous_names":["trickygo/sinmpi"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/TrickyGo/SinMPI","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TrickyGo%2FSinMPI","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TrickyGo%2FSinMPI/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TrickyGo%2FSinMPI/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TrickyGo%2FSinMPI/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TrickyGo","download_url":"https://codeload.github.com/TrickyGo/SinMPI/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TrickyGo%2FSinMPI/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260233869,"owners_count":22978890,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-26T00:37:41.035Z","updated_at":"2025-06-16T20:37:42.738Z","avatar_url":"https://github.com/TrickyGo.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"### Welcome to SinMPI! \n\n[\"SinMPI: Novel View Synthesis from a Single Image with Expanded Multiplane Images\" (SIGGRAPH Asia 2023)](https://arxiv.org/abs/2312.11037).\n\n## Quick demo\n\n### 1. Prepare\n\n(1) Create a new conda environment specified in requirements.txt.\n\n(2) Download pretrained weights of depth-aware inpainter [ecweights](https://drive.google.com/drive/folders/1FZZ6laPuqEMSfrGvEWYaDZWEPaHvGm6r) and put them into 'warpback/ecweights/xxx.pth'.\n\n### 2. Run demo\n```\nsh scripts/train_all.sh\n```\nThis demo converts 'test_images/Syndney.jpg' to an expanded MPI and renders novel views as in 'ckpts/Exp-Syndney-new/MPI_rendered_views.mp4'.\n\n## What happens when running the demo?\n\n### 1. Outpaint the input image\n\nIn the above demo, we specify 'test_images/Syndney.jpg'\n\n\u003cimg src='test_images/Syndney.jpg' width=\"30%\" \u003e\n\nas the input image, then we continuously outpaint the input image:\n```\nCUDA_VISIBLE_DEVICES=$cuda python outpaint_rgbd.py \\\n    --width $width \\\n    --height $height \\\n    --ckpt_path $ckpt_path \\\n    --img_path $img_path \\\n    --extrapolate_times $extrapolate_times\n```\nThen we get the outpainted image and its depth estimated by a monocular depth estimator (DPT):\n\n\u003cimg src='ckpts/Exp-syndney/canvas.png' width=\"30%\" \u003e \u003cimg src='ckpts/Exp-syndney/canvas_depth.png' width=\"30%\" \u003e\n\n### 2. Finetune Depth-aware Inpainter and create Pseudo-multi-view images\n\n```\nCUDA_VISIBLE_DEVICES=$cuda python train_inpainting.py \\\n     --width $width \\\n     --height $height \\\n     --ckpt_path $ckpt_path \\\n     --img_path $img_path \\\n     --num_epochs 10  \\\n     --extrapolate_times $extrapolate_times \\\n     --batch_size 1  #--load_warp_pairs --debugging \n```\n\n### 3. Optimizing the expanded MPI\n\n```\nCUDA_VISIBLE_DEVICES=$cuda python train_mpi.py \\\n    --width $width \\\n    --height $height \\\n    --ckpt_path $ckpt_path \\\n    --img_path $img_path \\\n    --num_epochs 10 \\\n    --extrapolate_times $extrapolate_times \\\n    --batch_size 1 #--debugging #--resume\n```\n\nAfter optimization, we render novel views:\n\n\u003cimg src=\"ckpts/Exp-syndney/MPI_rendered_views.gif\" width=\"30%\" \u003e\n\n## Toward Better quality and robustness\n\nNotice the above demo is designed for fast illustration (FPS is low). For better quality: \n\n#### Pesudo-multi-views should be more and cover more areas.\nIncreasing the sample rate and sample areas helps to optimize MPI with better quality. To add training and rendering view trajectories, modify 'dataloaders/single_img_data.py'.\n#### More training epochs are needed. \n\n## Cite our paper\n\nIf you find our work helpful, please cite our paper. Thank you!\n\nACM Reference Format:\n```\nGuo Pu, Peng-Shuai Wang, and Zhouhui Lian. 2023. SinMPI: Novel View\nSynthesis from a Single Image with Expanded Multiplane Images. In SIGGRAPH Asia 2023 Conference Papers (SA Conference Papers '23), December\n12–15, 2023, Sydney, NSW, Australia. ACM, New York, NY, USA, 10 pages.\nhttps://doi.org/10.1145/3610548.3618155\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftrickygo%2Fsinmpi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftrickygo%2Fsinmpi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftrickygo%2Fsinmpi/lists"}