{"id":13586240,"url":"https://github.com/shunsukesaito/PIFu","last_synced_at":"2025-04-07T14:34:17.135Z","repository":{"id":37631212,"uuid":"186289660","full_name":"shunsukesaito/PIFu","owner":"shunsukesaito","description":"This repository contains the code for the paper \"PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization\"","archived":false,"fork":false,"pushed_at":"2023-11-24T17:55:31.000Z","size":2495,"stargazers_count":1767,"open_issues_count":57,"forks_count":342,"subscribers_count":58,"default_branch":"master","last_synced_at":"2024-10-29T17:52:04.630Z","etag":null,"topics":["3d-printing","3d-reconstruction","arvr","computer-graphics","fashion","geometry-processing","human","iccv2019","pifu","pytorch"],"latest_commit_sha":null,"homepage":"https://shunsukesaito.github.io/PIFu/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shunsukesaito.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-05-12T18:24:48.000Z","updated_at":"2024-10-22T05:59:53.000Z","dependencies_parsed_at":"2024-01-07T00:13:19.214Z","dependency_job_id":"e3017561-f450-4a69-86af-6fb9f42f54d3","html_url":"https://github.com/shunsukesaito/PIFu","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shunsukesaito%2FPIFu","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shunsukesaito%2FPIFu/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shunsukesaito%2FPIFu/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shunsukesaito%2FPIFu/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shunsukesaito","download_url":"https://codeload.github.com/shunsukesaito/PIFu/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223101367,"owners_count":17087555,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-printing","3d-reconstruction","arvr","computer-graphics","fashion","geometry-processing","human","iccv2019","pifu","pytorch"],"created_at":"2024-08-01T15:05:25.312Z","updated_at":"2024-11-06T04:31:25.009Z","avatar_url":"https://github.com/shunsukesaito.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization\n\n[![report](https://img.shields.io/badge/arxiv-report-red)](https://arxiv.org/abs/1905.05172) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1GFSsqP2BWz4gtq0e-nki00ZHSirXwFyY)\n\nNews:\n* \\[2020/05/04\\] Added EGL rendering option for training data generation. Now you can create your own training data with headless machines!\n* \\[2020/04/13\\] Demo with Google Colab (incl. visualization) is available. Special thanks to [@nanopoteto](https://github.com/nanopoteto)!!!\n* \\[2020/02/26\\] License is updated to MIT license! Enjoy!\n\nThis repository contains a pytorch implementation of \"[PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://arxiv.org/abs/1905.05172)\".\n\n[Project Page](https://shunsukesaito.github.io/PIFu/)\n![Teaser Image](https://shunsukesaito.github.io/PIFu/resources/images/teaser.png)\n\nIf you find the code useful in your research, please consider citing the paper.\n\n```\n@InProceedings{saito2019pifu,\nauthor = {Saito, Shunsuke and Huang, Zeng and Natsume, Ryota and Morishima, Shigeo and Kanazawa, Angjoo and Li, Hao},\ntitle = {PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization},\nbooktitle = {The IEEE International Conference on Computer Vision (ICCV)},\nmonth = {October},\nyear = {2019}\n}\n```\n\n\nThis codebase provides: \n- test code\n- training code\n- data generation code\n\n## Requirements\n- Python 3\n- [PyTorch](https://pytorch.org/) tested on 1.4.0\n- json\n- PIL\n- skimage\n- tqdm\n- numpy\n- cv2\n\nfor training and data generation\n- [trimesh](https://trimsh.org/) with [pyembree](https://github.com/scopatz/pyembree)\n- [pyexr](https://github.com/tvogels/pyexr)\n- PyOpenGL\n- freeglut (use `sudo apt-get install freeglut3-dev` for ubuntu users)\n- (optional) egl related packages for rendering with headless machines. (use `apt install libgl1-mesa-dri libegl1-mesa libgbm1` for ubuntu users)\n\nWarning: I found that outdated NVIDIA drivers may cause errors with EGL. If you want to try out the EGL version, please update your NVIDIA driver to the latest!!\n\n## Windows demo installation instuction\n\n- Install [miniconda](https://docs.conda.io/en/latest/miniconda.html)\n- Add `conda` to PATH\n- Install [git bash](https://git-scm.com/downloads)\n- Launch `Git\\bin\\bash.exe`\n- `eval \"$(conda shell.bash hook)\"` then `conda activate my_env` because of [this](https://github.com/conda/conda-build/issues/3371)\n- Automatic `env create -f environment.yml` (look [this](https://github.com/conda/conda/issues/3417))\n- OR manually setup [environment](https://towardsdatascience.com/a-guide-to-conda-environments-bc6180fc533)\n    - `conda create —name pifu python` where `pifu` is name of your environment\n    - `conda activate`\n    - `conda install pytorch torchvision cudatoolkit=10.1 -c pytorch`\n    - `conda install pillow`\n    - `conda install scikit-image`\n    - `conda install tqdm`\n    - `conda install -c menpo opencv`\n- Download [wget.exe](https://eternallybored.org/misc/wget/)\n- Place it into `Git\\mingw64\\bin`\n- `sh ./scripts/download_trained_model.sh`\n- Remove background from your image ([this](https://www.remove.bg/), for example)\n- Create black-white mask .png\n- Replace original from sample_images/\n- Try it out - `sh ./scripts/test.sh`\n- Download [Meshlab](http://www.meshlab.net/) because of [this](https://github.com/shunsukesaito/PIFu/issues/1)\n- Open .obj file in Meshlab\n\n\n## Demo\nWarning: The released model is trained with mostly upright standing scans with weak perspectie projection and the pitch angle of 0 degree. Reconstruction quality may degrade for images highly deviated from trainining data.\n1. run the following script to download the pretrained models from the following link and copy them under `./PIFu/checkpoints/`.\n```\nsh ./scripts/download_trained_model.sh\n```\n\n2. run the following script. the script creates a textured `.obj` file under `./PIFu/eval_results/`. You may need to use `./apps/crop_img.py` to roughly align an input image and the corresponding mask to the training data for better performance. For background removal, you can use any off-the-shelf tools such as [removebg](https://www.remove.bg/).\n```\nsh ./scripts/test.sh\n```\n\n## Demo on Google Colab\nIf you do not have a setup to run PIFu, we offer Google Colab version to give it a try, allowing you to run PIFu in the cloud, free of charge. Try our Colab demo using the following notebook: \n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1GFSsqP2BWz4gtq0e-nki00ZHSirXwFyY)\n\n## Data Generation (Linux Only)\nWhile we are unable to release the full training data due to the restriction of commertial scans, we provide rendering code using free models in [RenderPeople](https://renderpeople.com/free-3d-people/).\nThis tutorial uses `rp_dennis_posed_004` model. Please download the model from [this link](https://renderpeople.com/sample/free/rp_dennis_posed_004_OBJ.zip) and unzip the content under a folder named `rp_dennis_posed_004_OBJ`. The same process can be applied to other RenderPeople data.\n\nWarning: the following code becomes extremely slow without [pyembree](https://github.com/scopatz/pyembree). Please make sure you install pyembree.\n\n1. run the following script to compute spherical harmonics coefficients for [precomputed radiance transfer (PRT)](https://sites.fas.harvard.edu/~cs278/papers/prt.pdf). In a nutshell, PRT is used to account for accurate light transport including ambient occlusion without compromising online rendering time, which significantly improves the photorealism compared with [a common sperical harmonics rendering using surface normals](https://cseweb.ucsd.edu/~ravir/papers/envmap/envmap.pdf). This process has to be done once for each obj file.\n```\npython -m apps.prt_util -i {path_to_rp_dennis_posed_004_OBJ}\n```\n\n2. run the following script. Under the specified data path, the code creates folders named `GEO`, `RENDER`, `MASK`, `PARAM`, `UV_RENDER`, `UV_MASK`, `UV_NORMAL`, and `UV_POS`. Note that you may need to list validation subjects to exclude from training in `{path_to_training_data}/val.txt` (this tutorial has only one subject and leave it empty). If you wish to render images with headless servers equipped with NVIDIA GPU, add -e to enable EGL rendering.\n```\npython -m apps.render_data -i {path_to_rp_dennis_posed_004_OBJ} -o {path_to_training_data} [-e]\n```\n\n## Training (Linux Only)\n\nWarning: the following code becomes extremely slow without [pyembree](https://github.com/scopatz/pyembree). Please make sure you install pyembree.\n\n1. run the following script to train the shape module. The intermediate results and checkpoints are saved under `./results` and `./checkpoints` respectively. You can add `--batch_size` and `--num_sample_input` flags to adjust the batch size and the number of sampled points based on available GPU memory.\n```\npython -m apps.train_shape --dataroot {path_to_training_data} --random_flip --random_scale --random_trans\n```\n\n2. run the following script to train the color module. \n```\npython -m apps.train_color --dataroot {path_to_training_data} --num_sample_inout 0 --num_sample_color 5000 --sigma 0.1 --random_flip --random_scale --random_trans\n```\n\n## Related Research\n**[Monocular Real-Time Volumetric Performance Capture (ECCV 2020)](https://project-splinter.github.io/)**  \n*Ruilong Li\\*, Yuliang Xiu\\*, Shunsuke Saito, Zeng Huang, Kyle Olszewski, Hao Li*\n\nThe first real-time PIFu by accelerating reconstruction and rendering!!\n\n**[PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization (CVPR 2020)](https://shunsukesaito.github.io/PIFuHD/)**  \n*Shunsuke Saito, Tomas Simon, Jason Saragih, Hanbyul Joo*\n\nWe further improve the quality of reconstruction by leveraging multi-level approach!\n\n**[ARCH: Animatable Reconstruction of Clothed Humans (CVPR 2020)](https://arxiv.org/pdf/2004.04572.pdf)**  \n*Zeng Huang, Yuanlu Xu, Christoph Lassner, Hao Li, Tony Tung*\n\nLearning PIFu in canonical space for animatable avatar generation!\n\n**[Robust 3D Self-portraits in Seconds (CVPR 2020)](http://www.liuyebin.com/portrait/portrait.html)**  \n*Zhe Li, Tao Yu, Chuanyu Pan, Zerong Zheng, Yebin Liu*\n\nThey extend PIFu to RGBD + introduce \"PIFusion\" utilizing PIFu reconstruction for non-rigid fusion.\n\n**[Learning to Infer Implicit Surfaces without 3d Supervision (NeurIPS 2019)](http://papers.nips.cc/paper/9039-learning-to-infer-implicit-surfaces-without-3d-supervision.pdf)**  \n*Shichen Liu, Shunsuke Saito, Weikai Chen, Hao Li*\n\nWe answer to the question of \"how can we learn implicit function if we don't have 3D ground truth?\"\n\n**[SiCloPe: Silhouette-Based Clothed People (CVPR 2019, best paper finalist)](https://arxiv.org/pdf/1901.00049.pdf)**  \n*Ryota Natsume\\*, Shunsuke Saito\\*, Zeng Huang, Weikai Chen, Chongyang Ma, Hao Li, Shigeo Morishima*\n\nOur first attempt to reconstruct 3D clothed human body with texture from a single image!\n\n**[Deep Volumetric Video from Very Sparse Multi-view Performance Capture (ECCV 2018)](http://openaccess.thecvf.com/content_ECCV_2018/papers/Zeng_Huang_Deep_Volumetric_Video_ECCV_2018_paper.pdf)**  \n*Zeng Huang, Tianye Li, Weikai Chen, Yajie Zhao, Jun Xing, Chloe LeGendre, Linjie Luo, Chongyang Ma, Hao Li*\n\nImplict surface learning for sparse view human performance capture!\n\n------\n\n\n\nFor commercial queries, please contact:\n\nHao Li: hao@hao-li.com ccto: saitos@usc.edu Baker!!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshunsukesaito%2FPIFu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshunsukesaito%2FPIFu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshunsukesaito%2FPIFu/lists"}