{"id":13778518,"url":"https://github.com/chenhsuanlin/signed-distance-SRN","last_synced_at":"2025-05-11T12:30:55.117Z","repository":{"id":110824395,"uuid":"305614832","full_name":"chenhsuanlin/signed-distance-SRN","owner":"chenhsuanlin","description":"SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Images 🎯 (NeurIPS 2020)","archived":false,"fork":false,"pushed_at":"2020-12-28T16:19:35.000Z","size":8276,"stargazers_count":126,"open_issues_count":1,"forks_count":17,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-08T12:45:11.249Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chenhsuanlin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-10-20T06:44:46.000Z","updated_at":"2025-02-18T13:29:06.000Z","dependencies_parsed_at":null,"dependency_job_id":"2a98b332-c31b-4142-b96c-200cca11ac41","html_url":"https://github.com/chenhsuanlin/signed-distance-SRN","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chenhsuanlin%2Fsigned-distance-SRN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chenhsuanlin%2Fsigned-distance-SRN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chenhsuanlin%2Fsigned-distance-SRN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chenhsuanlin%2Fsigned-distance-SRN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chenhsuanlin","download_url":"https://codeload.github.com/chenhsuanlin/signed-distance-SRN/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253566378,"owners_count":21928628,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T18:00:54.397Z","updated_at":"2025-05-11T12:30:54.161Z","avatar_url":"https://github.com/chenhsuanlin.png","language":"Python","funding_links":[],"categories":["Implicit representations of Geometry and Appearance"],"sub_categories":["From 2D supervision only (“inverse graphics”)"],"readme":"## SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Images\n[Chen-Hsuan Lin](https://chenhsuanlin.bitbucket.io/),\n[Chaoyang Wang](https://mightychaos.github.io/),\nand [Simon Lucey](http://ci2cv.net/people/simon-lucey/)  \nAdvances in Neural Information Processing Systems (NeurIPS), 2020  \n\nProject page: https://chenhsuanlin.bitbucket.io/signed-distance-SRN  \nPaper: https://chenhsuanlin.bitbucket.io/signed-distance-SRN/paper.pdf  \narXiv preprint: https://arxiv.org/abs/2010.10505  \n\n\u003cp align=\"center\"\u003e\u003cimg src=\"https://chenhsuanlin.bitbucket.io/signed-distance-SRN/teaser.png\"\u003e\u003c/p\u003e\n\u003cp align=\"center\"\u003e\u003cimg src=\"images/teaser_mov.gif\"\u003e\u003c/p\u003e\n\nWe provide PyTorch code for both the ShapeNet and PASCAL3D+ experiments.\n\n--------------------------------------\n\n### Prerequisites\n\nThis code is developed with Python3 (`python3`). PyTorch 1.4+ is required.  \nIt is recommended to install the dependencies with `conda` by running\n```bash\nconda env create --file requirements.yaml python=3\n```\nThis creates a conda environment named `sdfsrn-env`. Activate it with\n```bash\nconda activate sdfsrn-env\n```\nYou may want to install with `virtualenv`; however, this repository depends on [VIGRA](http://ukoethe.github.io/vigra/) to compute the distance transforms, which does not seem to be pip installable.\nSome workarounds would include (a) installing VIGRA from source, or (b) replacing the VIGRA distance transform function with `scipy.ndimage.distance_transform_edt` (significantly slower).\n\n--------------------------------------\n\n### Dataset\n\n- #### ShapeNet\n\tDownload the ShapeNet renderings of [Kato et al.](https://arxiv.org/abs/1711.07566) from the [DVR](https://github.com/autonomousvision/differentiable_volumetric_rendering) repository (33GB):  \n\t(this file is huge and takes a long time to fully unzip, so we extract only the 3 categories of interest in this work)\n  ```bash\n  wget https://s3.eu-central-1.amazonaws.com/avg-projects/differentiable_volumetric_rendering/data/NMR_Dataset.zip\n  unzip NMR_Dataset.zip NMR_Dataset/02691156/* # airplane\n  unzip NMR_Dataset.zip NMR_Dataset/02958343/* # car\n  unzip NMR_Dataset.zip NMR_Dataset/03001627/* # chair\n  rm NMR_Dataset.zip\n  ```\n  In the `data/NMR_Dataset` directory, download the post-processed surface point clouds:\n  ```bash\n  wget https://cmu.box.com/shared/static/yvencf3ts8itfgyuh5sap9q7dy5r1elg.gz\n  tar -zxvf yvencf3ts8itfgyuh5sap9q7dy5r1elg.gz\n  rm yvencf3ts8itfgyuh5sap9q7dy5r1elg.gz\n  ```\n  There should be a `pointcloud3.npz` within each shape directory, along with the original `pointcloud.npz`. You can check with\n  ```bash\n  ls NMR_Dataset/02691156/10155655850468db78d106ce0a280f87\n  ```\n  If you're interested in creating ground-truth point clouds for other object categories, please refer to the [README](data/README.md) in `data`.\n- #### PASCAL3D+\n  Download the [PASCAL3D+](https://cvgl.stanford.edu/projects/pascal3d.html) (v1.1) dataset under the `data` directory (7.7GB):\n  ```bash\n  wget ftp://cs.stanford.edu/cs/cvgl/PASCAL3D+_release1.1.zip\n  unzip PASCAL3D+_release1.1.zip\n  rm PASCAL3D+_release1.1.zip\n  ```\n  Also under the `data` directory, download the object masks and ground-truth point clouds for the 3 categories (23MB):\n  ```bash\n  wget https://cmu.box.com/shared/static/uyz0txthw0ufjwury0f3z3iuhqdbaet9.gz\n  tar -zxvf uyz0txthw0ufjwury0f3z3iuhqdbaet9.gz\n  rm uyz0txthw0ufjwury0f3z3iuhqdbaet9.gz\n  ```\n\n--------------------------------------\n\n### Pretrained models\nFirst, create a directory to store the pretrained models:\n```bash\nmkdir -p pretrained\n```\nThen under `pretrained`, download the pretrained model(s) by running the commands\n```bash\n# ShapeNet (trained on multi-view renderings, 615MB each)\nwget https://cmu.box.com/shared/static/cgrzlaudm2ojs5l3nmmbbr7ovsvbqhtv.ckpt -O shapenet_airplane.ckpt  # airplane\nwget https://cmu.box.com/shared/static/lclrhwae5xu6z7f2fc3qnkeon3q5ljfg.ckpt -O shapenet_car.ckpt  # car\nwget https://cmu.box.com/shared/static/58dsppp8hq0yqj216tqm573or9porq2m.ckpt -O shapenet_chair.ckpt  # chair\n# PASCAL3D+ (197MB each)\nwget https://cmu.box.com/shared/static/gvslqtye7p0pzgaspmwvq7pggnmxsu3x.ckpt -O pascal3d_airplane.ckpt # airplane\nwget https://cmu.box.com/shared/static/kh8mrrufol3u1mm6duaym5sygfd42d5p.ckpt -O pascal3d_car.ckpt # car\nwget https://cmu.box.com/shared/static/ty0ywyeud1n1n9uu169xoag9m35me267.ckpt -O pascal3d_chair.ckpt # chair\n```\n\n--------------------------------------\n\n### Compiling the CUDA libraries\nThe Chamfer distance function can be compiled by running `python3 setup.py install` under `external/chamfer3D`.\nThe source code is taken/modified from the [AtlasNet](https://github.com/ThibaultGROUEIX/AtlasNet) repository.  \nWhen compiling CUDA code, you may need to modify `CUDA_PATH` accordingly.  \n\n--------------------------------------\n\n### Running the code\n\n- #### Evaluating the downloaded pretrained models\n  ```bash\n  # ShapeNet (trained on multi-view renderings)\n  python3 evaluate.py --model=sdf_srn --yaml=options/shapenet/sdf_srn.yaml --name=airplane_pretrained --data.shapenet.cat=plane --load=pretrained/shapenet_airplane.ckpt --tb= --visdom= --eval.vox_res=128\n  python3 evaluate.py --model=sdf_srn --yaml=options/shapenet/sdf_srn.yaml --name=car_pretrained --data.shapenet.cat=car --load=pretrained/shapenet_car.ckpt --tb= --visdom= --eval.vox_res=128\n  python3 evaluate.py --model=sdf_srn --yaml=options/shapenet/sdf_srn.yaml --name=chair_pretrained --data.shapenet.cat=chair --load=pretrained/shapenet_chair.ckpt --tb= --visdom= --eval.vox_res=128\n  # PASCAL3D+\n  python3 evaluate.py --model=sdf_srn --yaml=options/pascal3d/sdf_srn.yaml --name=airplane_pretrained --data.pascal3d.cat=plane --load=pretrained/pascal3d_airplane.ckpt --tb= --visdom= --eval.vox_res=128 --eval.icp\n  python3 evaluate.py --model=sdf_srn --yaml=options/pascal3d/sdf_srn.yaml --name=car_pretrained --data.pascal3d.cat=car --load=pretrained/pascal3d_car.ckpt --tb= --visdom= --eval.vox_res=128 --eval.icp\n  python3 evaluate.py --model=sdf_srn --yaml=options/pascal3d/sdf_srn.yaml --name=chair_pretrained --data.pascal3d.cat=chair --load=pretrained/pascal3d_chair.ckpt --tb= --visdom= --eval.vox_res=128 --eval.icp\n  ```\n  This will create the following files in the output directory (e.g. `output/sdf_srn_pascal3d/car_pretrained`):\n  - `chamfer.txt`: the (bidirectional) Chamfer distance error for each example.\n  - `dump/*_mesh.ply`: the resulting 3D meshes (from zero isosurface extraction with marching cubes).\n  - `dump/*.png`: images including input/rendered RGB images, input/predicted masks, depth maps and surface normal maps.\n  - `dump/vis.html`: a webpage to visualize all the images for convenience.\n  \n  The overall Chamfer distance error (the numbers reported in the paper) will also be shown on screen.  \n  Note that it takes longer to evaluate the PASCAL3D+ models since we run ICP to pre-align the predictions to the ground-truth shapes.\n\n- #### Training from scratch\n  To train SDF-SRN, we first quickly pretrain the generator with a spherical SDF for 1000 iterations with:\n  ```bash\n  # ShapeNet\n  python3 train.py --model=sdf_srn_pretrain --yaml=options/shapenet/sdf_srn_pretrain.yaml\n  # PASCAL3D+\n  python3 train.py --model=sdf_srn_pretrain --yaml=options/pascal3d/sdf_srn_pretrain.yaml\n  ```\n  This helps SDF-SRN converge to a feasible solution, otherwise it may get stuck in bad local minima.\n  \n  For the main training:\n  ```bash\n  # ShapeNet (~100K iterations for airplanes and cars, ~200K iterations for chairs)\n  python3 train.py --model=sdf_srn --yaml=options/shapenet/sdf_srn.yaml --name=airplane --data.shapenet.cat=plane --max_epoch=24 --loss_weight.shape_silh=1\n  python3 train.py --model=sdf_srn --yaml=options/shapenet/sdf_srn.yaml --name=car --data.shapenet.cat=car --max_epoch=27\n  python3 train.py --model=sdf_srn --yaml=options/shapenet/sdf_srn.yaml --name=chair --data.shapenet.cat=chair --max_epoch=28\n  # PASCAL3D+ (~30K iterations)\n  python3 train.py --model=sdf_srn --yaml=options/pascal3d/sdf_srn.yaml --name=airplane --data.pascal3d.cat=plane --freq.eval=30 --freq.ckpt=30 --max_epoch=500\n  python3 train.py --model=sdf_srn --yaml=options/pascal3d/sdf_srn.yaml --name=car --data.pascal3d.cat=car --freq.eval=10 --freq.ckpt=10 --max_epoch=170\n  python3 train.py --model=sdf_srn --yaml=options/pascal3d/sdf_srn.yaml --name=chair --data.pascal3d.cat=chair --freq.eval=60 --freq.ckpt=60 --max_epoch=900\n  ```\n  The above command for ShapeNet runs single-view training on multi-view data.\n  To train on single-view ShapeNet data (only 1 view is available per CAD model) with the reported settings, run\n  ```bash\n  # single-view ShapeNet chairs (~50K iterations)\n  python3 train.py --model=sdf_srn --yaml=options/shapenet/sdf_srn.yaml --name=chair_1view_1kcad --data.shapenet.cat=chair --data.shapenet.train_view=1 --data.train_sub=1000 --data.augment.brightness=0.2 --data.augment.contrast=0.2 --data.augment.saturation=0.2 --data.augment.hue=0.5 --freq.eval=100 --freq.ckpt=100 --max_epoch=800\n  ```\n  This trains on a subset of 1000 chair CAD models with 1 viewpoint each while randomly jittering the colors.\n\n  To evaluate the trained models:\n  ```bash\n  # ShapeNet\n  python3 evaluate.py --model=sdf_srn --yaml=options/shapenet/sdf_srn.yaml --name=airplane --data.shapenet.cat=plane --tb= --visdom= --eval.vox_res=128 --resume\n  python3 evaluate.py --model=sdf_srn --yaml=options/shapenet/sdf_srn.yaml --name=car --data.shapenet.cat=car --tb= --visdom= --eval.vox_res=128 --resume\n  python3 evaluate.py --model=sdf_srn --yaml=options/shapenet/sdf_srn.yaml --name=chair --data.shapenet.cat=chair --tb= --visdom= --eval.vox_res=128 --resume\n  # PASCAL3D+\n  python3 evaluate.py --model=sdf_srn --yaml=options/pascal3d/sdf_srn.yaml --name=airplane --data.pascal3d.cat=plane --tb= --visdom= --eval.vox_res=128 --eval.icp --resume\n  python3 evaluate.py --model=sdf_srn --yaml=options/pascal3d/sdf_srn.yaml --name=car --data.pascal3d.cat=car --tb= --visdom= --eval.vox_res=128 --eval.icp --resume\n  python3 evaluate.py --model=sdf_srn --yaml=options/pascal3d/sdf_srn.yaml --name=chair --data.pascal3d.cat=chair --tb= --visdom= --eval.vox_res=128 --eval.icp --resume\n  ```\n  The expected output is similar to those described above (in the pretrained models section).\n\n- #### Visualizing the results\n  We have included code to visualize the training over TensorBoard. The TensorBoard events include the following:\n  - **SCALARS**: the losses and bidirectional Chamfer distances (for both training and validation sets).  \n  - **IMAGES**: visualization of the RGB/mask/depth/normal images.\n  \n  We also provide visualization of dense point clouds sampled on the zero isosurface in Visdom.\n\n- #### General usage of the codebase\n  The simplest command to run training is:\n  ```bash\n  python3 train.py --model=sdf_srn\n  ```\n  This will run `model/sdf_srn.py` as the main engine with `options/sdf_srn.yaml` as the main config file.\n  Note that `sdf_srn` is hierarchically inherited from `implicit` and `base`, which makes the codebase customizable.  \n  The complete configuration will be printed upon execution. To override specific options, add `--\u003ckey\u003e=value` or `--\u003ckey1\u003e.\u003ckey2\u003e=value` (and so on) to the command line. The configuration will be loaded as the variable `opt` throughout the codebase.  \n  If you want to reproduce the reported results, load preset configurations with the `yaml` option (details below).\n  \n  Some tips on using and understanding the codebase:\n  - The computation graph for forward/backprop is stored in `var` throughout the codebase.\n  - The losses are stored in `loss`. To add a new loss function, just implement it in `compute_loss()` and add its weight to `opt.loss_weight.\u003cname\u003e`. It will automatically be added to the overall loss and logged to Tensorboard.\n  - If you are using a multi-GPU machine, you can add `--gpu=\u003cgpu_number\u003e` to specify which GPU to use. Multi-GPU training/evaluation is currently not supported.\n  - To resume from a previous checkpoint, add `--resume=\u003cepoch_number\u003e`, or just `--resume` to resume from the latest checkpoint.\n  - (to be continued....)\n  \n--------------------------------------\n\nIf you find our code useful for your research, please cite\n```\n@inproceedings{lin2020sdfsrn,\n  title={SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Images},\n  author={Lin, Chen-Hsuan and Wang, Chaoyang and Lucey, Simon},\n  booktitle={Advances in Neural Information Processing Systems ({NeurIPS})},\n  year={2020}\n}\n```\n\nPlease contact me (chlin@cmu.edu) if you have any questions!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchenhsuanlin%2Fsigned-distance-SRN","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchenhsuanlin%2Fsigned-distance-SRN","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchenhsuanlin%2Fsigned-distance-SRN/lists"}