{"id":13442793,"url":"https://github.com/ThibaultGROUEIX/3D-CODED","last_synced_at":"2025-03-20T15:31:01.138Z","repository":{"id":46035349,"uuid":"137830527","full_name":"ThibaultGROUEIX/3D-CODED","owner":"ThibaultGROUEIX","description":"Pytorch Implementation for the project : `3D-CODED` and `Learning Elementary Structure`","archived":false,"fork":false,"pushed_at":"2021-11-18T16:26:14.000Z","size":6376,"stargazers_count":318,"open_issues_count":2,"forks_count":50,"subscribers_count":9,"default_branch":"master","last_synced_at":"2024-08-01T03:41:58.471Z","etag":null,"topics":["3d-correspondences","atlasnet","deep-learning"],"latest_commit_sha":null,"homepage":"http://imagine.enpc.fr/~groueixt/3D-CODED/index.html","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ThibaultGROUEIX.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-06-19T02:31:35.000Z","updated_at":"2024-05-16T06:57:43.000Z","dependencies_parsed_at":"2022-09-10T08:40:33.087Z","dependency_job_id":null,"html_url":"https://github.com/ThibaultGROUEIX/3D-CODED","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThibaultGROUEIX%2F3D-CODED","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThibaultGROUEIX%2F3D-CODED/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThibaultGROUEIX%2F3D-CODED/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThibaultGROUEIX%2F3D-CODED/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ThibaultGROUEIX","download_url":"https://codeload.github.com/ThibaultGROUEIX/3D-CODED/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221772560,"owners_count":16878128,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-correspondences","atlasnet","deep-learning"],"created_at":"2024-07-31T03:01:50.992Z","updated_at":"2024-10-28T03:31:05.586Z","avatar_url":"https://github.com/ThibaultGROUEIX.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# **3D-CODED** [[Project Page]](http://imagine.enpc.fr/~groueixt/3D-CODED/index.html) [[Paper]](https://arxiv.org/abs/1806.05228) [[Talk]](http://imagine.enpc.fr/~groueixt/3D-CODED/ECCV_6Dpose_Workshop_Groueix.pptx) + **Learning Elementary Structure** [[Project Page]](http://imagine.enpc.fr/~deprellt/atlasnet2/) [[Paper]](https://arxiv.org/abs/1908.04725) [[Code]](https://github.com/TheoDEPRELLE/AtlasNetV2)\n\n\n\n **3D-CODED : 3D Correspondences by Deep Deformation** \u003cbr\u003eThibault Groueix,  Matthew Fisher, Vladimir G. Kim , Bryan C. Russell, Mathieu Aubry  \u003cbr\u003e\nIn [ECCV, 2018](http://openaccess.thecvf.com/ECCV2018.py).\n\n **Learning elementary structures for 3D shape generation and matching** \u003cbr\u003eTheo Deprelle, Thibault Groueix,  Matthew Fisher, Vladimir G. Kim , Bryan C. Russell, Mathieu Aubry  \u003cbr\u003e\nIn [Neurips, 2019](https://neurips.cc/Register2?showPanel=2). [Official Code](https://github.com/TheoDEPRELLE/AtlasNetV2)\n\n\n## A note on data.\n\nData download should be automatic. However, due to the new google drive traffic caps, you may have to download manually. If you run into an error during the data download, you can refer to https://github.com/ThibaultGROUEIX/AtlasNet/issues/61.\n\nYou can manually download the data from these sources :\n* Google drive **datas_surreal_test.pth**: https://drive.google.com/file/d/1VGax9j64AvCVORtiQ3ZSPecI0bfZrEVe/view?usp=sharing\n* Google drive **datas_surreal_train.pth**: https://drive.google.com/file/d/1HVReM43YtJqhGfbmE58dc1-edI_oz9YG/view?usp=sharing\n__________________________________________________\n\n\u003cdetails open\u003e\u003csummary\u003eLearned templates\u003c/summary\u003e\n\n\n![image-20190912160913089](./README/learned_templates.png)\n\n\u003cdetails\u003e\u003csummary\u003eFaust results\u003c/summary\u003e\n\n\n| Method                                               | L2 Train SURREAL | L2 Val SURREAL | Faust Intra results | Faust Inter results |\n| ---------------------------------------------------- | --------------- |  --------------- | ------------------- | ------------------- |\n| **3D-CODED**                                         | 1.098 | 1.315   | 1.747         |2.641               |\n| **Points Translation 3D**                              |9.980 | 1.263 | **1.626** | 2.714 |\n| **Patch Deformation  3D**                             | 1.028| 1.436      |1.742               | **2.578**    |\n| **Points Translation + Patch Deformation 3D**           |**0.969**| **1.173** |1.676              | 2.779       |\n| **Points Translation 2D**                              | 1.09 | 1.54 | 2.054 | 3.005 |\n| **Patch Deformation 2D**                             | 6.354            | 6.767          | 4.46                | 5.420               |\n| **Points Translation 10D**                              | **0.906**        | **1.064**      | 1.799               | 2.707               |\n| **Patch Deformation  10D**                             | 0.952            | 1.183          | 1.683               | 2.83                |\n\n\n\u003c/details\u003e\n\n\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eSample results\u003c/summary\u003e\nInput : 2 meshes\u003cbr\u003e\nTask : put them in point-wise correspondence. (suggested by color)\n\n\u003cimg src=\"README/mesh25.ply.gif\" style=\"zoom:80%\" /\u003e\u003cimg src=\"README/25RecBestRotReg.ply.gif\" style=\"zoom:80%\" /\u003e\n\n\u003cimg src=\"README/mesh8.ply.gif\" style=\"zoom:80%\" /\u003e\u003cimg src=\"README/8RecBestRotReg.ply.gif\" style=\"zoom:80%\" /\u003e\n\u003c/details\u003e\n\n\n## Install :construction_worker: [[Pytorch](http://pytorch.org/), Conda]\n\nThis implementation uses [Pytorch](http://pytorch.org/). \n\n```shell\ngit clone https://github.com/ThibaultGROUEIX/3D-CODED.git ## Download the repo\ncd 3D-CODED; git submodule update --init\nconda env create -f 3D-CODED-ENV.yml ## Create python env\nsource activate pytorch-3D-CODED\npip install http://imagine.enpc.fr/~langloip/data/pymesh2-0.2.1-cp37-cp37m-linux_x86_64.whl\ncd extension; python setup.py install; cd ..\n```\n\n\n\n## Demo :train2: and Inference [Trained Models](https://drive.google.com/file/d/1ZAjOuTaeDrKJbFffzLnLn_K-C86fYCXs/view)\n\n```shell\npython inference/correspondences.py --dir_name learning_elementary_structure_trained_models/1patch_deformation\n```\nThis script takes as input 2 meshes from ```data``` and compute correspondences in ```results```. Reconstruction are saved in ```dir_name```.\n\n\u003cdetails\u003e\u003csummary\u003eOptions  (Usually default parameters are good)\u003c/summary\u003e\n\n\n```python\n# Key parameters\n'--dir_name', type=str, default=\"\",  help='dirname')\n'--inputA', type=str, default =  \"data/example_0.ply\",  help='your path to mesh 0'\n'--inputB', type=str, default =  \"data/example_1.ply\",  help='your path to mesh 1'\n\n# Secondary parameters\n'--HR', type=int, default=1, help='Use high Resolution template for better precision in the nearest neighbor step ?'\n'--reg_num_steps', type=int, default=3000, help='number of epochs to train for during the regression step'\n'--num_points', type=int, default = 6890,  help='number of points fed to poitnet'\n'--num_angles', type=int, default = 100,  help='number of angle in the search of optimal reconstruction. Set to 1, if you mesh are already facing the cannonical \t\t\t\tdirection as in data/example_1.ply'\n'--env', type=str, default=\"CODED\", help='visdom environment'\n'--clean', type=int, default=0, help='if 1, remove points that dont belong to any edges'\n'--scale', type=int, default=0, help='if 1, scale input mesh to have same volume as the template'\n'--project_on_target', type=int, default=0, help='if 1, projects predicted correspondences point on target mesh'\n'--randomize', type=int, default=0, help='if 1, projects predicted correspondences point on target mesh'\n'--LR_input', type=int, default=1, help='Use Low Resolution Input '\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eResults  \u003c/summary\u003e\n\n\n* **Initial guesses** for *example0* and *example1*:\n\n\u003cimg src=\"README/example_0InitialGuess.ply.gif\" style=\"zoom:80%\" /\u003e\u003cimg src=\"README/example_1InitialGuess.ply.gif\" style=\"zoom:80%\" /\u003e\n\n* **Final reconstruction** for *example0* and *example1*:\n\n\u003cimg src=\"README/example_0FinalReconstruction.ply.gif\" style=\"zoom:80%\" /\u003e\u003cimg src=\"README/example_1FinalReconstruction.ply.gif\" style=\"zoom:80%\" /\u003e\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eOn your own meshes  \u003c/summary\u003e\n\nYou need to make sure your meshes are preprocessed correctly :\n\n* The meshes are loaded with **Trimesh**, which should support a bunch of formats, but I only tested ```.ply``` files. Good converters include [**Assimp**](https://github.com/assimp/assimp) and [Pymesh](https://github.com/qnzhou/PyMesh).\n\n\n* The trunk axis is the **Y axis** (visualize your mesh against the mesh in ```data``` to make sure they are normalized in the same way). \n* the **scale** should be about 1.7 for a standing human (meaning the unit for the point cloud is the ```m```). You can automatically scale them with the flag ```--scale 1```\n\n\n\n\n--\u003e Failure modes instruction : :warning:\n\n- Sometimes the reconstruction is flipped, which break the correspondences. In the easiest case where you meshes are registered in the same orientation, you can just fix this angle in ```correspondence.py``` line 240, to avoid the flipping problem. Also note from this line that the angle search only looks in [-90°,+90°].\n\n- Check the presence of lonely outliers that break the Pointnet encoder. You could try to remove them with the ```--clean``` flag.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eFAUST \u003c/summary\u003e\n\n* If you want to use ```inference/correspondences.py``` to process a hole dataset, like FAUST test set, you can use ```./inference/script.py```, for the FAUST inter challenge. **Good luck :-)**\n\n\u003c/details\u003e\n\n\n\n## Training\n\n\n\n```shell\npython ./training/train.py\n```\n\n\u003cdetails\u003e\u003csummary\u003e Trainer's Options\u003c/summary\u003e\n\n```python\n'--point_translation', type=int, default=0, help='point_translation'\n'--dim_template', type=int, default=3, help='dim_template'\n'--patch_deformation', type=int, default=0, help='patch_deformation'\n'--dim_out_patch', type=int, default=3, help='dim_out_patch'\n'--start_from', type=str, default=\"TEMPLATE\", choices=[\"TEMPLATE, SOUP, TRAINDATA\"] ,help='dim_out_patch'\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003e Monitor your training on http://localhost:8888/\u003c/summary\u003e\n\n![visdom](./README/1532524819586.png)\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003e Note on data preprocessing  \u003c/summary\u003e\n\n\nThe generation process of the dataset is quite heavy so we provide our processed data. Should you want to reproduce the preprocessing, go to ```data/README.md```. Brace yourselve :-)\n\n\u003c/details\u003e\n\n\n\n## Reproduce the paper :train2:\n\n```shell\npython script/launch.py --mode training #Launch 4 trainings with different parameters.\npython script/launch.py --mode inference #Eval the 4 pre-trained models.\n```\n\n\n\n## Citing this work \n\nIf you find this work useful in your research, please consider citing:\n\n\n\n```\n@inproceedings{deprelle2019learning,\n  \t\t\ttitle={Learning elementary structures for 3D shape generation and matching},\n  \t\t\tauthor={Deprelle, Theo and Groueix, Thibault and Fisher, Matthew and Kim, Vladimir G and Russell, Bryan C and Aubry, Mathieu},\n  \t\t\tbooktitle={Neurips},\n  \t\t\tyear={2019}\n}\n```\n\n##  \n\n```\n@inproceedings{groueix2018b,\n          title = {3D-CODED : 3D Correspondences by Deep Deformation},\n          author={Groueix, Thibault and Fisher, Matthew and Kim, Vladimir G. and Russell, Bryan and Aubry, Mathieu},\n          booktitle = {ECCV},\n          year = {2018}\n        }\n```\n\n## \n\n## License\n\n[MIT](https://github.com/ThibaultGROUEIX/AtlasNet/blob/master/license_MIT)\n\n\n\n## Cool Contributions\n\n* **[Zhongshi Jiang](https://cs.nyu.edu/~zhongshi/)** applying trained model on a monster model :japanese_ogre: (**left**: original , **right:** reconstruction)\n\n![visdom](./README/image.png)\n\n[![Analytics](https://ga-beacon.appspot.com/UA-91308638-2/github.com/ThibaultGROUEIX/3D-CODED/readme.md?pixel)](https://github.com/ThibaultGROUEIX/3D-CODED/)\n\n\n\n## Acknowledgement\n\n- The code for the Chamfer Loss was adapted from **[Fei Xia](http://fxia.me/)**'a repo : [PointGan](https://github.com/fxia22/pointGAN). Many thanks to him !\n- The code for the Laplacian regularization comes from [**Angjoo** **Kanazawa**](https://people.eecs.berkeley.edu/~kanazawa/) and [**Shubham** **Tulsiani**](https://people.eecs.berkeley.edu/~shubhtuls/). This was so helpful, thanks !\n- Part of the SMPL parameters used in the training data comes from [**Gül** **Varol**](https://www.di.ens.fr/~varol/)'s repo : https://github.com/gulvarol/surreal But most of all, thanks for all the advices :)\n- The FAUST Team for their prompt reaction in resolving a benchmark issue the week of the deadline, especially to [**Federica** **Bogo**](https://ps.is.tuebingen.mpg.de/person/fbogo) and **Jonathan Williams**.\n- The efficient code for to compute geodesic errors comes from  https://github.com/zorah/KernelMatching. Thanks!\n- The [SMAL](http://smalr.is.tue.mpg.de/) team, and [SCAPE](https://ai.stanford.edu/~drago/Projects/scape/scape.html) team for their help in generating the training data.\n- [DeepFunctional Maps](https://arxiv.org/abs/1704.08686) authors for their fast reply the week of the rebuttal ! Many thanks.\n- **[Hiroharu Kato](http://hiroharu-kato.com/projects_en/neural_renderer.html)** for his very clean neural renderer code, that I used for the gifs :-)\n- [Pytorch developpers](https://github.com/pytorch/pytorch) for making DL code so easy.\n- This work was funded by [Ecole Doctorale MSTIC](http://www.univ-paris-est.fr/fr/-ecole-doctorale-mathematiques-et-stic-mstic-ed-532/). Thanks !\n- And last but not least, my great co-authors : **[Theo Deprelle](http://imagine.enpc.fr/~deprellt/)**,  **[Matthew Fisher](http://graphics.stanford.edu/~mdfisher/publications.html), [Vladimir G. Kim](http://vovakim.com/), [Bryan C. Russell](http://bryanrussell.org/), and [Mathieu Aubry](http://imagine.enpc.fr/~aubrym/cv.html)**\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FThibaultGROUEIX%2F3D-CODED","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FThibaultGROUEIX%2F3D-CODED","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FThibaultGROUEIX%2F3D-CODED/lists"}