{"id":18614422,"url":"https://github.com/stevenygd/NFGP","last_synced_at":"2025-04-11T00:30:42.312Z","repository":{"id":43017725,"uuid":"418386835","full_name":"stevenygd/NFGP","owner":"stevenygd","description":"Pytorch implementation of NeurIPS 2021 paper: Geometry Processing with Neural Fields.","archived":false,"fork":false,"pushed_at":"2023-10-25T06:07:20.000Z","size":43770,"stargazers_count":196,"open_issues_count":0,"forks_count":18,"subscribers_count":9,"default_branch":"master","last_synced_at":"2024-11-07T03:31:30.382Z","etag":null,"topics":["geometry-processing","machine-learning","neural-fields","neural-network"],"latest_commit_sha":null,"homepage":"https://www.guandaoyang.com/NFGP","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stevenygd.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-18T07:15:35.000Z","updated_at":"2024-11-06T02:39:15.000Z","dependencies_parsed_at":"2024-11-07T03:40:35.288Z","dependency_job_id":null,"html_url":"https://github.com/stevenygd/NFGP","commit_stats":null,"previous_names":[],"tags_count":0,"template":true,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stevenygd%2FNFGP","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stevenygd%2FNFGP/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stevenygd%2FNFGP/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stevenygd%2FNFGP/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stevenygd","download_url":"https://codeload.github.com/stevenygd/NFGP/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248322216,"owners_count":21084333,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["geometry-processing","machine-learning","neural-fields","neural-network"],"created_at":"2024-11-07T03:25:56.379Z","updated_at":"2025-04-11T00:30:42.305Z","avatar_url":"https://github.com/stevenygd.png","language":"Jupyter Notebook","funding_links":[],"categories":["Papers"],"sub_categories":["NeRF Related Tasks"],"readme":"# Geometry Processing with Neural Fields\n\nPytorch implementation for the **NeurIPS 2021** paper [(project page)](https://www.guandaoyang.com/NFGP):\n\n[Geometry Processing with Neural Fields](https://openreview.net/pdf?id=JG-SlCAx5_K)\n\n[Guandao Yang](https://www.guandaoyang.com/), \n[Serge Belongie](https://blogs.cornell.edu/techfaculty/serge-belongie/),\n[Bharath Hariharan](http://home.bharathh.info/),\n[Vladlen Koltun](http://vladlen.info/)\n\n![Teaser](docs/assets/teaser.png)\n\n\n\n## Introduction\n\nMost existing geometry processing algorithms use meshes as the default shape representation.  \nManipulating meshes, however, requires one to maintain high quality in the surface discretization.  \nFor example, changing the topology of a mesh usually requires additional procedures such as remeshing. \nThis paper instead proposes the use of neural fields for geometry processing. \nNeural fields can compactly store complicated shapes without spatial discretization. \nMoreover, neural fields are infinitely differentiable, which allows them to be optimized for objectives that involve higher-order derivatives. \nThis raises the question: _can geometry processing be done entirely using neural fields?_ \nWe introduce loss functions and architectures to show that some of the most challenging geometry processing tasks, such as deformation and filtering, can be done with neural fields. \nExperimental results show that our methods are on par with the well-established mesh-based methods without committing to a particular surface discretization.\n\n## Installation \n\nThis repository provides a [Anaconda](https://www.anaconda.com/) environment, and requires NVIDIA GPU to run the\n optimization routine. \nThe code is tested in [CUDA 10.2](), ubuntu 16.04.\nThe environment can be set-up using the following commands:\n```bash\nconda env create -f environment.yml\nconda activate NFGP\n```\n\n## Dataset\n\nWe offer pre-processed dataset for reproducing the experimental results in the paper. \nThe instruciton is in section [_Download Dataset_](#download-dataset). \nIf you want to run experiment for your own shape, you need to prepare data in the following two steps:\n1. Create SDF samples for the shape (See section [_Create SDF Samples_](#create-sdf-samples))\n2. Train a neural field to fit those samples (See section [Create Input Neural Fields](#create-input-neural-fields)).\n3. Depending on the tasks, create the user specified input. For deofmration task, please refer to section [_Create\n Deformation Handles_](#create-deformation-handles). For sharpening and smoothing, you can directly set it through\n  the configuration file or the hyper-parameter.\n\n#### Download Dataset\n\nYou can find the dataset in the following Google drive folder: [Google Drive](https://drive.google.com/drive/folders/1oHS1jpVwUJ7mbKKdmfHsfXsDwS0kEZtW?usp=sharing). \nAlternatively, you can also download data using the following command:\n```bash\n# Download data to train Neural Fields\nwget https://geometry-processing-with-neural-fields.s3.us-east-2.amazonaws.com/nf_data.zip\nunzip nf_data.zip\n\n# Download the deformation Data\nwget https://geometry-processing-with-neural-fields.s3.us-east-2.amazonaws.com/deform_data.zipA\nunzip deform_data.zip\n```\n\nWe also provide pretrained neural fields for convenience:\n```bash\nwget https://geometry-processing-with-neural-fields.s3.us-east-2.amazonaws.com/nf_pretrained.zip\nunzip nf_pretrained.zip\n```\n\n#### Create SDF Samples \n\nFollowing command creates SDF ground truth samples for obtaining the Neural Fields that approximates the SDF of the\n shape.\n```bash\npython scripts/prep_sdf_data.py \u003cmesh_file_name\u003e --out_path data/\u003cmesh_file_name\u003e\n\n# Example\npython scripts/prep_sdf_data.py armidillo.obj --out_path data/armidillo\n```\n\nThe data preprocessing pipeline will sample 5M points uniformly withint `[-1, 1]^3` and another 5M near surface (i.e\n. surface plus a Gaussian with standard deviation of `0.1`).\nWe then use the `mesh_to_sdf` package to compute the SDF of the points.\nThese points will be saved to following files:\n- `\u003cout_path\u003e/mesh.obj` : a copy of the original mesh.\n- `\u003cout_path\u003e/sdf.npy` : a numpy file that contains the sampled points, their SDF, and the original mesh.\n\nNote that this pipeline works the best with watertight mesh.\nThis processing pipeline takes about 5 - 10 minutes to finish for a mesh with 30k faces.\n\n#### Create Input Neural Fields \n\nOnce you've obtained the `sdf.npy` file in the previous subsection,\nyou can use those data to train a neural field:\n```bash\npython train.py configs/recon/create_neural_fields.yaml --hparams data.path=\u003cyour_sdf.npy\u003e\n```\nYou can also create your own config following the examples in folder `configs/recon`.\n\n#### Create Deformation Handles \n\nPlease see `notebooks/deformation-handles-*.ipynb` for examples.\n\n## Smoothing and Sharpening \n\nTo run our methods on the smoothing or sharpening task, you can use the following configurations:\n```bash\n# Armadillo\npython train.py configs/filtering/filtering_Armadillo_beta0.yaml  # smoothing\npython train.py configs/filtering/filtering_Armadillo_beta2.yaml  # sharpening\n# Noisy sphere\npython train.py configs/filtering/filtering_HalfNoisySphere_beta0.yaml # smoothing\npython train.py configs/filtering/filtering_HalfNoisySphere_beta2.yaml # sharpening\n# Noisy torus\npython train.py configs/filtering/filtering_NoisyTorus_beta0.yaml # smoothing\npython train.py configs/filtering/filtering_NoisyTorus_beta2.yaml # sharpening\n```\n\nYou can change the value of `beta` by adding `--hparams trainer.beta=\u003cyourbeta\u003e`.\nTo run it on a different shape, you need to change the `models.decoder` to load the appropriate neural fields.\n\n## Deformation\n\nTo run the deformation experiments, you can use the following configurations:\n```bash\n# Jolteon\npython train.py configs/deformation/jolteon_jump_s1e-1_b1e-3.yaml\npython train.py configs/deformation/jolteon_nosedown_s1e-1_b1e-3.yaml\n```\n\nTo change the amount of bending or stretching resistent, \nyou can change the value of `trainer.loss_stretch.weight` and `trainer.loss_bend,weught` by\nadding `--hparams trainer.loss_stretch.weight=\u003cnew_weight\u003e` or `--hparams trainer.loss_bend.weight=\u003cnew_weight\u003e`\n\nTo visualize the results, you can use the notebook `notebooks/visualize-deformation.ipynb`.\n\n## Citation \n\nIf you find our paper or code useful, please cite us:\n```\n@inproceedings{yang2021geometry,\n  title={Geometry Processing with Neural Fields},\n  author={Yang, Guandao and Belongie, Serge and Hariharan, Bharath and Koltun, Vladlen},\n  booktitle={Thirty-Fifth Conference on Neural Information Processing Systems},\n  year={2021}\n}\n```\n\n## Acknowledgement\nGuandao’s PhD was supported in part by a research gift from Magic Leap and\na donation from NVIDIA. We want to thank Wenqi Xian, Professor Steve Marschner, and members\nof Intel Labs for providing insightful feedback for this project.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstevenygd%2FNFGP","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstevenygd%2FNFGP","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstevenygd%2FNFGP/lists"}