{"id":18614406,"url":"https://github.com/zju3dv/neumesh","last_synced_at":"2025-04-09T07:05:49.095Z","repository":{"id":48997940,"uuid":"517106635","full_name":"zju3dv/NeuMesh","owner":"zju3dv","description":"Code for \"MeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing\", ECCV 2022 Oral","archived":false,"fork":false,"pushed_at":"2024-09-28T09:59:08.000Z","size":12340,"stargazers_count":385,"open_issues_count":0,"forks_count":13,"subscribers_count":25,"default_branch":"main","last_synced_at":"2025-04-02T05:56:55.255Z","etag":null,"topics":["3d-vision","nerf","neural-rendering"],"latest_commit_sha":null,"homepage":"https://zju3dv.github.io/neumesh/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zju3dv.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-07-23T16:21:07.000Z","updated_at":"2025-03-18T07:26:04.000Z","dependencies_parsed_at":"2024-05-16T13:43:01.651Z","dependency_job_id":"bc1b78a0-17c5-4351-a4f5-f9f562ab3868","html_url":"https://github.com/zju3dv/NeuMesh","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zju3dv%2FNeuMesh","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zju3dv%2FNeuMesh/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zju3dv%2FNeuMesh/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zju3dv%2FNeuMesh/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zju3dv","download_url":"https://codeload.github.com/zju3dv/NeuMesh/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247994119,"owners_count":21030050,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-vision","nerf","neural-rendering"],"created_at":"2024-11-07T03:25:56.137Z","updated_at":"2025-04-09T07:05:49.066Z","avatar_url":"https://github.com/zju3dv.png","language":"Python","funding_links":[],"categories":["Papers"],"sub_categories":["NeRF Related Tasks"],"readme":"# NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing\n\n### [Project Page](https://zju3dv.github.io/neumesh/) | [Video](https://www.youtube.com/watch?v=8Td3Oy7y_Sc) | [Paper](http://www.cad.zju.edu.cn/home/gfzhang/papers/neumesh/neumesh.pdf)\n\u003cdiv align=center\u003e\n\u003cimg src=\"assets/teaser.gif\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\n\u003e [NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing](http://www.cad.zju.edu.cn/home/gfzhang/papers/neumesh/neumesh.pdf)  \n\u003e \n\u003e [[Bangbang Yang](https://ybbbbt.com), [Chong Bao](https://github.com/1612190130/)]\u003csup\u003eCo-Authors\u003c/sup\u003e, [Junyi Zeng](https://github.com/LangHiKi/), [Hujun Bao](http://www.cad.zju.edu.cn/home/bao/), [Yinda Zhang](https://www.zhangyinda.com/), [Zhaopeng Cui](https://zhpcui.github.io/), [Guofeng Zhang](http://www.cad.zju.edu.cn/home/gfzhang/). \n\u003e \n\u003e ECCV 2022 Oral\n\u003e \n\n\n\u003c!-- ⚠️ Note: This is only a preview version of the code. Full code (with training scripts) will be released soon. --\u003e\n\n## Installation\nWe have tested the code on Python 3.8.0 and PyTorch 1.8.1, while a newer version of pytorch should also work.\nThe steps of installation are as follows:\n\n* create virtual environmental: `conda env create --file environment.yml`\n* install pytorch 1.8.1: `pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111  -f https://download.pytorch.org/whl/torch_stable.html`\n* install [open3d **development**](http://www.open3d.org/docs/latest/getting_started.html) version: `pip install [open3d development package url]`\n* install [FRNN](https://github.com/lxxue/FRNN), a fixed radius nearest neighbors search implemented on CUDA.\n\n## Data\nWe use DTU data of [NeuS version](https://github.com/Totoro97/NeuS) and [NeRF synthetic data](https://www.dropbox.com/scl/fi/i06pz7b6frvfvqtzmv74o/nerf_synthetic.zip?rlkey=je4q2vfen166jcxrqw86nfbqj\u0026st=9s62dxiz\u0026dl=0).\n\u003c!-- Our code reads the poses following the format of `camera_sphere.npz`. \nTherefore, we convert the poses of NeRF synthetic data to [`camera_sphere.npz`](). --\u003e\n\n[Update]: We release the [test image names](https://www.dropbox.com/scl/fo/o7vvobhspv6r2uw08p4hm/AGUlk0SpTLHme8cFcOoSKm0?rlkey=7s15mi3qr0ku85xmwqq6i2157\u0026st=6iwn840t\u0026dl=0) for our pre-trained model in the DTU dataset, which is randomly selected for evaluating PSNR/SSIM/LPIPS. Each sequence has a `val_names.txt` that contains the names of test images.\n\nP.S. Please enable the `intrinsic_from_cammat: True` for `hotdog`, `chair`, `mic` if you use the provided NeRF synthetic dataset.\n\n## Train\nHere we show how to run our code on one example scene.\nNote that the `data_dir` should be specified in the `configs/*.yaml`.\n\n1. Train the teacher network (NeuS) from multi-view images.\n```python\npython train.py --config configs/neus_dtu_scan63.yaml\n```\n2. Extract a triangle mesh from a trained teacher network.\n```python\npython extract_mesh.py --config configs/neus_dtu_scan63.yaml --ckpt_path logs/neus_dtuscan63/ckpts/latest.pt --output_dir out/neus_dtuscan63/mesh\n```\n3. Train NeuMesh from multi-view images and the teacher network. Note that the `prior_mesh`, `teacher_ckpt`, `teacher_config` should be specified in the `neumesh*.yaml`\n```python\npython train.py --config configs/neumesh_dtu_scan63.yaml\n```\n## Evaluation\n\nHere we provide all [pre-trained models](https://www.dropbox.com/scl/fo/3pmq3139vtifnaak3h41a/AMOH18OVsLBp9M72WyPjitI?rlkey=q77k3bbkl1bcil3qrvrsvz7se\u0026st=ay9fm5t8\u0026dl=0) of DTU and NeRF synthetic dataset.\n\nYou can evaluate images with the trained models. \n\n```python\npython -m render --config configs/neumesh_dtu_scan63.yaml   --load_pt logs/neumesh_dtuscan63/ckpts/latest.pt --camera_path spiral --background 1 --test_frame 24 --spiral_rad 1.2\n```\nP.S. If the time of inference costs too much, `--downscale` can be enabled for acceleration.\n\n\n\u003cdiv align=center\u003e\n\u003cimg src=\"assets/nvs.gif\" width=\"60%\"/\u003e\n\u003c/div\u003e\n\n## Manipulation\nPlease refer to [`editing/README.md`](editing/README.md).\n\n\n\n\n\n## Citing\n```\n@inproceedings{neumesh,\n    title={NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing},\n    author={{Chong Bao and Bangbang Yang} and Zeng Junyi and Bao Hujun and Zhang Yinda and Cui Zhaopeng and Zhang Guofeng},\n    booktitle={European Conference on Computer Vision (ECCV)},\n    year={2022}\n}\n```\nNote: joint first-authorship is not really supported in BibTex; you may need to modify the above if not using CVPR's format. For the SIGGRAPH (or ACM) format you can try the following:\n```\n@inproceedings{neumesh,\n    title={NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing},\n    author={{Bao and Yang} and Zeng Junyi and Bao Hujun and Zhang Yinda and Cui Zhaopeng and Zhang Guofeng},\n    booktitle={European Conference on Computer Vision (ECCV)},\n    year={2022}\n}\n```\n## Acknowledgement\nIn this project we use parts of the implementations of the following works:\n\n* [NeuS](https://github.com/Totoro97/NeuS) by Peng Wang\n* [neurecon](https://github.com/ventusff/neurecon) by ventusff\n\nWe thank the respective authors for open sourcing their methods.\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzju3dv%2Fneumesh","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzju3dv%2Fneumesh","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzju3dv%2Fneumesh/lists"}