https://github.com/iveevi/ngf
Source code for the SIGGRAPH 2024 conference paper "Neural Geometry Fields for Meshes"
https://github.com/iveevi/ngf
Last synced: about 1 year ago
JSON representation
Source code for the SIGGRAPH 2024 conference paper "Neural Geometry Fields for Meshes"
- Host: GitHub
- URL: https://github.com/iveevi/ngf
- Owner: iveevi
- Created: 2023-08-14T16:15:26.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2025-02-12T05:39:55.000Z (over 1 year ago)
- Last Synced: 2025-03-29T17:06:13.516Z (about 1 year ago)
- Language: Python
- Homepage: https://iveevi.github.io/ngf/index.html
- Size: 293 MB
- Stars: 140
- Watchers: 3
- Forks: 8
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Neural Geometry Fields for Meshes
# Training
**Requirements:** NVIDIA GPU (with CUDA support), Python, Assimp
The necessary python packages are listed in `requirements.txt` so installing
them with `pip install -r requirements.txt` is simplest. If there are errors in
install the library in `extensions` it can likely be resolved by installing
PyTorch and/or Wheel beforehand.
Then run `python source/train.py` on any target mesh:
```
usage: train.py [-h] [--mesh MESH] [--lod LOD] [--features FEATURES] [--display DISPLAY] [--batch BATCH] [--fixed-seed]
options:
-h, --help show this help message and exit
--mesh MESH Target mesh
--lod LOD Number of patches to partition
--features FEATURES Feature vector size
--display DISPLAY Display the result after training
--batch BATCH Batch size for training
--fixed-seed Fixed random seed (for debugging)
```
The results of the training will be placed into a local `results` directory as follows:
```
results
├── binaries (Binaries for trained neural geometry fields)
├── loss (Loss plots)
├── meta (Generic metadata)
├── quadrangulated (Partitioned surfaces)
├── stl (Final surfaces exported as STLs)
└── torched (Pytorch binary data)
```
The memory usage is relatively modest (under 8 GB for the default 1K patches
and 10 feature channels), but it can be adjusted with the batch size option.
Some tips to consider if errors appear:
* The STL format for meshes is most reliable; if the program complains from the
meshio library, try again with a STL rather than an OBJ or etc.
* PyMeshLab is used for simplification and quadrangulation, but the execution
is not always reliable. For this reason we time out the quadrangulation process
after a minute. If this happens, the target mesh is likely too large.
# Rasterization
**Requirements:** Vulkan, GLFW, CMake, Mesh shaders[^1]
Source code for the real-time rasterizer is provided in the `rasterizer`
directory. Make sure that the submodules have also been cloned. We rely on CMake
to compile the program:
```
cmake -B build .
cmake --build build -j
```
To run the rasterizer, execute the resulting `build/testbed` binary by
providing a path to the neural geometry field binary file (e.g. within
`results/binaries`). For example:
```
./build/testbed results/binaries/nefertiti-lod1000-f20.bin
```
A few binaries have been provided in the `resources/samples` directory to
explore the rasterizer on pretrained NGFs.
Here are some performance statistics for the rasterizer:
| GPU | Patch Count | Framerate (Frametime) |
| --------------- | ----------- | --------------------- |
| RTX 3060 Mobile | 1K | 250 FPS (4 ms) |
| RTX 3060 Mobile | 2.5K | 100 FPS (10 ms) |
| RTX 4090 | 1K | 1200 FPS (0.8 ms) |
| RTX 4090 | 2.5K | 600 FPS (1.6 ms) |
[^1]: Check `vulkaninfo` from the command-line or [search for your GPU model here.](https://vulkan.gpuinfo.org/listdevicescoverage.php?extension=VK_EXT_mesh_shader&platform=all)
# Citation
```
@inproceedings{vs2024ngfs,
title = {Neural Geometry Fields for Meshes},
author = {Sivaram, Venkataram and Ramamoorthi, Ravi and Li, Tzu-Mao},
numpages = {11},
year = {2024},
series = {SIGGRAPH '24}
}
```