Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/Mirgahney/dynsurf
https://github.com/Mirgahney/dynsurf
Last synced: 28 days ago
JSON representation
- Host: GitHub
- URL: https://github.com/Mirgahney/dynsurf
- Owner: Mirgahney
- License: mit
- Created: 2023-11-21T19:14:43.000Z (about 1 year ago)
- Default Branch: master
- Last Pushed: 2023-11-27T19:16:05.000Z (about 1 year ago)
- Last Synced: 2024-12-06T19:56:36.070Z (about 1 month ago)
- Language: Python
- Size: 151 MB
- Stars: 3
- Watchers: 1
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-dynamic-NeRF - Code
README
Neural Surface Reconstruction of Dynamic Scenes with Monocular RGB-D Camera
Mirgahney Mohamed
·
Lourdes Agapito
3DV 2024
Paper | Project Page | Poster
We propose DynamicSurf, a model-free neural implicit surface reconstruction method for high-fidelity 3D modelling of non-rigid surfaces from monocular RGB-D video.
## Usage
### Data Convention
The data is organized as [NeuS](https://github.com/Totoro97/NeuS#data-convention)
```
|-- cameras_sphere.npz # camera parameters
|-- depth
|-- # target depth for each view
...
|-- image
|-- # target RGB each view
...
|-- mask
|-- # target mask each view (For unmasked setting, set all pixels as 255)
...
```
Here `cameras_sphere.npz` follows the data format in [IDR](https://github.com/lioryariv/idr/blob/main/DATA_CONVENTION.md), where `world_mat_xx` denotes the world-to-image projection matrix, and `scale_mat_xx` denotes the normalization matrix.
### Pre-processed Data
You can download a part of pre-processed [KillingFusion](https://campar.in.tum.de/personal/slavcheva/deformable-dataset/index.html) data [here](https://drive.google.com/file/d/1wQ4yB7r-a8sFkwB6bEJIDp14AhjG8g_B/view?usp=sharing) and unzip it into `./`.
Important Tips: If the pre-processed data is useful, please cite the related paper(s) and strictly abide by related open-source license(s).
### Setup
Clone this repository and create the environment (please notice CUDA version)
```shell
git clone https://github.com/Mirgahney/dysurf.git
cd dysurf
```
The code is tested with Python 3.9 and PyTorch 1.11 with CUDA 11.3. DynamicSurf requires [smooth_sampler](https://github.com/tymoteuszb/smooth-sampler), which is a drop-in replacement for PyTorch's grid sampler that support double back-propagation. Also the following packages are required:
```shell
conda env create -f dysurf.yml
conda activate dysurf
```
Dependencies (click to expand)
* torch==1.11
* pytorch3d
* scikit-image
* open3d
* imageio
* matplotlib
* configargparse
* tensorboard
* opencv-contrib-python
* opencv_python==4.5.2.52
* trimesh==3.9.8
* numpy==1.21.2
* scipy==1.7.0
* PyMCubes==0.1.2
### Running
- **Training**
```shell
python3 -W ignore exp_runner_grid_hydra.py
```
- **Evaluating pre-trained model**
Coming Soon
### Data Pre-processing
Coming Soon
### Geometric Projection
Coming Soon
## Todo List
- [ ] Code of Data Pre-processing
- [ ] Code of Geometric Projection
- [ ] Pre-trained Models and Evaluation Code
- [x] Training Code
## Acknowledgements
This project is built upon [NDR](https://github.com/USTC3DV/NDR-code.git), [Go-Surf](https://github.com/JingwenWang95/go-surf), and [NeuS](https://github.com/Totoro97/NeuS). Some code snippets are also borrowed from [IDR](https://github.com/lioryariv/idr) and [NeRF-pytorch](https://github.com/yenchenlin/nerf-pytorch). The pre-processing code for camera pose initialization is borrowed from [Fast-Robust-ICP](https://github.com/yaoyx689/Fast-Robust-ICP). The evaluation code for geometry rendering is borrowed from [StereoPIFu_Code](https://github.com/CrisHY1995/StereoPIFu_Code). Thanks for these great projects. We thank all the authors for their great work and repos.
## Contact
If you have questions, please contact [Mirgahney Mohamed](https://mirgahney.github.io/).
## Citation
If you find our code or paper useful, please cite
```bibtex
@misc{mohamed2023dynamicsurf,
title={DynamicSurf: Dynamic Neural RGB-D Surface Reconstruction with an Optimizable Feature Grid},
author={Mirgahney Mohamed and Lourdes Agapito},
year={2023},
eprint={2311.08159},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```
If you find our pre-processed data useful, please cite the related paper(s) and strictly abide by related open-source license(s).