Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/fengq1a0/FOF

Last synced: 24 days ago
JSON representation

Host: GitHub
URL: https://github.com/fengq1a0/FOF
Owner: fengq1a0
Created: 2022-10-16T22:21:35.000Z (about 2 years ago)
Default Branch: main
Last Pushed: 2024-07-31T15:35:53.000Z (5 months ago)
Last Synced: 2024-08-09T13:18:45.906Z (4 months ago)
Language: Python
Size: 5.23 MB
Stars: 83
Watchers: 8
Forks: 8
Open Issues: 7
Metadata Files:
- Readme: readme.md

Awesome Lists containing this project

StarryDivineSky - fengq1a0/FOF

README

# FOF: Learning Fourier Occupancy Field for Monocular Real-time Human Reconstruction

Releasing...

## Environment
To run this code, follow the following instructions.
```
conda create -n fof python=3.10
conda activate fof

# visit https://pytorch.org/
# install latest pytorch
# for example:
# conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia

pip install tqdm
pip install opencv-python
pip install open3d
pip install scikit-image
```

You will need the cuda corresponding to your pytorch to compile the tools below.
```
pip install Cython
pip install pybind11
# 1. fof
cd lib/fof
sh compile.sh
cd ../..

# 2. mc
# You need Eigen to compile this.
cd lib/mc
sh compile.sh
cd ../..
```
Check train.py and test.py, have fun with this repo!
## Checkpoints
You can get pretrained checkpoints [here](https://drive.google.com/drive/folders/1ocS0YND9vtdFN8Z99BoUdPu-ktSUwt5x?usp=sharing).
Put them in ```./ckpt```. They should be organized as shown below:
```
FOF
├─lib
└─ckpt
├─model32.pth
├─model48.pth
├─netB.pth
└─netF.pth
```
```model32.pth``` and ```model48.pth``` correspond to HR-Net-32 and HR-Net-48 version, respectively. Both of them are trained on THuman2.0 dataset only.

## Data preprocess
You will need triangle mesh and corresponding SMPL-X mesh to train the model. Texture is not needed and the meshes don't have to be watertight (which is better)! However, we prefer the meshes are clean, without floating fragments. Check the data preprocess code ```preprocess.py``` for more details. You can see the seven steps we do in the file. Before run it, you will need to
```
pip install cyminiball
```
Check the code, you can use the function ```work``` to process you data. If you are using THuman2.0 dataset, you can organize your data as shown below.

Your THuman2.0 mesh or your scanned mesh. (Only xxxx.obj is used)
```
$base_mesh # line 139 in preprocess.py
├───0000
| ├─0000.obj
| ├─material0.jpeg
| └─material0.mtl
├───0001
├───0002
...
└───0525
```

Corresponding SMPL-X mesh. (SMPL is also OK, but you will need to modify the dataloader yourself. Just a few lines.)
```
$base_smpl # line 140 in preprocess.py
├───0000
| ├─mesh_smplx.obj
| └─smplx_param.pkl
├───0001
├───0002
...
└───0525
```

The path you want to store the processed data. Dataloader also read data from it. You don't need to create it. After running ```preprocess.py```, it looks like:
```
$base_out # line 141 in preprocess.py
├───100k-180
| ├─0000.ply
| ├─0001.ply
| ...
| └─0525.ply
├───smpl-180
| ├─0000.ply
| ├─0001.ply
| ...
| └─0525.ply
└───train_th2.txt
```
```train_th2.txt``` lists the meshes used for training. It should look like:
```
0000
0001
0008
...
```
You can check the ```.ply``` meshes in meshlab. Make sure them are well-aligned.

Finally, you can change the ```line 26``` and ```line 27``` in the ```lib\dataset_mesh_only.py``` to the right path. Alternatively, you can identify them when using the TrainSet class. Now you can enjoy your training!