https://github.com/vigneshs10/tri-stage-occlusion-handling-normal-map-estimation-algorithm
3D normal map estimation pipeline using low resolution and highly occluded images
https://github.com/vigneshs10/tri-stage-occlusion-handling-normal-map-estimation-algorithm
deep-lab-v3-plus generative-adversarial-network normal-mapping occlusion-handling pifuhd real-esrgan
Last synced: 7 months ago
JSON representation
3D normal map estimation pipeline using low resolution and highly occluded images
- Host: GitHub
- URL: https://github.com/vigneshs10/tri-stage-occlusion-handling-normal-map-estimation-algorithm
- Owner: VigneshS10
- Created: 2022-10-29T15:45:52.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2022-10-29T16:46:22.000Z (almost 3 years ago)
- Last Synced: 2025-01-16T18:37:01.650Z (9 months ago)
- Topics: deep-lab-v3-plus, generative-adversarial-network, normal-mapping, occlusion-handling, pifuhd, real-esrgan
- Language: Jupyter Notebook
- Homepage:
- Size: 1.1 MB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# TRI-STAGE OCCLUSION HANDLING NORMAL MAP ESTIMATION ALGORITHM (TSOHNMEA)
**_SAMSUNG PRISM PROGRAM_**
---
> This repository is the official implementation of the TSOHNMEA. The project report can be viewed using this [link](https://drive.google.com/file/d/1FKWe7SpYEyDQn0eXGucMvCyW7rve7kDI/view?usp=share_link). Complete end-to-end pipeline code is not included yet.
## Requirements
To install requirements:
```setup
pip install -r requirements.txt
```
## Output
As the end-to-end pipeline code is not included, users must manually take the output from each .ipynb section. The order to extracts and input the outputs is
* background_occlusion_handler.ipynb (not included yet, so we suggest to use an image without any background to test the rest of the pipeline)
* Real_ESRGAN_blur_occlusion.ipynb
* PiFUHD_normal_map_estimation.ipynb
## Results
* ORDINARY NORMAL MAP ESTIMATION USING PIFU-HD MODEL:
https://user-images.githubusercontent.com/61982600/198841624-401f64ed-0bea-4c2f-9664-1ec43f2b82b1.mp4
* OUR TRI-STAGE OCCLUSION HANDLING NORMAL MAP ESTIMATION PIPELINE:
Background, Shadow and Blur occlusion of the original input image is handled and the
preprocessed input image is sent to the normal map estimation model
https://user-images.githubusercontent.com/61982600/198841628-c79d56c3-103b-4078-99aa-8b19621eb040.mp4
## TODO
* [ ] Upload complete end-to-end pipeline code (not included right now because further improvements are being made).
* [ ] Include background occlusion handler code (not included right now because more advanced and better architectures are being explored).
* [ ] Handle object occlusion.
## References
https://github.com/tensorflow/models/tree/master/research/deeplab \
https://github.com/xinntao/Real-ESRGAN \
https://github.com/facebookresearch/pifuhd
## Contact
For any queries, feel free to contact at vignesh.nitt10@gmail.com.