https://github.com/ndrplz/differentiable-renderer

Rastering algorithm to approximate the rendering of a 3D model silhouette in a fully differentiable way.
https://github.com/ndrplz/differentiable-renderer

differentiable-rendering eccv-2018 python pytorch rendering rendering-engine tensorflow tensorflow-models

Last synced: about 21 hours ago
JSON representation

Rastering algorithm to approximate the rendering of a 3D model silhouette in a fully differentiable way.

• Host: GitHub
• URL: https://github.com/ndrplz/differentiable-renderer
• Owner: ndrplz
• License: mit
• Created: 2018-09-06T09:50:21.000Z (about 6 years ago)
• Default Branch: master
• Last Pushed: 2018-12-22T09:34:51.000Z (almost 6 years ago)
• Last Synced: 2024-09-18T07:49:06.846Z (7 days ago)
• Topics: differentiable-rendering, eccv-2018, python, pytorch, rendering, rendering-engine, tensorflow, tensorflow-models
• Language: Python
• Size: 7.12 MB
• Stars: 80
• Watchers: 5
• Forks: 12
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Differentiable mesh renderer [Tensorflow]

Rastering algorithm to approximate the rendering of a 3D model silhouette in a fully differentiable way.

**:arrow_right: PyTorch implementation now available in the [`pytorch branch`](https://github.com/ndrplz/tensorflow-mesh-renderer/tree/pytorch)!**

---

This code accompanies the **[paper](https://iris.unimore.it/retrieve/handle/11380/1167726/205862/palazzi_eccvw.pdf)**:

> "End-to-end 6-DoF Object Pose Estimation through Differentiable Rasterization"

> Andrea Palazzi, Luca Bergamini, Simone Calderara, Rita Cucchiara

appeared in "*Second Workshop on 3D Reconstruction Meets Semantics (3DRMS)*" at ECCVW 2018.

    

        

            


            

            
Rendering while rotating the camera around Y axis.

            

        

        

            


            

            
Rendering while rotating the camera around Z axis.

            

        

     

 


---

## Docs

#### Input meshes

The input meshes are expected to be `np.ndarray` of shape (n_triangles, 3, 3). Each mesh can be composed of a variable number of triangles. Five meshes of car 3D models are already in [data/](./data/) directory to test the Rasterer.

#### Hello World

A short snippet to test the Rasterer is available in [`main.py`](./main.py). Just run it.

Three 3D models are randomly sampled from the dummy dataset and rendered in a batch. In this snippet the position of the camera is kept fixed for all three (but it may be changed).

If everything went fine, you should see the rendering output for the 3D models sampled. Something like this:



 

 
Output with resolution=(128, 128)



#### Moving the camera in the 3D world

There is a class [`RotoTranslation`](./rastering/rototranslation.py) which encapsulates the camera matrix, providing an easy way to change the camera location while always mantaining a geometrically consistent pose.

The initial camera pose is set as:

```

camera_pose = RotoTranslation(rotation=Vector(x=90., y=0., z=0.),

                              translation=Vector(x=0., y=-8., z=0.),

                              angle_unit='degrees')

```

To change its position, simply use its setters:

* `alpha_x`, `alpha_y`, `alpha_z` to change the rotation angle around respective axes

* `t_x`, `t_y`, `t_z` to change the translation along respective axes

## License

This code is released under MIT license. In case you use this code in your academic work, please cite the following paper:

```

@inproceedings{palazzi2018end,

  title={End-to-end 6-DoF Object Pose Estimation through Differentiable Rasterization},

  author={Palazzi, Andrea and Bergamini, Luca and Calderara, Simone and Cucchiara, Rita},

  booktitle={Second Workshop on 3D Reconstruction Meets Semantics (3DRMS)},

  year={2018}

}

```