https://github.com/sayakpaul/spatial-transformer-networks-with-keras

This repository provides a Colab Notebook that shows how to use Spatial Transformer Networks inside CNNs in Keras.
https://github.com/sayakpaul/spatial-transformer-networks-with-keras

affine-transformation cnn keras mnist spatial-transformer-network tensorflow vision

Last synced: 17 days ago
JSON representation

This repository provides a Colab Notebook that shows how to use Spatial Transformer Networks inside CNNs in Keras.

• Host: GitHub
• URL: https://github.com/sayakpaul/spatial-transformer-networks-with-keras
• Owner: sayakpaul
• License: apache-2.0
• Created: 2021-04-17T16:04:19.000Z (almost 4 years ago)
• Default Branch: main
• Last Pushed: 2022-05-23T17:44:13.000Z (over 2 years ago)
• Last Synced: 2025-01-09T23:22:37.732Z (20 days ago)
• Topics: affine-transformation, cnn, keras, mnist, spatial-transformer-network, tensorflow, vision
• Language: Jupyter Notebook
• Homepage:
• Size: 7.2 MB
• Stars: 36
• Watchers: 2
• Forks: 7
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Spatial-Transformer-Networks-with-Keras





  








This repository provides a Colab Notebook that shows how to use [Spatial Transformer Networks (STN)](https://arxiv.org/abs/1506.02025) inside CNNs build in Keras. I have used utility functions mostly from [this repository](https://github.com/kevinzakka/spatial-transformer-network) to demonstrate an end-to-end example. STNs allow a (vision) network to learn the optimal spatial transformations for maximizing its performance. In other words, we can expect when STNs are incorporated inside a network, it would learn how much to rotate or crop (or any affine transformations) the given input images so as to make itself more invariant to these changes.

Here's a demonstration:

https://user-images.githubusercontent.com/22957388/115120399-e8084b80-9fca-11eb-97e1-c72228c3edc4.mov

Notice how the STN module is able to figure out transformations for the dataset that may be helpful to boost the end performance. Here are the original images for reference: