https://github.com/zimmerrol/tf_keras_attention

Implementation and demonstration of how to apply 2D Gaussian Attention with Keras/TensorFlow
https://github.com/zimmerrol/tf_keras_attention

attention attention-mechanism deep-learning gaussian-attention keras neural-network tensorflow

Last synced: about 1 month ago
JSON representation

Implementation and demonstration of how to apply 2D Gaussian Attention with Keras/TensorFlow

• Host: GitHub
• URL: https://github.com/zimmerrol/tf_keras_attention
• Owner: zimmerrol
• License: apache-2.0
• Created: 2018-06-05T21:51:02.000Z (over 6 years ago)
• Default Branch: master
• Last Pushed: 2018-06-05T22:20:30.000Z (over 6 years ago)
• Last Synced: 2024-10-29T13:55:47.911Z (3 months ago)
• Topics: attention, attention-mechanism, deep-learning, gaussian-attention, keras, neural-network, tensorflow
• Language: Jupyter Notebook
• Homepage:
• Size: 8.79 KB
• Stars: 2
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# 2D (Visual) Attention for TensorFlow and Keras

The blog post [*Attention in Neural Networks and How to Use It*](http://akosiorek.github.io/ml/2017/10/14/visual-attention.html#mjx-eqn-att) by Adam Kosiorek shows a easy way to use Gaussian Attention on 2D data (e.g. images). However, this code can not directly be used for arbitrary batch sizes and numbers of channels in the input data. The changes required to use the described method on this kind of data are included in this repository.

The repository contains the pure *TensorFlow* based implementation and a *Keras* *Layer* which is wrapped around the code. Furthermore, a small example notebook is added.