https://github.com/cyberzhg/keras-position-wise-feed-forward

Feed forward layer implemented in Keras
https://github.com/cyberzhg/keras-position-wise-feed-forward

attention keras layer

Last synced: about 1 month ago
JSON representation

Feed forward layer implemented in Keras

• Host: GitHub
• URL: https://github.com/cyberzhg/keras-position-wise-feed-forward
• Owner: CyberZHG
• License: mit
• Archived: true
• Created: 2018-11-08T08:14:57.000Z (almost 6 years ago)
• Default Branch: master
• Last Pushed: 2022-01-22T10:04:37.000Z (almost 3 years ago)
• Last Synced: 2024-09-30T16:08:11.283Z (about 1 month ago)
• Topics: attention, keras, layer
• Language: Python
• Homepage: https://pypi.org/project/keras-position-wise-feed-forward/
• Size: 14.6 KB
• Stars: 8
• Watchers: 4
• Forks: 5
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Keras Position-Wise Feed Forward

[![Version](https://img.shields.io/pypi/v/keras-position-wise-feed-forward.svg)](https://pypi.org/project/keras-position-wise-feed-forward/)

![License](https://img.shields.io/pypi/l/keras-position-wise-feed-forward.svg)

Implementation of position-wise feed forward layer in the paper: [Attention is All You Need](https://arxiv.org/pdf/1706.03762.pdf)

## Install

```bash

pip install keras-position-wise-feed-forward

```

## Usage

```python

from tensorflow import keras

from keras_position_wise_feed_forward import FeedForward

input_layer = keras.layers.Input(shape=(None, 32))

feed_forward_layer = FeedForward(units=128)(input_layer)

model = keras.models.Model(inputs=input_layer, outputs=feed_forward_layer)

model.compile(optimizer='adam', loss='mse')

model.summary()

```