https://github.com/zimmerrol/attention-is-all-you-need-keras
Implementation of the Transformer architecture described by Vaswani et al. in "Attention Is All You Need"
https://github.com/zimmerrol/attention-is-all-you-need-keras
attention-is-all-you-need keras neural-network nlp seq2seq transformer
Last synced: 18 days ago
JSON representation
Implementation of the Transformer architecture described by Vaswani et al. in "Attention Is All You Need"
- Host: GitHub
- URL: https://github.com/zimmerrol/attention-is-all-you-need-keras
- Owner: zimmerrol
- License: mit
- Created: 2018-07-07T08:11:36.000Z (almost 7 years ago)
- Default Branch: master
- Last Pushed: 2019-04-21T13:57:50.000Z (about 6 years ago)
- Last Synced: 2025-03-30T20:11:18.532Z (about 2 months ago)
- Topics: attention-is-all-you-need, keras, neural-network, nlp, seq2seq, transformer
- Language: Python
- Size: 3.98 MB
- Stars: 28
- Watchers: 1
- Forks: 11
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Attention is all you need (Keras) [WIP]
Implementation of the Transformer architecture described by Vaswani et al. in "Attention Is All You Need" using the [Keras Utility & Layer Collection (kulc)](https://github.com/FlashTek/keras-layer-collection).
This repository contains the code the create the model, train and evaluate it. Furthermore, it contains utility code to load a translation dataset (`en2de`) to run the experiments on it.