https://github.com/graykode/nlp-advance
Simple Paper Implementation Code about all model after Attention is all you need(Transformer)
https://github.com/graykode/nlp-advance
Last synced: 3 months ago
JSON representation
Simple Paper Implementation Code about all model after Attention is all you need(Transformer)
- Host: GitHub
- URL: https://github.com/graykode/nlp-advance
- Owner: graykode
- License: mit
- Created: 2019-04-09T13:46:05.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2019-04-09T14:36:55.000Z (about 7 years ago)
- Last Synced: 2025-08-11T20:04:46.204Z (11 months ago)
- Size: 1000 Bytes
- Stars: 2
- Watchers: 3
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
## GeneralLM
`GeneralLM` is a code that shows the trend of NLP(Natural Language Processing) after Transformer(Attention is all you need) in a very short, simple code similar to [nlp-tutorial](https://github.com/graykode/nlp-tutorial).
#### List up Paper to be implementated
- Transformer([Attention is all you need](https://arxiv.org/abs/1706.03762), 2017.06)
- Weight Transformer([Weighted Transformer Network for Machine Translation](https://arxiv.org/abs/1711.02132), 2017.11)
- ELMo([Deep contextualized word representations](https://arxiv.org/abs/1802.05365), 2018.02)
- LISA([Linguistically-Informed Self-Attention for Semantic Role Labeling](https://arxiv.org/abs/1804.08199), 2018.04)
- OpenAI GPT([Improving Language Understanding by Generative Pre-Training](), 2018.06)
- Universal Transformer([Universal Transformers](https://arxiv.org/abs/1807.03819), 2018.07)
- BERT([BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://arxiv.org/abs/1810.04805), 2018.10)
- Transformer XL([Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://arxiv.org/abs/1901.02860), 2019.01)