https://github.com/sovit-123/attention_is_all_you_need
Implementation of language model papers along with several examples [NOT ALL WRITTEN FROM SCRATCH].
https://github.com/sovit-123/attention_is_all_you_need
attention-mechanism neural-attention nlp
Last synced: 10 months ago
JSON representation
Implementation of language model papers along with several examples [NOT ALL WRITTEN FROM SCRATCH].
- Host: GitHub
- URL: https://github.com/sovit-123/attention_is_all_you_need
- Owner: sovit-123
- Created: 2023-05-21T14:41:16.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2024-10-02T15:14:20.000Z (over 1 year ago)
- Last Synced: 2025-03-25T08:38:14.732Z (about 1 year ago)
- Topics: attention-mechanism, neural-attention, nlp
- Language: Python
- Homepage:
- Size: 7.8 MB
- Stars: 12
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# attention_is_all_you_need
A repository containing the implementation of [Attention is All You Need](https://arxiv.org/abs/1706.03762) paper. Text classification, text generation, and other tasks mentioned in the paper
## Setup
```
git clone https://github.com/sovit-123/attention_is_all_you_need.git
pip install requirements.txt
pip install .
```
## Executing Examples
* All example notebooks/scripts are in the `examples` directory.
* All models are in the `attention` directory.
* All examples are self contained inside their respective directories in the `examples` folder except a few imports from the common `examples/utils` directory.
## Data
* Datasets smaller than 25 MB are in the `examples/data` directory.
* Larger datasets are either automatically downloaded or need to manually put in the root folder `input` directory.