https://github.com/reshalfahsi/neural-machine-translation
English-To-German Neural Machine Translation Using Transformer
https://github.com/reshalfahsi/neural-machine-translation
machine-translation multi30k natural-language-processing neural-machine-translation nlp pytorch-lightning text-processing transformer
Last synced: 4 months ago
JSON representation
English-To-German Neural Machine Translation Using Transformer
- Host: GitHub
- URL: https://github.com/reshalfahsi/neural-machine-translation
- Owner: reshalfahsi
- Created: 2023-08-15T07:46:32.000Z (almost 2 years ago)
- Default Branch: master
- Last Pushed: 2023-08-17T00:14:51.000Z (almost 2 years ago)
- Last Synced: 2025-01-15T07:14:44.044Z (6 months ago)
- Topics: machine-translation, multi30k, natural-language-processing, neural-machine-translation, nlp, pytorch-lightning, text-processing, transformer
- Language: Jupyter Notebook
- Homepage:
- Size: 352 KB
- Stars: 1
- Watchers: 3
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# English-To-German Neural Machine Translation Using Transformer
Neural Machine Translation (NMT) is a family model or an approach to solving machine translation problems through an artificial neural network, typically deep learning. In other words, the model is dispatched to translate a sequence of words from the source language to the target language. In this case, the source language would be English and the target would be German. To fabricate the model, the Transformer layers are leveraged. The NMT model is trained on the Multi30K dataset. The model is then assessed on a subset of the dataset, which is the Flickr 2016 test dataset.
## Experiment
Follow this [link](https://github.com/reshalfahsi/neural-machine-translation/blob/master/EN-DE_Neural_Machine_Translation.ipynb) to play along and explore the NMT model.
## Result
## Quantitative Result
The performance of the model in terms of cross-entropy loss and translation edit rate (TER) on the test dataset.
Metrics | Score |
------------ | ------------- |
Loss | 1.951 |
TER | 0.811 |
## Loss Curve
Loss of training and validation versus the epochs.
## Qualitative Result
Here, the NMT model's qualitative performance is associated with the Transformer's attention maps.
The attention maps from each of the Transformer's heads. Almost every corresponding word pair (English-German) at each head pays attention mutually.
## Credit
- [Language Translation With TorchText](https://pytorch.org/tutorials/beginner/torchtext_translation_tutorial.html)
- [6 - Attention is All You Need Notebook Tutorial](https://github.com/bentrevett/pytorch-seq2seq/blob/master/6%20-%20Attention%20is%20All%20You%20Need.ipynb)
- [Multi30K Dataset](https://github.com/multi30k/dataset)
- [Attention Is All You Need](https://arxiv.org/abs/1706.03762)
- [Self-Attention and Positional Encoding](https://d2l.ai/chapter_attention-mechanisms-and-transformers/self-attention-and-positional-encoding.html)
- [PyTorch Lightning](https://lightning.ai/docs/pytorch/latest/)