https://github.com/reshalfahsi/neural-machine-translation

English-To-German Neural Machine Translation Using Transformer
https://github.com/reshalfahsi/neural-machine-translation

machine-translation multi30k natural-language-processing neural-machine-translation nlp pytorch-lightning text-processing transformer

Last synced: about 2 months ago
JSON representation

English-To-German Neural Machine Translation Using Transformer

• Host: GitHub
• URL: https://github.com/reshalfahsi/neural-machine-translation
• Owner: reshalfahsi
• Created: 2023-08-15T07:46:32.000Z (over 1 year ago)
• Default Branch: master
• Last Pushed: 2023-08-16T19:21:35.000Z (over 1 year ago)
• Last Synced: 2023-08-16T20:51:20.600Z (over 1 year ago)
• Topics: machine-translation, multi30k, natural-language-processing, neural-machine-translation, nlp, pytorch-lightning, text-processing, transformer
• Language: Jupyter Notebook
• Homepage:
• Size: 252 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# English-To-German Neural Machine Translation Using Transformer

 


    

    


 


Neural Machine Translation (NMT) is a family model or an approach to solving machine translation problems through an artificial neural network, typically deep learning. In other words, the model is dispatched to translate a sequence of words from the source language to the target language. In this case, the source language would be English and the target would be German. To fabricate the model, the Transformer layers are leveraged. The NMT model is trained on the Multi30K dataset. The model is then assessed on a subset of the dataset, which is the Flickr 2016 test dataset.

## Experiment

Follow this [link](https://github.com/reshalfahsi/neural-machine-translation/blob/master/EN-DE_Neural_Machine_Translation.ipynb) to play along and explore the NMT model.

## Result

## Quantitative Result

The performance of the model in terms of cross-entropy loss and translation edit rate (TER) on the test dataset.

Metrics | Score |

------------ | ------------- |

Loss | 1.951 |

TER | 0.811 |

## Loss Curve

  
 Loss of training and validation versus the epochs. 


## Qualitative Result

Here, the NMT model's qualitative performance is associated with the Transformer's attention maps.

  
 The attention maps from each of the Transformer's heads. Almost every corresponding word pair (English-German) at each head pays attention mutually. 


## Credit

- [Language Translation With TorchText](https://pytorch.org/tutorials/beginner/torchtext_translation_tutorial.html)

- [6 - Attention is All You Need Notebook Tutorial](https://github.com/bentrevett/pytorch-seq2seq/blob/master/6%20-%20Attention%20is%20All%20You%20Need.ipynb)

- [Multi30K Dataset](https://github.com/multi30k/dataset)

- [Attention Is All You Need](https://arxiv.org/abs/1706.03762)

- [Self-Attention and Positional Encoding](https://d2l.ai/chapter_attention-mechanisms-and-transformers/self-attention-and-positional-encoding.html)

- [PyTorch Lightning](https://lightning.ai/docs/pytorch/latest/)