https://github.com/suyash/mlt
Multilingual Neural Machine Translation using Transformers with Conditional Normalization.
https://github.com/suyash/mlt
machine-translation natural-language-processing transformers translation
Last synced: about 1 month ago
JSON representation
Multilingual Neural Machine Translation using Transformers with Conditional Normalization.
- Host: GitHub
- URL: https://github.com/suyash/mlt
- Owner: suyash
- License: mit
- Created: 2019-09-14T07:45:26.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2023-03-24T22:33:04.000Z (about 2 years ago)
- Last Synced: 2025-03-25T01:43:49.520Z (about 2 months ago)
- Topics: machine-translation, natural-language-processing, transformers, translation
- Language: Jupyter Notebook
- Homepage:
- Size: 30.3 KB
- Stars: 18
- Watchers: 2
- Forks: 3
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# mlt
Multilingual Neural Machine Translation using Transformers with Conditional Normalization.
Transformers with One-to-Many, Many-to-One, Many-to-Many translation, with another demo for fine-tuning, by replacing regular layer norm in both encoder and decoder with Conditional Norm.
Conditional Norm is a technique from Style Transfer, used to train a single network with multiple styles, as well as mix styles, for example
Broader set of demos at https://github.com/suyash/stylizer/blob/master/image_stylization_demo.ipynb
The goal here is similar, make the rest of the network learn a common representation, while making the normalization parameters learn language specific semantics.
The One-to-Many and Many-to-One models are trained for English to French, German, Italian and Spanish Translation and Vice-Versa.
The Many to Many model is trained on English-French, French-English, English-German and German-English.
The image stylization paper specifies how a N-style network can pick up an N+1th style through fine-tuning an existing model. Similarly, I fine-tune my Many-to-Many model to pick up Portuguese.
I also provide demos for Few-Shot and Zero-Shot translation.
Have provided `SavedModel`s, as well as standalone notebooks that run in Google Colaboratory.
Code and Demos for the following use-cases
#### One to Many
> Demo: https://colab.research.google.com/github/suyash/mlt/blob/master/one_to_many_demo.ipynb
>
> Code: https://github.com/suyash/mlt/blob/master/mlt/one_to_many.py
>
> Model: https://dl.dropboxusercontent.com/s/bssvce8t00zjwxj/one_to_many.zip?dl=0
English to French, German, Spanish and Italian translation.
This uses split vocabularies of size 8192 each.
#### Many to One
> Demo: https://colab.research.google.com/github/suyash/mlt/blob/master/many_to_one_demo.ipynb
>
> Code: https://github.com/suyash/mlt/blob/master/mlt/many_to_one.py
>
> Model: https://dl.dropboxusercontent.com/s/xmroc5u6k17t5ds/many_to_one.zip?dl=0
French, German, Spanish and Italian to English translation.
This uses split vocabularies of size 8192 each.
#### Many to Many
> Demo: https://colab.research.google.com/github/suyash/mlt/blob/master/many_to_many_demo.ipynb
>
> Code: https://github.com/suyash/mlt/blob/master/mlt/many_to_many.py
>
> Model: https://dl.dropboxusercontent.com/s/kzxryp6snnx5dw6/many_to_many.zip?dl=0
English, French and German on both ends.
This uses a shared vocabulary of size 40,960.
#### Many to Many Fine-Tuned
> Demo: https://colab.research.google.com/github/suyash/mlt/blob/master/many_to_many_fine_tune_demo.ipynb
>
> Code: https://github.com/suyash/mlt/blob/master/mlt//many_to_many_fine_tune.py
>
> Model: https://dl.dropboxusercontent.com/s/3ixjojbpho2emd8/many_to_many_fine_tune.zip?dl=0
Portuguese added from ted_hrlr_translate to the Many to Many model.