Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
awesome-transformer
A collection of transformer's guides, implementations and variants.
https://github.com/ictnlp/awesome-transformer
- Sequence to Sequence Learning with Neural Networks
- Neural Machine Translation by Jointly Learning to Align and Translate
- Effective Approaches to Attention-based Neural Machine Translation
- DL4MT
- Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- Neural Machine Translation in Linear Time
- Convolutional Sequence to Sequence Learning
- Neural Machine Translation of Rare Words with Subword Units
- Japanese and Korean Voice Search
- A Structured Self-attentive Sentence Embedding
- Attention is All You Need
- transformer result
- code
- code
- sacrebleu - v13a.pl` but more convenient, to calculate bleu score and report the signature as `BLEU+case.mixed+lang.de-en+test.wmt17 = 32.97 66.1/40.2/26.6/18.1 (BP = 0.980 ratio = 0.980 hyp_len = 63134 ref_len = 64399)` for easy reproduction.
- tensor2tensor transformer.py
- OpenNMT-tf
- tensor2tensor
- “变形金刚”为何强大：从模型到代码全面解析Google Tensor2Tensor系统
- get_ende_bleu.sh
- transformer_base_multistep8
- t2t issue 539
- t2t issue 444
- t2t issue 317
- Tensor2Tensor for Neural Machine Translation
- OpenNMT-py
- OpenNMT-py doc - py opts.py](https://github.com/OpenNMT/OpenNMT-py/blob/master/onmt/opts.py)
- corpus preprocessed by OpenNMT - trainingdata/wmt_ende_sp_model.tar.gz). Note that the preprocess procedure includes tokenization, bpe/word-piece operation(here using [sentencepiece](https://github.com/google/sentencepiece) powered by Google which implements word-piece algorithm), see [OpenNMT-tf script](https://github.com/OpenNMT/OpenNMT-tf/blob/master/scripts/wmt/prepare_data.sh) for more details.
- sentencepiece - encoded style, so we need a decoding procedure to obtain a detokenized plain prediction.
- bpe-version - nmt](https://github.com/rsennrich/subword-nmt) for bpe encoding and decoding.
- OpenNMT-py FAQ
- OpenNMT-py issue
- OpenNMT: Open-Source Toolkit for Neural Machine Translation
- fairseq-py
- doc - -update-freq` when training to accumulate every `N` batches loss to backward, so it's `8` for 1 GPU, `2` for 4 GPUs and so on.
- the preprocessed WMT'16 EN-DE data provided by Google
- fairseq-py example
- fairseq-py issue
- Marian
- Training Tips for the Transformer Model
- The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation
- Scaling Neural Machine Translation
- Universal Transformer
- Self-Attention with Relative Position Representations
- Improving Language Understanding by Generative Pre-Training
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- Yong Shan
- Jinchao Zhang
