Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.
Awesome Lists | Featured Topics | Projects
https://github.com/ashishpatel26/Treasure-of-Transformers

💁 Awesome Treasure of Transformers Models for Natural Language processing contains papers, videos, blogs, official repo along with colab Notebooks. 🛫☑️
https://github.com/ashishpatel26/Treasure-of-Transformers
List: Treasure-of-Transformers
awesome bert jax language-model language-models model-hub natural-language-generation natural-language-processing natural-language-understanding nlp nlp-library pretrained-models python pytorch pytorch-transformers seq2seq speech-recognition tensorflow transformer
Last synced: 3 months ago
JSON representation
💁 Awesome Treasure of Transformers Models for Natural Language processing contains papers, videos, blogs, official repo along with colab Notebooks. 🛫☑️
Host: GitHub
URL: https://github.com/ashishpatel26/Treasure-of-Transformers
Owner: ashishpatel26
License: mit
Created: 2021-12-13T12:03:54.000Z (about 3 years ago)
Default Branch: main
Last Pushed: 2024-07-16T08:21:56.000Z (6 months ago)
Last Synced: 2024-10-15T03:01:29.408Z (3 months ago)
Topics: awesome, bert, jax, language-model, language-models, model-hub, natural-language-generation, natural-language-processing, natural-language-understanding, nlp, nlp-library, pretrained-models, python, pytorch, pytorch-transformers, seq2seq, speech-recognition, tensorflow, transformer
Language: Jupyter Notebook
Homepage: https://github.com/ashishpatel26/Treasure-of-Transformers.git
Size: 370 KB
Stars: 917
Watchers: 29
Forks: 202
Open Issues: 1
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project

100-AI-Machine-learning-Deep-learning-Computer-vision-NLP - 👆
awesome-rainmana - ashishpatel26/Treasure-of-Transformers - 💁 Awesome Treasure of Transformers Models for Natural Language processing contains papers, videos, blogs, official repo along with colab Notebooks. 🛫☑️ (Jupyter Notebook)
README

        # Awesome Treasure of Transformers Models Collection

![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/tot.jpg)

![fw.jpg](https://github.com/ashishpatel26/Treasure-of-Transformers/blob/main/images/fw.jpg?raw=true)

---

###  🧑‍💻👩‍💻Collection of All NLP Deep learning algorithm list with Code 🧑‍💻👩‍💻![Jupyter Notebook](https://img.shields.io/badge/jupyter-%23FA0F00.svg?style=for-the-badge&logo=jupyter&logoColor=white)

---

| Sr No | Algorithm Name 
| ----- | --------------------------------------------- 
| 1     | [GPT-Neo](https://github.com/EleutherAI/gpt-neo) 
| 2     | [Transformer](https://arxiv.org/abs/1706.03762v5) 
| 3     | [BERT](https://arxiv.org/abs/1810.04805v2) 
| 4     | [GPT](https://s3-us-west-2.amazonaws.com/open 
| 5     | [Universal Transformer](https://arxiv.org/abs 
| 6     | [T-D](https://arxiv.org/abs/1801.10198v1) 
| 7     | [GPT-2](https://d4mucfpksywv.cloudfront.net/b 
| 8     | [T5](https://arxiv.org/abs/1910.10683v3) 
| 9     | [BART](https://arxiv.org/abs/1910.13461v1) 
| 10    | [XLNet](https://arxiv.org/abs/1906.08237v2) 
| 11    | [ALBERT](https://arxiv.org/abs/1909.11942v6) 
| 12    | [Distil-BERT](https://arxiv.org/abs/1910.01108v4) 
| 13    | [Transformer-XL](https://arxiv.org/abs/1901.02860v3) 
| 14    | [XLM](https://arxiv.org/abs/1901.07291v1) 
| 15    | [ViLBERT](https://arxiv.org/abs/1908.02265v1) 
| 16    | [Sparse Transformer](https://arxiv.org/abs/1904.10509v1) 
| 17    | [Levenshtein Transformer](https://arxiv.org/a 
| 18    | [CTRL](https://arxiv.org/abs/1909.05858v2) 
| 19    | [VideoBERT](https://arxiv.org/abs/1904.01766v2) 
| 20    | [Compressive Transformer](https://arxiv.org/a 
| 21    | [CuBERT](https://arxiv.org/abs/2001.00059v3) 
| 22    | [BP-Transformer](https://arxiv.org/abs/1911.04070v1) 
| 23    | [Adaptively Sparse Transformer](https://arxiv 
| 24    | [Sandwich Transformer](https://arxiv.org/abs/1911.03864v2) 
| 25    | [FSMT](https://arxiv.org/abs/1907.06616) 
| 26    | [LXMERT](https://arxiv.org/abs/1908.07490) 
| 27    | [VisualBERT](https://arxiv.org/pdf/1908.03557) 
| 28    | [GPT-3](https://arxiv.org/abs/2005.14165v4) 
| 29    | [ELECTRA](https://arxiv.org/abs/2003.10555v1) 
| 30    | [Electric](https://arxiv.org/abs/2012.08561v1) 
| 31    | [LongFormer](https://arxiv.org/abs/2004.05150v2) 
| 32    | [mBART](https://arxiv.org/abs/2001.08210v2) 
| 33    | [Performer](https://arxiv.org/abs/2009.14794v3) 
| 34    | [ETC](https://arxiv.org/abs/2004.08483v5) 
| 35    | [CodeBERT](https://arxiv.org/abs/2002.08155v4) 
| 36    | [mT5](https://arxiv.org/abs/2010.11934v3) 
| 37    | [Reformer](https://arxiv.org/abs/2001.04451v2) 
| 38    | [DeBERTa & DeBERTa-v2](https://arxiv.org/ 
| 39    | [Linformer](https://arxiv.org/abs/2006.04768v3) 
| 40    | [RAG](https://arxiv.org/abs/2005.11401v4) 
| 41    | [ProphetNet](https://arxiv.org/abs/2001.04063v3) 
| 42    | [BigBird](https://arxiv.org/abs/2007.14062v2) 
| 43    | [PLATO-2](https://arxiv.org/abs/2006.16779v4) 
| 44    | [Routing Transformer](https://arxiv.org/abs/2003.05997v5) 
| 45    | [DeeBERT](https://arxiv.org/abs/2004.12993v1) 
| 46    | [DynaBERT](https://arxiv.org/abs/2004.04037v2) 
| 47    | [TernaryBERT](https://arxiv.org/abs/2009.12812v3) 
| 48    | [MobileBERT](https://arxiv.org/abs/2004.02984v2) 
| 49    | [Bort](https://arxiv.org/abs/2010.10499v2) 
| 50    | [DeLighT](https://arxiv.org/abs/2008.00623v2) 
| 51    | [PAR Transformer](https://arxiv.org/abs/2009.04534v3) 
| 52    | [ConvBERT](https://arxiv.org/abs/2008.02496v3) 
| 53    | [IB-BERT](https://arxiv.org/abs/2004.02984v2) 
| 54    | [MacBERT](https://arxiv.org/abs/2004.13922v2) 
| 55    | [RealFormer](https://arxiv.org/abs/2012.11747v3) 
| 56    | [Sinkhorn Transformer](https://arxiv.org/abs/2002.11296v1) 
| 57    | [SongNet](https://arxiv.org/abs/2004.08022v2) 
| 58    | [Funnel Transformer](https://arxiv.org/abs/2006.03236v1) 
| 59    | [SC-GPT](https://arxiv.org/abs/2002.12328v1) 
| 60    | [SMITH](https://arxiv.org/abs/2004.12297v2) 
| 61    | [BinaryBERT](https://arxiv.org/abs/2012.15701v2) 
| 62    | [SqueezeBERT](https://arxiv.org/abs/2006.11316v1) 
| 63    | [Feedback Transformer](https://arxiv.org/abs/2002.09402v3) 
| 64    | [CamemBERT](https://arxiv.org/abs/1911.03894) 
| 65    | [CPM](https://arxiv.org/abs/2012.00413) 
| 66    | [DialoGPT](https://arxiv.org/abs/1911.00536) 
| 67    | [DPR](https://arxiv.org/abs/2004.04906) 
| 68    | [FlauBERT](https://arxiv.org/abs/1912.05372) 
| 69    | [HerBERT](https://www.aclweb.org/anthology/20 
| 70    | [LayoutLM](https://arxiv.org/abs/1912.13318) 
| 71    | [LED](https://arxiv.org/abs/2004.05150) 
| 72    | [LUKE](https://arxiv.org/abs/2010.01057) 
| 73    | [M2M100](https://arxiv.org/abs/2010.11125) 
| 74    | [MBart and MBart-50](https://arxiv.org/abs/2001.08210) 
| 75    | [MegatronBERT](https://arxiv.org/abs/1909.08053) 
| 76    | [MegatronGPT2](https://arxiv.org/abs/1909.08053) 
| 77    | [MPNet](https://arxiv.org/abs/2004.09297) 
| 78    | [Pegasus](https://arxiv.org/pdf/1912.08777.pdf) 
| 79    | [PhoBERT](https://arxiv.org/abs/2003.00744v3) 
| 80    | [QDQBERT](https://arxiv.org/abs/2004.09602) 
| 81    | [RemBERT](https://arxiv.org/abs/2010.12821) 
| 82    | [RetriBERT](https://yjernite.github.io/lfqa.html) 
| 83    | [Speech2Text](https://arxiv.org/abs/2010.05171) 
| 84    | [T5v1.1](https://arxiv.org/abs/2002.05202) 
| 85    | [TAPAS](https://arxiv.org/abs/2004.02349) 
| 86    | [Wav2Vec2](https://arxiv.org/abs/2006.11477) 
| 87    | [XLM-ProphetNet](https://arxiv.org/abs/2001.04063) 
| 88    | [XLM-RoBERTa](https://arxiv.org/abs/1911.02116) 
| 89    | [XLSR-Wav2Vec2](https://arxiv.org/abs/2006.13979) 
| 90    | [Switch Transformer](https://arxiv.org/abs/2101.03961v1) 
| 91    | [TNT](https://arxiv.org/abs/2103.00112v3) 
| 92    | [Adaptive Span Transformer](https://arxiv.org 
| 93    | [Primer](https://arxiv.org/abs/2109.08668v1) 
| 94    | [Charformer](https://arxiv.org/abs/2106.12672v2) 
| 95    | [MATE](https://arxiv.org/abs/2109.04312v1) 
| 96    | [Nystromformer](https://arxiv.org/abs/2102.03902v3) 
| 97    | [Subformer](https://arxiv.org/abs/2101.00234v3) 
| 98    | [ESACL](https://arxiv.org/abs/2108.11992v1) 
| 99    | [PermuteFormer](https://arxiv.org/abs/2109.02377v2) 
| 100   | [NormFormer](https://arxiv.org/abs/2110.09456v2) 
| 101   | [Fastformer](https://arxiv.org/abs/2108.09084v6) 
| 102   | [AutoTinyBERT](https://arxiv.org/abs/2107.13686v1) 
| 103   | [EGT](https://arxiv.org/abs/2108.03348v2) 
| 104   | [Chinese Pre-trained Unbalanced   Transformer 
| 105   | [GANFormer](https://arxiv.org/abs/2103.01209v3) 
| 106   | [ClipBERT](https://arxiv.org/abs/2102.06183v1) 
| 107   | [CodeT5](https://arxiv.org/abs/2109.00859v1) 
| 108   | [I-BERT](https://paperswithcode.com/method/i-bert) 
| 109   | [ByT5](https://arxiv.org/abs/2105.13626) 
| 110   | [CANINE](https://arxiv.org/abs/2103.06874) 
| 111   | [FNet](https://arxiv.org/abs/2105.03824) 
| 112   | [LayoutLMV2](https://arxiv.org/abs/2012.14740) 
| 113   | [LayoutXLM](https://arxiv.org/abs/2104.08836) 
| 114   | [GPT-J](https://huggingface.co/docs/transform 
| 115   | [Hubert](https://arxiv.org/abs/2106.07447) 
| 116   | [Perceiver](https://arxiv.org/abs/2107.14795) 
| 117   | [RoFormer](https://arxiv.org/pdf/2104.09864v1.pdf) 
| 118   | [SegFormer](https://arxiv.org/abs/2105.15203) 
| 119   | [SEW](https://arxiv.org/abs/2109.06870) 
| 120   | [SEW-D](https://arxiv.org/abs/2109.06870) 
| 121   | [Speech2Text2](https://arxiv.org/abs/2104.06678) 
| 122   | [Splinter](https://arxiv.org/abs/2101.00438) 
| 123   | [TrOCR](https://arxiv.org/abs/2109.10282) 
| 124   | [UniSpeech](https://arxiv.org/abs/2101.07597) 
| 125   | [UniSpeech-SAT](https://arxiv.org/abs/2110.05752) 
| 126   | [MarianMT](https://huggingface.co/docs/transf

| Year | Blog                                                         | Video                                                        | Official Repo                                                | Code                                                         | --------------- | ---- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3rYanJk) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/6MI0f6YjJIk) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/EleutherAI/gpt-neo) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/EleutherAI/GPTNeo/blob/master/GPTNeo_example_notebook.ipynb) | | 2017 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3DNsrIp) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/iDulhoQ2pro) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/tensorflow/models/tree/master/official/nlp/transformer) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/bentrevett/pytorch-seq2seq/blob/master/6%20-%20Attention%20is%20All%20You%20Need.ipynb) | | 2018 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3pPV8PS) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/7kLi8u2dJz0) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/bert) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/NielsRogge/Transformers-Tutorials/blob/master/BERT/Custom_Named_Entity_Recognition_with_BERT_only_first_wordpiece.ipynb) | ai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf) | 2018 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3ENPuEn) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/9ebPNEHRwXU) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/huggingface/transformers) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/keras-team/keras-io/blob/master/examples/generative/ipynb/text_generation_with_miniature_gpt.ipynb) | /1807.03819v3)  | 2018 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3lRi6VG) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/tensorflow/tensor2tensor) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tensorflow/tensor2tensor/blob/master/tensor2tensor/notebooks/hello_t2t.ipynb) | | 2018 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3F17Ba5) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/__ALQCud-iA) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/tensorflow/tensor2tensor) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tensorflow/tensor2tensor/blob/master/tensor2tensor/notebooks/hello_t2t.ipynb) | etter-language-models/language-models.pdf) | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3rXapBi) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/X-B3nAN7YRM) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/openai/gpt-2) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/02_how_to_generate.ipynb) | | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3yjbrZf) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/jgKj-7v2UYU) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/text-to-text-transfer-transformer) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/NielsRogge/Transformers-Tutorials/blob/master/T5/Fine_tune_CodeT5_for_generating_docstrings_from_Ruby_code.ipynb) | | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3yead1D) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/Xz4fiQNmGSA) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/facebookresearch/GENRE) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mrm8488/shared_colab_notebooks/blob/master/BART_with_JS_UI.ipynb) | | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3EGEHM5) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/8J8KkuncAmY) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/zihangdai/xlnet) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1YXojI6USFGV3498Pygllv0z8vTMVIiEt) | | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3lW9L34) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/Lwtj2yUAMgI) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/ALBERT) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/google-research/albert/blob/master/albert_glue_fine_tuning_tutorial.ipynb) | | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3GPQKHN) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/cCs8exFrGE8) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)]() | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]() | | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3lW6qkC) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/mqsRZAJav70) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/kimiyoung/transformer-xl) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/lab-ml/nn/blob/master/labml_nn/transformers/xl/experiment.ipynb) | | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/31M5BDH) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/caZLVcJqsqo) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/facebookresearch/XLM) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Tikquuss/meta_XLM/blob/master/notebooks/demo/tuto.ipynb) | | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/31M5KqJ) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/dd7nE4nbxN0) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/facebookresearch/vilbert-multi-task/) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/facebookresearch/vilbert-multi-task/blob/9d645085b54fac6a781257133edabefdfb23e547/demo.ipynb) | | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3ELosND) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/jaqjcHmCBfY) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/openai/sparse_attention) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://github.com/ptillet/torch-blocksparse/blob/master/examples/simple.py) | bs/1905.11006v2) | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3rUU4wQ) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/FtUbMG3rlFs) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/pytorch/fairseq) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]() | | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/33pJRhX) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/q5zw2Wh4Rdo) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/salesforce/ctrl) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1nDh3ayRPJGK5ciPO2D3TFkYZFqclBWHY) | | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/31PHkN2) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/EkuyLsJX4QM) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/MDSKUL/MasterProject) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://github.com/ammesatyajit/VideoBERT) | bs/1911.05507v1) | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3rX1q2L) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/0TMKCQiT1T0) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/deepmind/pg19) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://github.com/lucidrains/compressive-transformer-pytorch) | | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3lXcxVH) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/bCPeg0Tp64s) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/google-research/tree/master/cubert) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ai-center-kth/cuBERT-source-code-clustering/blob/main/src/evaluation.ipynb) | | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3yuzqFj) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/yzh119/BPT) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]() | .org/abs/1909.00015v2) | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3lXLbyG) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/_pYxa50HTBw) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/deep-spin/entmax) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/prajjwal1/adaptive_transformer/blob/8f062a7d5c6573ebdb1aa2b57113db918156b15a/nbs/adaptive_train.ipynb) | | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/31V686t) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/EM8xFAjtZUQ) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/ofirpress/sandwich_transformer) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://github.com/lucidrains/x-transformers) | | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3oIFarp) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/pytorch/fairseq/tree/main/examples/wmt19) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://huggingface.co/transformers/v3.5.1/model_doc/fsmt.html) | | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3oJq19i) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/airsplay/lxmert) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/18TyuMfZYlgQ_nXo-tr8LCnzUaoX0KS-h?usp=sharing) | | 2019 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3s0AgIq) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/Lhi1UFsXvpk) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/uclanlp/visualbert) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/uclanlp/visualbert/blob/e49b61906dff12f1b2992226ae84c538fcf438e4/unsupervised_visualbert/data/nlvr2/nlvr/nlvr2/NLVR2BiasAnalysis.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3oIFwhJ) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/SboKeK6FFHQ) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/openai/gpt-3) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/rafajak/gpt3_examples/blob/master/GPT3_search_example.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3lUZGDy) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/p9c7SoD4AMc) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/electra) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1DiOwhRjQbtYRgFWG7e3dybcXJsZcu86l) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/31SfM9E) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/electra) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]() | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3oJ03mh) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/_8KNb5iqblE) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/allenai/longformer) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/patil-suraj/Notebooks/blob/master/longformer_qa_training.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3EPC0Ii) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/oO7k5lH8Oe8) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/pytorch/fairseq/tree/master/examples/mbart) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/sdhilip200/Machine-Translation-using-mBART-50-and-Hugging-Face/blob/main/Machine_Translation.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3pLxvYW) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/xJrKIPwVwGM) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/google-research) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1ICW0DpXfUuVYsnNkt1DHwUyyTduHHvE3?usp=sharing) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3rWIy3W) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/TyydOhj28A4) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/google-research/tree/master/etcmodel) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]() | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3pPuIhc) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/YmAXluUDPPI) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/microsoft/CodeBERT) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/graykode/commit-autosuggestions/blob/579d7a34f3ebb735d2a9f13afc74eeb66c0d7427/commit_autosuggestions.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3DJ76jh) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)]() | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ejmejm/multilingual-nmt-mt5/blob/main/nmt_full_version.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3pPSTMn) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/i4H0kjxrias) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google/trax/tree/master/trax/models/reformer) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/patrickvonplaten/notebooks/blob/master/PyTorch_Reformer.ipynb) | abs/2006.03654v6)   | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3GyaT4F) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/_c6A33Fg5Ns) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/microsoft/DeBERTa) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/musthofasyarifudin/Deberta-Language-Modeling-For-NLP/blob/main/MLM_Deberta_Base.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3DMx5GH) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/-_2AF9Lhweo) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/kuixu/Linear-Multihead-Attention) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1zHenqau3rMo3oS_7EisfGsahSs-1_sok?usp=sharing) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/30f4ig2) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/dzChvuZI6D4) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://huggingface.co/docs/transformers/model_doc/rag?utm_source=catalyzex.com) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ipavlopoulos/modern_nlp/blob/f2a7af7e9de68d505dd1585c04cc51f30433122b/Modern_NLP_S5.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/31RisVr) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/9Hoo_qdjSio) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/microsoft/ProphetNet) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/11uPpxZzoTofnHZsyDPaJr5dFKsgOmTD9#scrollTo=j5CRrgNFc_Jq) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3pPTSMz) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/WVPE62Gk3EM) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/bigbird) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/vasudevgupta7/bigbird/blob/main/notebooks/evaluate_nq.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3DF7hMA) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/0zlRRC0jPf4) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/PaddlePaddle/Knover) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/examples/dialogue/plato-2) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3IGcabW) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/lucidrains/routing-transformer) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1sw1Hjx3EllhKZh4nhJ3TIZ978HjKVUFQ?usp=sharing) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3GB1Ks8) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/castorini/DeeBERT) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/castorini/DeeBERT/blob/3efc01db07a0bed88e19551992b8378994195d1c/notebooks/Comparing-TF-and-PT-models-SQuAD.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3EGKyRz) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/UuGYwjCOGQo) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/DynaBERT) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/DynaBERT) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3dECdlu) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://papertalk.org/papertalks/11743) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/TernaryBERT) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/TernaryBERT) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3yirFlJ) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/google-research) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/sayakpaul/BERT-for-Mobile/blob/master/MobileBERT_Keras.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3oMixme) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/alexa/bort) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]() | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3DRXQtq) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/-AVqR2qZHb4) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/sacmehta/delight) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]() | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3s26Nhk) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/Jmkernes/PAR-Transformer-XL) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://github.com/Jmkernes/PAR-Transformer-XL/blob/ee2c2bd5cd672eddbe06d68614298de8283c5b2f/colab.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3pKkqPi) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://bit.ly/321uKKx) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/yitu-opensource/ConvBert) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1WIu2Cc1C8E7ayZBzEmpfd5sXOhe7Ehhz?usp=sharing) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3dKRvVT) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/google-research) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/lordtt13/transformers-experiments/blob/5ea4537ee0662fb6ca2ba736a50f4894197eec60/Custom%20Tasks/emo-mobilebert.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3GTTdkv) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/ymcui/MacBERT) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/shibing624/nlp-tutorial/blob/c9c00b2160397248381203658194a63044e7d7d5/03_language_model/03_Bert%E5%AE%8C%E5%BD%A2%E5%A1%AB%E7%A9%BA.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3EQSoYW) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://bit.ly/3DMWyzG) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/google-research) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://github.com/jaketae/realformer) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3yjJOiV) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/lucidrains/sinkhorn-transformer) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1Eej8U4pP5ldZOz3tHwpoBFgmQqLhQLUq) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3ym2hLH) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/lipiji/SongNet) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]() | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/31WfMWc) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/QsIcEqGriGg) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/laiguokun/Funnel-Transformer) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/iIsunnyIi/Funnel-Transformer/blob/main/Funnel_IntentRecognition.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/31VCldV) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/NWHqFbgHqng) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/pengbaolin/SC-GPT) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]() | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3dHyOm2) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/VHsYJu9VzyI) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/google-research/tree/master/smith) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]() | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3dNguYx) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/BinaryBERT) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]() | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3EPfqiT) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/kPMaEYSywdI) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_squeezebert.py) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/urainshah/Use-of-transformers-hugging-face/blob/main/fakenews_squeezebert1.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/31VDQJ5) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/-QH8fRhqFHM) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/facebookresearch/transformer-sequential) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/rajaswa/feedback-and-memory-in-transformers/blob/main/Feedback_and_Memory_in_Transformers.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3yj1AD0) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://huggingface.co/docs/transformers/model_doc/camembert) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/SimonLembeye/french-sentence-embedding/blob/master/sentence_transformers_camembert.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/30jaW4W) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/dFTG3ohAcso) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](github.com/TsinghuaAI/CPM-1-Generate) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]() | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3dKCT8T) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/Zo679MYoJns) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/microsoft/DialoGPT) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1PslHE4Rl4RqSa20s7HEp0ZKITBir6ezE) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3oLM0g0) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/DBsxUSUhfRg) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/facebookresearch/DPR) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/deepset-ai/haystack/blob/master/tutorials/Tutorial1_Basic_QA_Pipeline.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3GENX40) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://bit.ly/3GElYkU) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/getalp/Flaubert) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Rushikesh10/FlauBERT-Text-Summarization/blob/main/FlauBERT%20Colab%20Notebook.ipynb) | 20.acl-main.111.pdf) | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3GyppJM) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://bit.ly/3dOIR92) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/allegro/klejbenchmark-baselines) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/klinkeklinke/TransformersTransferLearning/blob/32aae9ee170a9c7fe4037a9dc7d3cee150805c38/transfer_learning_nlp.ipynb#scrollTo=c4cf3720) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3rWIukZ) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/zazN_yFWlLM) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/microsoft/unilm) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/NielsRogge/Transformers-Tutorials/blob/master/LayoutLM/Fine_tuning_LayoutLMForTokenClassification_on_FUNSD.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3pTAGh4) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/_8KNb5iqblE) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/allenai/longformer) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/allenai/longformer/blob/master/scripts/convert_model_to_long.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3rYxxiC) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/oho-i5Ws07g) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/studio-ousia/luke) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/NielsRogge/Transformers-Tutorials/blob/master/LUKE/Supervised_relation_extraction_with_LukeForEntityPairClassification.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/33jc6P3) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://bit.ly/30u83yw) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/pytorch/fairseq) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/harveenchadha/bol/blob/main/demos/gr/hindi/Speech_ASR_plus_Translation_Hindi.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3lZelh1) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/oO7k5lH8Oe8) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/pytorch/fairseq/tree/master/examples/mbart) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/sdhilip200/Machine-Translation-using-mBART-50-and-Hugging-Face/blob/main/Machine_Translation.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3IMtdZO) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/wBgpMf_KQVw) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/NVIDIA/Megatron-LM) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/kssteven418/Q-ASR/blob/0ad68cdeddbd5a4c8edba6a0876470ec48186584/tutorials/nlp/Relation_Extraction-BioMegatron.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/33tQMGV) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)]() | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/877152223/BugInjector/blob/4cf14e237b97a724edbb4b1300170c3cc832fc2f/BugInjector-GPT2.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3pPD6gO) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/WS1uVMGhlWQ) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/microsoft/MPNet) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/04mayukh/Neutral-tweet-identification-on-OLID-using-Neutrosophy/blob/45bbd6c57054337e8af6429554f2dda4aef266d1/Models/MPNet.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3mkVsWb) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/naRdmLvlEzE) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/pegasus) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/CoGian/pegasus_demo_huggingface/blob/master/pegasus_demo_huggingface.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/31Kahul) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/7d--8leiZgA) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://paperswithcode.com/paper/phobert-pre-trained-language-models-for#code) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ncthuan/text_cls_phobert/blob/master/phobert_clf.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3pXReVs) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/mDefFU7rbV0) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/huggingface/pytorch-transformers) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ELS-RD/transformer-deploy/blob/814477ce3b3279cff8f8c3947294119f399903b6/demo/quantization_end_to_end.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/31KaNIN) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/JK1002497318/RemBERT) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/kldarek/chaii/blob/194f92b808cb1c81b70b889ab8d061344ef34ef9/training_notebooks/REMBERT_pretraining77.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3s0Ta1O) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)]() | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/robhaslinger/EALI5-redux/blob/50ce17c33c333a0d9781f00e29888f2844ad6aae/notebooks/qa_step_by_step.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/31ZJWI7) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/a-OwY2SqKQI) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/pytorch/fairseq/tree/master/examples/speech_to_text) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]() | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3pSWCJm) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/text-to-text-transfer-transformer/) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://github.com/lucidrains/x-transformers) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3lZtfnE) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/ZnuEOQrT4h0) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/tapas) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/NielsRogge/Transformers-Tutorials/blob/master/TAPAS/Evaluating_TAPAS_on_the_Tabfact_test_set.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3GJBADT) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/aUSXvoWfy3w) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/chuachinhon/wav2vec2_transformers) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/chuachinhon/wav2vec2_transformers/blob/main/notebooks/2.2_wav2vec2_poetry_alt.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3EQ6KbP) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://huggingface.co/docs/transformers/model_doc/xlmprophetnet) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Biswajit7890/ADV-DL-NLP-Notebooks/blob/2258022d0c06599317ef3db3d53ef8d4826fd0c2/custom_language_translation_Training_with_XLMProphetNet_.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3IHnHI3) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://www.youtube.com/watch?v=Ot6A3UFY72c&ab_channel=AISuisse) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/facebookresearch/cc_net) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/edoost/pert/blob/29fc78bc36110ea031083a3e7294ce9135026ee1/pos_xlmroberta_multi.ipynb) | | 2020 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/33qEO0D) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://bit.ly/3DSHm4e) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/HLasse/wav2vec_finetune) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/kingabzpro/WOLOF-ASR-Wav2Vec2/blob/e389abae6887788894795b7fd0171b306e3ca752/3-asr-fine-tune-wolof-gdrive.ipynb) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3IHgLup) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/2pbvnxdaKaw) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/tensorflow/mesh) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/LoniQin/english-spanish-translation-switch-transformer/blob/main/english_spanish_translation_switch_transformer.ipynb) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3pRazY7) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/HWna2c5VXDg) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/huawei-noah/CV-Backbones/tree/master/tnt_pytorch) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Rishit-dagli/Transformer-in-Transformer/blob/main/example/tnt-example.ipynb) | /abs/1905.07799v2) | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/31SYHwT) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://bit.ly/3GFNqib) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/facebookresearch/adaptive-span) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/prajjwal1/adaptive_transformer/blob/8f062a7d5c6573ebdb1aa2b57113db918156b15a/nbs/adaptive_train.ipynb) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3s1eYKD) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/ze7poOz-5Pw) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/google-research/tree/master/primer) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://github.com/lucidrains/x-transformers#relu) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3GH3tfQ) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/debgj24BAZE) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/google-research/tree/master/charformer) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://github.com/lucidrains/charformer-pytorch) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3lZ37cs) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/TsTib7b0zqU) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/tapas) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/google-research/tapas/blob/master/notebooks/sqa_predictions.ipynb) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3IIzoy4) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/m-zrcmRd7E4) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/mlpen/Nystromformer) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://github.com/lucidrains/nystrom-attention) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3IKod7X) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/machelreid/subformer) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]() | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/33qph0M) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/dHHvdubDnYM) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/chz816/esacl) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]() | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3pT0ABv) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/cpcp1998/permuteformer) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]() | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3DSZbQq) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/pytorch/fairseq) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]() | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3lZ4nME) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/qgUegkefocg) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/microsoft/fastformers) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/wuch15/Fastformer/blob/main/Fastformer.ipynb) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/31Kfxy5) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/huawei-noah/Pretrained-Language-Model) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/AutoTinyBERT) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3rXLWvt) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/51_K8RDVlXY) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/shamim-hussain/egt) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]() | ](https://arxiv.org/abs/2109.05729v3) | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3oOI6mr) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/fastnlp/cpt) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://github.com/fastnlp/CPT) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3dJjZzt) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/HO-_t0UArd4) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/dorarad/gansformer) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/GiorgiaAuroraAdorni/gansformer-reproducibility-challenge/blob/83c145d9e706d3814cf0eef7036a8e59c2accb2b/colab%20notebooks/Reproducibility_model_trainer.ipynb) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3ERUCY6) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/cBVdz1FjwVM) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/jayleicn/ClipBERT) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]() | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3oMJp5t) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/salesforce/codet5) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Dehbaiyor/IDLFall2021Project/blob/b100172b4477447db82a5c1c96dcef2349592ae4/Baselines/codet5-baseline.ipynb) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3yiN521) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)]() | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://huggingface.co/docs/transformers/model_doc/ibert) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3dLRpxh) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/vEOYfPPu3ik) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/byt5) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1rxpI8IlKk-D2crFqi2hdzbTBIezqgsCg?usp=sharing) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3EQym0N) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/E17qE148aSM) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/google-research/language/tree/master/language/canine) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/CANINE/Fine_tune_CANINE_on_IMDb_(movie_review_binary_classification).ipynb) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3DR2yrl) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/JJR3pBl78zw) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/rishikksh20/FNet-pytorch) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_FNet/test_sample_FNet.ipynb) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3dQwpFq) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://bit.ly/3DRodzt) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/microsoft/unilm) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/jdegange/MultimodalDocumentTransformers/blob/cc03caeca7efc1b17789567cc7a0b7e1c8bdac1a/Fine_tuning_LayoutLMv2ForTokenClassification_on_CORD.ipynb) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3DPjZZ6) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/microsoft/unilm/tree/master/layoutxlm) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://github.com/hieuthehungry/layout_xlm_vnm/blob/b35c9f13cd9ea75832d39a74587ca84610d8c659/fine_tuning_layoutxlm_fortokenclassification_on_cord.py) | ers/model_doc/gptj) | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3EV9WmI) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/S9FhhhQEqiE) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/kingoflolz/mesh-transformer-jax) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/NielsRogge/Transformers-Tutorials/blob/master/GPT-J-6B/Inference_with_GPT_J_6B.ipynb) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3ETLjGU) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/p5M7CYZAqfA) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/huseinzol05/malaya-speech) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/shamik-biswas-rft/Speech2Text/blob/8c8663b6cd3dbff557e90a6e813b58b17005a2bf/notebooks/automatic_transcriptions.ipynb) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3dNUWLE) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/WJWBq4NZfvY) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/deepmind/deepmind-research/tree/master/perceiver) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/NielsRogge/Transformers-Tutorials/blob/master/Perceiver/Perceiver_for_masked_language_modeling_and_image_classification.ipynb) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/31TNbAX) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/rNG_hpSyZcE) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/ZhuiyiTechnology/roformer) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://github.com/JunnYu/RoFormer_pytorch) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3ESxzw3) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/J0MoRQzZe8U) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/NVlabs/SegFormer) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/NielsRogge/Transformers-Tutorials/blob/master/SegFormer/Segformer_inference_notebook.ipynb) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3EQ4n8V) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/asappresearch/sew) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://huggingface.co/docs/transformers/model_doc/sew) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3dI3iV3) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/asappresearch/sew) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://huggingface.co/docs/transformers/model_doc/sew_d) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3dNMGLn) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/dJAoK5zK36M) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://arxiv.org/abs/2104.06678) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://arxiv.org/abs/2104.06678) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3dLX86f) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://bit.ly/3oUTyNi) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/oriram/splinter) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Qiwei97/Machine-Learning-Models/blob/67c713c411ec824a2dfcb0d75f1185d2d1b6e354/NLP%20Projects/Question%20Answering/Splinter%20TPU.ipynb) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/30mlLmR) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)]() | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/microsoft/unilm/tree/master/trocr) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/NielsRogge/Transformers-Tutorials/blob/master/TrOCR/Inference_with_TrOCR_%2B_Gradio_demo.ipynb) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3rZ5Aaq) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/hS42xD3O55E) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/microsoft/UniSpeech) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://huggingface.co/docs/transformers/model_doc/unispeech) | | 2021 | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/b1.jpg)](https://bit.ly/3rY9wZ1) | [![Youtube](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/yt1.jpg)](https://youtu.be/l4t_ENqo5rE) | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)](https://github.com/microsoft/unispeech) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://huggingface.co/docs/transformers/model_doc/unispeech_sat) | ormers/model_doc/marian) | -    |                                                              |                                                              | [![](https://raw.githubusercontent.com/ashishpatel26/Treasure-of-Transformers/main/images/git.jpg)]() | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://huggingface.co/docs/transformers/model_doc/marian) |

---

#### Instruction

* All blogs have links and some links contains different languages blog such as Chinese, Korean etc. for this Please use [Google Tranlater Page Convert](https://chrome.google.com/webstore/detail/google-translate/aapbdbdomjkkjkaonfhkkikfgjllcleb?hl=en).

---

#### How to Contribute

---

if you want to contribute on this project please send us email: [email protected] . 

🙏🙏Special Thanks to [**Komal Lamba**](https://github.com/komal11lamba) for contributing.

---

Copyright for source code belongs to the original author(s). However, under fair use you are encouraged to fork and contribute minor corrections and updates for the benefit of the reader(s).

---

**_Thanks for  Reading ...!!!_**

---