{"id":15132609,"url":"https://github.com/mesolitica/nlp-models-tensorflow","last_synced_at":"2025-09-29T00:32:49.512Z","repository":{"id":64958235,"uuid":"130690315","full_name":"mesolitica/NLP-Models-Tensorflow","owner":"mesolitica","description":"Gathers machine learning and Tensorflow deep learning models for NLP problems, 1.13 \u003c Tensorflow \u003c 2.0","archived":true,"fork":false,"pushed_at":"2020-07-20T09:04:56.000Z","size":48393,"stargazers_count":1786,"open_issues_count":8,"forks_count":725,"subscribers_count":95,"default_branch":"master","last_synced_at":"2025-09-19T13:27:41.385Z","etag":null,"topics":["attention","chatbot","deep-learning","dnc-seq2seq","embedded","language-detection","lstm","lstm-seq2seq-tf","luong-api","machine-learning","neural-machine-translation","nlp","optical-character-recognition","pos-tagging","speech-to-text","summarization"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mesolitica.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-04-23T12:04:17.000Z","updated_at":"2025-08-15T10:10:45.000Z","dependencies_parsed_at":"2022-12-20T02:19:26.809Z","dependency_job_id":null,"html_url":"https://github.com/mesolitica/NLP-Models-Tensorflow","commit_stats":null,"previous_names":["mesolitica/nlp-models-tensorflow","huseinzol05/nlp-models-tensorflow"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mesolitica/NLP-Models-Tensorflow","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mesolitica%2FNLP-Models-Tensorflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mesolitica%2FNLP-Models-Tensorflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mesolitica%2FNLP-Models-Tensorflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mesolitica%2FNLP-Models-Tensorflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mesolitica","download_url":"https://codeload.github.com/mesolitica/NLP-Models-Tensorflow/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mesolitica%2FNLP-Models-Tensorflow/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":277450939,"owners_count":25819971,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-28T02:00:08.834Z","response_time":79,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["attention","chatbot","deep-learning","dnc-seq2seq","embedded","language-detection","lstm","lstm-seq2seq-tf","luong-api","machine-learning","neural-machine-translation","nlp","optical-character-recognition","pos-tagging","speech-to-text","summarization"],"created_at":"2024-09-26T04:21:38.988Z","updated_at":"2025-09-29T00:32:44.504Z","avatar_url":"https://github.com/mesolitica.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n    \u003ca href=\"#readme\"\u003e\n        \u003cimg alt=\"logo\" width=\"40%\" src=\"nlp-tf.png\"\u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/huseinzol05/NLP-Models-Tensorflow/blob/master/LICENSE\"\u003e\u003cimg alt=\"MIT License\" src=\"https://img.shields.io/badge/License-MIT-yellow.svg\"\u003e\u003c/a\u003e\n  \u003ca href=\"#\"\u003e\u003cimg src=\"https://img.shields.io/badge/total%20notebooks-335--models-blue.svg\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n---\n\n**NLP-Models-Tensorflow**, Gathers machine learning and tensorflow deep learning models for NLP problems, **code simplify inside Jupyter Notebooks 100%**.\n\n## Table of contents\n  * [Abstractive Summarization](#abstractive-summarization)\n  * [Chatbot](#chatbot)\n  * [Dependency Parser](#dependency-parser)\n  * [Entity Tagging](#entity-tagging)\n  * [Extractive Summarization](#extractive-summarization)\n  * [Generator](#generator)\n  * [Language Detection](#language-detection)\n  * [Neural Machine Translation](neural-machine-translation)\n  * [OCR](#ocr-optical-character-recognition)\n  * [POS Tagging](#pos-tagging)\n  * [Question-Answers](#question-answers)\n  * [Sentence pairs](#sentence-pair)\n  * [Speech-to-Text](#speech-to-text)\n  * [Spelling correction](#spelling-correction)\n  * [SQUAD Question-Answers](#squad-question-answers)\n  * [Stemming](#stemming)\n  * [Text Augmentation](#text-augmentation)\n  * [Text Classification](#text-classification)\n  * [Text Similarity](#text-similarity)\n  * [Text-to-Speech](#text-to-speech)\n  * [Topic Generator](#topic-generator)\n  * [Topic Modeling](#topic-modeling)\n  * [Unsupervised Extractive Summarization](#unsupervised-extractive-summarization)\n  * [Vectorizer](#vectorizer)\n  * [Old-to-Young Vocoder](#old-to-young-vocoder)\n  * [Visualization](#visualization)\n  * [Attention](#attention)\n\n## Objective\n\nOriginal implementations are quite complex and not really beginner friendly. So I tried to simplify most of it. Also, there are tons of not-yet release papers implementation. So feel free to use it for your own research!\n\nI will attached github repositories for models that I not implemented from scratch, basically I copy, paste and fix those code for deprecated issues.\n\n## Tensorflow version\n\nTensorflow version 1.13 and above only, not included 2.X version. 1.13 \u003c Tensorflow \u003c 2.0 \n\n```bash\npip install -r requirements.txt\n```\n\n## Contents\n\n### [Abstractive Summarization](abstractive-summarization)\n\nTrained on [India news](abstractive-summarization/dataset).\n\nAccuracy based on 10 epochs only, calculated using word positions.\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (12 notebooks)\u003c/summary\u003e\n\n1. LSTM Seq2Seq using topic modelling, test accuracy 13.22%\n2. LSTM Seq2Seq + Luong Attention using topic modelling, test accuracy 12.39%\n3. LSTM Seq2Seq + Beam Decoder using topic modelling, test accuracy 10.67%\n4. LSTM Bidirectional + Luong Attention + Beam Decoder using topic modelling, test accuracy 8.29%\n5. Pointer-Generator + Bahdanau, https://github.com/xueyouluo/my_seq2seq, test accuracy 15.51%\n6. Copynet, test accuracy 11.15%\n7. Pointer-Generator + Luong, https://github.com/xueyouluo/my_seq2seq, test accuracy 16.51%\n8. Dilated Seq2Seq, test accuracy 10.88%\n9. Dilated Seq2Seq + Self Attention, test accuracy 11.54%\n10. BERT + Dilated CNN Seq2seq, test accuracy 13.5%\n11. self-attention + Pointer-Generator, test accuracy 4.34%\n12. Dilated-CNN Seq2seq + Pointer-Generator, test accuracy 5.57%\n\n\u003c/details\u003e\n\n### [Chatbot](chatbot)\n\nTrained on [Cornell Movie Dialog corpus](chatbot/dataset.tar.gz), accuracy table in [chatbot](chatbot).\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (54 notebooks)\u003c/summary\u003e\n\n1. Basic cell Seq2Seq-manual\n2. LSTM Seq2Seq-manual\n3. GRU Seq2Seq-manual\n4. Basic cell Seq2Seq-API Greedy\n5. LSTM Seq2Seq-API Greedy\n6. GRU Seq2Seq-API Greedy\n7. Basic cell Bidirectional Seq2Seq-manual\n8. LSTM Bidirectional Seq2Seq-manual\n9. GRU Bidirectional Seq2Seq-manual\n10. Basic cell Bidirectional Seq2Seq-API Greedy\n11. LSTM Bidirectional Seq2Seq-API Greedy\n12. GRU Bidirectional Seq2Seq-API Greedy\n13. Basic cell Seq2Seq-manual + Luong Attention\n14. LSTM Seq2Seq-manual + Luong Attention\n15. GRU Seq2Seq-manual + Luong Attention\n16. Basic cell Seq2Seq-manual + Bahdanau Attention\n17. LSTM Seq2Seq-manual + Bahdanau Attention\n18. GRU Seq2Seq-manual + Bahdanau Attention\n19. LSTM Bidirectional Seq2Seq-manual + Luong Attention\n20. GRU Bidirectional Seq2Seq-manual + Luong Attention\n21. LSTM Bidirectional Seq2Seq-manual + Bahdanau Attention\n22. GRU Bidirectional Seq2Seq-manual + Bahdanau Attention\n23. LSTM Bidirectional Seq2Seq-manual + backward Bahdanau + forward Luong\n24. GRU Bidirectional Seq2Seq-manual + backward Bahdanau + forward Luong\n25. LSTM Seq2Seq-API Greedy + Luong Attention\n26. GRU Seq2Seq-API Greedy + Luong Attention\n27. LSTM Seq2Seq-API Greedy + Bahdanau Attention\n28. GRU Seq2Seq-API Greedy + Bahdanau Attention\n29. LSTM Seq2Seq-API Beam Decoder\n30. GRU Seq2Seq-API Beam Decoder\n31. LSTM Bidirectional Seq2Seq-API + Luong Attention + Beam Decoder\n32. GRU Bidirectional Seq2Seq-API + Luong Attention + Beam Decoder\n33. LSTM Bidirectional Seq2Seq-API + backward Bahdanau + forward Luong + Stack Bahdanau Luong Attention + Beam Decoder\n34. GRU Bidirectional Seq2Seq-API + backward Bahdanau + forward Luong + Stack Bahdanau Luong Attention + Beam Decoder\n35. Bytenet\n36. LSTM Seq2Seq + tf.estimator\n37. Capsule layers + LSTM Seq2Seq-API Greedy\n38. Capsule layers + LSTM Seq2Seq-API + Luong Attention + Beam Decoder\n39. LSTM Bidirectional Seq2Seq-API + backward Bahdanau + forward Luong + Stack Bahdanau Luong Attention + Beam Decoder + Dropout + L2\n40. DNC Seq2Seq\n41. LSTM Bidirectional Seq2Seq-API + Luong Monotic Attention + Beam Decoder\n42. LSTM Bidirectional Seq2Seq-API + Bahdanau Monotic Attention + Beam Decoder\n43. End-to-End Memory Network + Basic cell\n44. End-to-End Memory Network + LSTM cell\n45. Attention is all you need\n46. Transformer-XL\n47. Attention is all you need + Beam Search\n48. Transformer-XL + LSTM\n49. GPT-2 + LSTM\n50. CNN Seq2seq\n51. Conv-Encoder + LSTM\n52. Tacotron + Greedy decoder\n53. Tacotron + Beam decoder\n54. Google NMT\n\n\u003c/details\u003e\n\n### [Dependency-Parser](dependency-parser)\n\nTrained on [CONLL English Dependency](https://github.com/UniversalDependencies/UD_English-EWT). Train set to train, dev and test sets to test.\n\nStackpointer and Biaffine-attention originally from https://github.com/XuezheMax/NeuroNLP2 written in Pytorch.\n\nAccuracy based on arc, types and root accuracies after 15 epochs only.\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (8 notebooks)\u003c/summary\u003e\n\n1. Bidirectional RNN + CRF + Biaffine, arc accuracy 70.48%, types accuracy 65.18%, root accuracy 66.4%\n2. Bidirectional RNN + Bahdanau + CRF + Biaffine, arc accuracy 70.82%, types accuracy 65.33%, root accuracy 66.77%\n3. Bidirectional RNN + Luong + CRF + Biaffine, arc accuracy 71.22%, types accuracy 65.73%, root accuracy 67.23%\n4. BERT Base + CRF + Biaffine, arc accuracy 64.30%, types accuracy 62.89%, root accuracy 74.19%\n5. Bidirectional RNN + Biaffine Attention + Cross Entropy, arc accuracy 72.42%, types accuracy 63.53%, root accuracy 68.51%\n6. BERT Base + Biaffine Attention + Cross Entropy, arc accuracy 72.85%, types accuracy 67.11%, root accuracy 73.93%\n7. Bidirectional RNN + Stackpointer, arc accuracy 61.88%, types accuracy 48.20%, root accuracy 89.39%\n8. XLNET Base + Biaffine Attention + Cross Entropy, arc accuracy 74.41%, types accuracy 71.37%, root accuracy 73.17%\n\n\u003c/details\u003e\n\n### [Entity-Tagging](entity-tagging)\n\nTrained on [CONLL NER](https://cogcomp.org/page/resource_view/81).\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (9 notebooks)\u003c/summary\u003e\n\n1. Bidirectional RNN + CRF, test accuracy 96%\n2. Bidirectional RNN + Luong Attention + CRF, test accuracy 93%\n3. Bidirectional RNN + Bahdanau Attention + CRF, test accuracy 95%\n4. Char Ngrams + Bidirectional RNN + Bahdanau Attention + CRF, test accuracy 96%\n5. Char Ngrams + Bidirectional RNN + Bahdanau Attention + CRF, test accuracy 96%\n6. Char Ngrams + Residual Network + Bahdanau Attention + CRF, test accuracy 69%\n7. Char Ngrams + Attention is you all Need + CRF, test accuracy 90%\n8. BERT, test accuracy 99%\n9. XLNET-Base, test accuracy 99%\n\n\u003c/details\u003e\n\n### [Extractive Summarization](extractive-summarization)\n\nTrained on [CNN News dataset](https://cs.nyu.edu/~kcho/DMQA/).\n\nAccuracy based on ROUGE-2.\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (4 notebooks)\u003c/summary\u003e\n\n1. LSTM RNN, test accuracy 16.13%\n2. Dilated-CNN, test accuracy 15.54%\n3. Multihead Attention, test accuracy 26.33%\n4. BERT-Base\n\n\u003c/details\u003e\n\n### [Generator](generator)\n\nTrained on [Shakespeare dataset](generator/shakespeare.txt).\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (15 notebooks)\u003c/summary\u003e\n\n1. Character-wise RNN + LSTM\n2. Character-wise RNN + Beam search\n3. Character-wise RNN + LSTM + Embedding\n4. Word-wise RNN + LSTM\n5. Word-wise RNN + LSTM + Embedding\n6. Character-wise + Seq2Seq + GRU\n7. Word-wise + Seq2Seq + GRU\n8. Character-wise RNN + LSTM + Bahdanau Attention\n9. Character-wise RNN + LSTM + Luong Attention\n10. Word-wise + Seq2Seq + GRU + Beam\n11. Character-wise + Seq2Seq + GRU + Bahdanau Attention\n12. Word-wise + Seq2Seq + GRU + Bahdanau Attention\n13. Character-wise Dilated CNN + Beam search\n14. Transformer + Beam search\n15. Transformer XL + Beam search\n\n\u003c/details\u003e\n\n### [Language-detection](language-detection)\n\nTrained on [Tatoeba dataset](http://downloads.tatoeba.org/exports/sentences.tar.bz2).\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (1 notebooks)\u003c/summary\u003e\n\n1. Fast-text Char N-Grams\n\n\u003c/details\u003e\n\n### [Neural Machine Translation](neural-machine-translation)\n\nTrained on [English-French](https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/data_generators/translate_enfr.py), accuracy table in [neural-machine-translation](neural-machine-translation).\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (53 notebooks)\u003c/summary\u003e\n\n1.basic-seq2seq\n2.lstm-seq2seq\n3.gru-seq2seq\n4.basic-seq2seq-contrib-greedy\n5.lstm-seq2seq-contrib-greedy\n6.gru-seq2seq-contrib-greedy\n7.basic-birnn-seq2seq\n8.lstm-birnn-seq2seq\n9.gru-birnn-seq2seq\n10.basic-birnn-seq2seq-contrib-greedy\n11.lstm-birnn-seq2seq-contrib-greedy\n12.gru-birnn-seq2seq-contrib-greedy\n13.basic-seq2seq-luong\n14.lstm-seq2seq-luong\n15.gru-seq2seq-luong\n16.basic-seq2seq-bahdanau\n17.lstm-seq2seq-bahdanau\n18.gru-seq2seq-bahdanau\n19.basic-birnn-seq2seq-bahdanau\n20.lstm-birnn-seq2seq-bahdanau\n21.gru-birnn-seq2seq-bahdanau\n22.basic-birnn-seq2seq-luong\n23.lstm-birnn-seq2seq-luong\n24.gru-birnn-seq2seq-luong\n25.lstm-seq2seq-contrib-greedy-luong\n26.gru-seq2seq-contrib-greedy-luong\n27.lstm-seq2seq-contrib-greedy-bahdanau\n28.gru-seq2seq-contrib-greedy-bahdanau\n29.lstm-seq2seq-contrib-beam-luong\n30.gru-seq2seq-contrib-beam-luong\n31.lstm-seq2seq-contrib-beam-bahdanau\n32.gru-seq2seq-contrib-beam-bahdanau\n33.lstm-birnn-seq2seq-contrib-beam-bahdanau\n34.lstm-birnn-seq2seq-contrib-beam-luong\n35.gru-birnn-seq2seq-contrib-beam-bahdanau\n36.gru-birnn-seq2seq-contrib-beam-luong\n37.lstm-birnn-seq2seq-contrib-beam-luongmonotonic\n38.gru-birnn-seq2seq-contrib-beam-luongmonotic\n39.lstm-birnn-seq2seq-contrib-beam-bahdanaumonotonic\n40.gru-birnn-seq2seq-contrib-beam-bahdanaumonotic\n41.residual-lstm-seq2seq-greedy-luong\n42.residual-gru-seq2seq-greedy-luong\n43.residual-lstm-seq2seq-greedy-bahdanau\n44.residual-gru-seq2seq-greedy-bahdanau\n45.memory-network-lstm-decoder-greedy\n46.google-nmt\n47.transformer-encoder-transformer-decoder\n48.transformer-encoder-lstm-decoder-greedy\n49.bertmultilanguage-encoder-bertmultilanguage-decoder\n50.bertmultilanguage-encoder-lstm-decoder\n51.bertmultilanguage-encoder-transformer-decoder\n52.bertenglish-encoder-transformer-decoder\n53.transformer-t2t-2gpu\n\n\u003c/details\u003e\n\n### [OCR (optical character recognition)](ocr)\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (2 notebooks)\u003c/summary\u003e\n\n1. CNN + LSTM RNN, test accuracy 100%\n2. Im2Latex, test accuracy 100%\n\n\u003c/details\u003e\n\n### [POS-Tagging](pos-tagging)\n\nTrained on [CONLL POS](https://cogcomp.org/page/resource_view/81).\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (8 notebooks)\u003c/summary\u003e\n\n1. Bidirectional RNN + CRF, test accuracy 92%\n2. Bidirectional RNN + Luong Attention + CRF, test accuracy 91%\n3. Bidirectional RNN + Bahdanau Attention + CRF, test accuracy 91%\n4. Char Ngrams + Bidirectional RNN + Bahdanau Attention + CRF, test accuracy 91%\n5. Char Ngrams + Bidirectional RNN + Bahdanau Attention + CRF, test accuracy 91%\n6. Char Ngrams + Residual Network + Bahdanau Attention + CRF, test accuracy 3%\n7. Char Ngrams + Attention is you all Need + CRF, test accuracy 89%\n8. BERT, test accuracy 99%\n\n\u003c/details\u003e\n\n### [Question-Answers](question-answer)\n\nTrained on [bAbI Dataset](https://research.fb.com/downloads/babi/).\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (4 notebooks)\u003c/summary\u003e\n\n1. End-to-End Memory Network + Basic cell\n2. End-to-End Memory Network + GRU cell\n3. End-to-End Memory Network + LSTM cell\n4. Dynamic Memory\n\n\u003c/details\u003e\n\n### [Sentence-pair](sentence-pair)\n\nTrained on [Cornell Movie--Dialogs Corpus](https://people.mpi-sws.org/~cristian/Cornell_Movie-Dialogs_Corpus.html)\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (1 notebooks)\u003c/summary\u003e\n\n1. BERT\n\n\u003c/details\u003e\n\n### [Speech to Text](speech-to-text)\n\nTrained on [Toronto speech dataset](https://tspace.library.utoronto.ca/handle/1807/24487).\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (11 notebooks)\u003c/summary\u003e\n\n1. Tacotron, https://github.com/Kyubyong/tacotron_asr, test accuracy 77.09%\n2. BiRNN LSTM, test accuracy 84.66%\n3. BiRNN Seq2Seq + Luong Attention + Cross Entropy, test accuracy 87.86%\n4. BiRNN Seq2Seq + Bahdanau Attention + Cross Entropy, test accuracy 89.28%\n5. BiRNN Seq2Seq + Bahdanau Attention + CTC, test accuracy 86.35%\n6. BiRNN Seq2Seq + Luong Attention + CTC, test accuracy 80.30%\n7. CNN RNN + Bahdanau Attention, test accuracy 80.23%\n8. Dilated CNN RNN, test accuracy 31.60%\n9. Wavenet, test accuracy 75.11%\n10. Deep Speech 2, test accuracy 81.40%\n11. Wav2Vec Transfer learning BiRNN LSTM, test accuracy 83.24%\n\n\u003c/details\u003e\n\n### [Spelling correction](spelling-correction)\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (4 notebooks)\u003c/summary\u003e\n\n1. BERT-Base\n2. XLNET-Base\n3. BERT-Base Fast\n4. BERT-Base accurate\n\n\u003c/details\u003e\n\n### [SQUAD Question-Answers](squad-qa)\n\nTrained on [SQUAD Dataset](https://rajpurkar.github.io/SQuAD-explorer/).\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (1 notebooks)\u003c/summary\u003e\n\n1. BERT,\n```json\n{\"exact_match\": 77.57805108798486, \"f1\": 86.18327335287402}\n```\n\n\u003c/details\u003e\n\n### [Stemming](stemming)\n\nTrained on [English Lemmatization](stemming/lemmatization-en.txt).\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (6 notebooks)\u003c/summary\u003e\n\n1. LSTM + Seq2Seq + Beam\n2. GRU + Seq2Seq + Beam\n3. LSTM + BiRNN + Seq2Seq + Beam\n4. GRU + BiRNN + Seq2Seq + Beam\n5. DNC + Seq2Seq + Greedy\n6. BiRNN + Bahdanau + Copynet\n\n\u003c/details\u003e\n\n### [Text Augmentation](text-augmentation)\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (8 notebooks)\u003c/summary\u003e\n\n1. Pretrained Glove\n2. GRU VAE-seq2seq-beam TF-probability\n3. LSTM VAE-seq2seq-beam TF-probability\n4. GRU VAE-seq2seq-beam + Bahdanau Attention TF-probability\n5. VAE + Deterministic Bahdanau Attention, https://github.com/HareeshBahuleyan/tf-var-attention\n6. VAE + VAE Bahdanau Attention, https://github.com/HareeshBahuleyan/tf-var-attention\n7. BERT-Base + Nucleus Sampling\n8. XLNET-Base + Nucleus Sampling\n\n\u003c/details\u003e\n\n### [Text classification](text-classification)\n\nTrained on [English sentiment dataset](text-classification/data), accuracy table in [text-classification](text-classification).\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (79 notebooks)\u003c/summary\u003e\n\n1. Basic cell RNN\n2. Basic cell RNN + Hinge\n3. Basic cell RNN + Huber\n4. Basic cell Bidirectional RNN\n5. Basic cell Bidirectional RNN + Hinge\n6. Basic cell Bidirectional RNN + Huber\n7. LSTM cell RNN\n8. LSTM cell RNN + Hinge\n9. LSTM cell RNN + Huber\n10. LSTM cell Bidirectional RNN\n11. LSTM cell Bidirectional RNN + Huber\n12. LSTM cell RNN + Dropout + L2\n13. GRU cell RNN\n14. GRU cell RNN + Hinge\n15. GRU cell RNN + Huber\n16. GRU cell Bidirectional RNN\n17. GRU cell Bidirectional RNN + Hinge\n18. GRU cell Bidirectional RNN + Huber\n19. LSTM RNN + Conv2D\n20. K-max Conv1d\n21. LSTM RNN + Conv1D + Highway\n22. LSTM RNN + Basic Attention\n23. LSTM Dilated RNN\n24. Layer-Norm LSTM cell RNN\n25. Only Attention Neural Network\n26. Multihead-Attention Neural Network\n27. Neural Turing Machine\n28. LSTM Seq2Seq\n29. LSTM Seq2Seq + Luong Attention\n30. LSTM Seq2Seq + Bahdanau Attention\n31. LSTM Seq2Seq + Beam Decoder\n32. LSTM Bidirectional Seq2Seq\n33. Pointer Net\n34. LSTM cell RNN + Bahdanau Attention\n35. LSTM cell RNN + Luong Attention\n36. LSTM cell RNN + Stack Bahdanau Luong Attention\n37. LSTM cell Bidirectional RNN + backward Bahdanau + forward Luong\n38. Bytenet\n39. Fast-slow LSTM\n40. Siamese Network\n41. LSTM Seq2Seq + tf.estimator\n42. Capsule layers + RNN LSTM\n43. Capsule layers + LSTM Seq2Seq\n44. Capsule layers + LSTM Bidirectional Seq2Seq\n45. Nested LSTM\n46. LSTM Seq2Seq + Highway\n47. Triplet loss + LSTM\n48. DNC (Differentiable Neural Computer)\n49. ConvLSTM\n50. Temporal Convd Net\n51. Batch-all Triplet-loss + LSTM\n52. Fast-text\n53. Gated Convolution Network\n54. Simple Recurrent Unit\n55. LSTM Hierarchical Attention Network\n56. Bidirectional Transformers\n57. Dynamic Memory Network\n58. Entity Network\n59. End-to-End Memory Network\n60. BOW-Chars Deep sparse Network\n61. Residual Network using Atrous CNN\n62. Residual Network using Atrous CNN + Bahdanau Attention\n63. Deep pyramid CNN\n64. Transformer-XL\n65. Transfer learning GPT-2 345M\n66. Quasi-RNN\n67. Tacotron\n68. Slice GRU\n69. Slice GRU + Bahdanau\n70. Wavenet\n71. Transfer learning BERT Base\n72. Transfer learning XL-net Large\n73. LSTM BiRNN global Max and average pooling\n74. Transfer learning BERT Base drop 6 layers\n75. Transfer learning BERT Large drop 12 layers\n76. Transfer learning XL-net Base\n77. Transfer learning ALBERT\n78. Transfer learning ELECTRA Base\n79. Transfer learning ELECTRA Large\n\n\u003c/details\u003e\n\n### [Text Similarity](text-similarity)\n\nTrained on [MNLI](https://cims.nyu.edu/~sbowman/multinli/).\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (10 notebooks)\u003c/summary\u003e\n\n1. BiRNN + Contrastive loss, test accuracy 73.032%\n2. BiRNN + Cross entropy, test accuracy 74.265%\n3. BiRNN + Circle loss, test accuracy 75.857%\n4. BiRNN + Proxy loss, test accuracy 48.37%\n5. BERT Base + Cross entropy, test accuracy 91.123%\n6. BERT Base + Circle loss, test accuracy 89.903%\n7. ELECTRA Base + Cross entropy, test accuracy 96.317%\n8. ELECTRA Base + Circle loss, test accuracy 95.603%\n9. XLNET Base + Cross entropy, test accuracy 93.998%\n10. XLNET Base + Circle loss, test accuracy 94.033%\n\n\u003c/details\u003e\n\n### [Text to Speech](text-to-speech)\n\nTrained on [Toronto speech dataset](https://tspace.library.utoronto.ca/handle/1807/24487).\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (8 notebooks)\u003c/summary\u003e\n\n1. Tacotron, https://github.com/Kyubyong/tacotron\n2. CNN Seq2seq + Dilated CNN vocoder\n3. Seq2Seq + Bahdanau Attention\n4. Seq2Seq + Luong Attention\n5. Dilated CNN + Monothonic Attention + Dilated CNN vocoder\n6. Dilated CNN + Self Attention + Dilated CNN vocoder\n7. Deep CNN + Monothonic Attention + Dilated CNN vocoder\n8. Deep CNN + Self Attention + Dilated CNN vocoder\n\n\u003c/details\u003e\n\n### [Topic Generator](topic-generator)\n\nTrained on [Malaysia news](https://github.com/huseinzol05/Malaya-Dataset/raw/master/news/news.zip).\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (4 notebooks)\u003c/summary\u003e\n\n1. TAT-LSTM\n2. TAV-LSTM\n3. MTA-LSTM\n4. Dilated CNN Seq2seq\n\n\u003c/details\u003e\n\n### [Topic Modeling](topic-model)\n\nExtracted from [English sentiment dataset](text-classification/data).\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (3 notebooks)\u003c/summary\u003e\n\n1. LDA2Vec\n2. BERT Attention\n3. XLNET Attention\n\n\u003c/details\u003e\n\n### [Unsupervised Extractive Summarization](unsupervised-extractive-summarization)\n\nTrained on [random books](extractive-summarization/books).\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (3 notebooks)\u003c/summary\u003e\n\n1. Skip-thought Vector\n2. Residual Network using Atrous CNN\n3. Residual Network using Atrous CNN + Bahdanau Attention\n\n\u003c/details\u003e\n\n### [Vectorizer](vectorizer)\n\nTrained on [English sentiment dataset](text-classification/data).\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (11 notebooks)\u003c/summary\u003e\n\n1. Word Vector using CBOW sample softmax\n2. Word Vector using CBOW noise contrastive estimation\n3. Word Vector using skipgram sample softmax\n4. Word Vector using skipgram noise contrastive estimation\n5. Supervised Embedded\n6. Triplet-loss + LSTM\n7. LSTM Auto-Encoder\n8. Batch-All Triplet-loss LSTM\n9. Fast-text\n10. ELMO (biLM)\n11. Triplet-loss + BERT\n\n\u003c/details\u003e\n\n### [Visualization](visualization)\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (4 notebooks)\u003c/summary\u003e\n\n1. Attention heatmap on Bahdanau Attention\n2. Attention heatmap on Luong Attention\n3. BERT attention, https://github.com/hsm207/bert_attn_viz\n4. XLNET attention\n\n\u003c/details\u003e\n\n### [Old-to-Young Vocoder](vocoder)\n\nTrained on [Toronto speech dataset](https://tspace.library.utoronto.ca/handle/1807/24487).\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (1 notebooks)\u003c/summary\u003e\n\n1. Dilated CNN\n\n\u003c/details\u003e\n\n### [Attention](attention)\n\n\u003cdetails\u003e\u003csummary\u003eComplete list (8 notebooks)\u003c/summary\u003e\n\n1. Bahdanau\n2. Luong\n3. Hierarchical\n4. Additive\n5. Soft\n6. Attention-over-Attention\n7. Bahdanau API\n8. Luong API\n\n\u003c/details\u003e\n\n### [Not-deep-learning](not-deep-learning)\n\n1. Markov chatbot\n2. Decomposition summarization (3 notebooks)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmesolitica%2Fnlp-models-tensorflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmesolitica%2Fnlp-models-tensorflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmesolitica%2Fnlp-models-tensorflow/lists"}