{"id":13400021,"url":"https://github.com/brianspiering/awesome-dl4nlp","last_synced_at":"2025-05-16T06:07:21.511Z","repository":{"id":42187407,"uuid":"60412130","full_name":"brianspiering/awesome-dl4nlp","owner":"brianspiering","description":"A curated list of awesome Deep Learning (DL) for Natural Language Processing (NLP) resources","archived":false,"fork":false,"pushed_at":"2023-01-05T17:45:18.000Z","size":92,"stargazers_count":1291,"open_issues_count":0,"forks_count":258,"subscribers_count":97,"default_branch":"master","last_synced_at":"2025-05-05T17:20:06.404Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/brianspiering.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-06-04T13:38:28.000Z","updated_at":"2025-04-27T03:01:15.000Z","dependencies_parsed_at":"2023-02-04T09:01:49.167Z","dependency_job_id":null,"html_url":"https://github.com/brianspiering/awesome-dl4nlp","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brianspiering%2Fawesome-dl4nlp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brianspiering%2Fawesome-dl4nlp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brianspiering%2Fawesome-dl4nlp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brianspiering%2Fawesome-dl4nlp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/brianspiering","download_url":"https://codeload.github.com/brianspiering/awesome-dl4nlp/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254478190,"owners_count":22077676,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T19:00:46.717Z","updated_at":"2025-05-16T06:07:21.468Z","avatar_url":"https://github.com/brianspiering.png","language":null,"funding_links":[],"categories":["Uncategorized","NLP","Learning Resources\u003ca title=\"Suggest an addition to the list!\" href=\"https://forms.gle/aPA41GT5AmbxrTwq5\"\u003e\u003cimg alt=\"Click button to suggest an addition\" align=\"right\" src=\"https://raw.githubusercontent.com/AI4LAM/awesome-ai4lam/main/.graphics/suggest-addition-small.svg\"\u003e\u003c/a\u003e","Other Curated Lists","Others","Awesome List","Natural Language Processing","Natural Language Understanding","Machine Learning \u0026 AI"],"sub_categories":["Uncategorized","Other \"awesome\" lists in AI and ML","Lexicons for Sentiment Analysis","Ukraine"],"readme":"Awesome Deep Learning for Natural Language Processing (NLP) [![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/sindresorhus/awesome)\n====\n\nTable of Contents\n----\n\n- __[Courses](#courses)__\n- __[Books](#books)__\n- __[Tutorials](#tutorials)__\n- __[Talks / Lectures](#talks)__\n- __[Frameworks / Models](#frameworks)__\n- __[Papers](#papers)__\n- __[Blog Posts](#blog-posts)__\n- __[Datasets](#datasets)__\n- __[Word Embeddings / Word Vectors](#word-embeddings)__\n- __[Contributing](#contributing)__\n\nCourses\n----\n1. NLP with Deep Learning / CS224N from Stanford (Winter 2019)\n\t- [Course homepage](http://web.stanford.edu/class/cs224n/index.html) A complete survey of the field with videos, lecture slides, and sample student projects.\n\t- [Course lectures](https://www.youtube.com/watch?v=8rXD5-xhemo\u0026list=PLoROMvodv4rOhcuXMZkNm7j3fVwBBY42z) Video playlist.\n\t- [Previous course notes](https://github.com/stanfordnlp/cs224n-winter17-notes) Probably the best \"book\" on DL for NLP.\n\t- [Course code](https://github.com/DSKSD/DeepNLP-models-Pytorch) Pytorch implementations of various Deep NLP models in cs-224n.\n1. Neural Networks for NLP from Carnegie Mellon University\n\t- [Course homepage](http://phontron.com/class/nn4nlp2017/)\n\t- [Course lectures](https://www.youtube.com/user/neubig/videos)\n\t- [Course code](https://github.com/neubig/nn4nlp2017-code/)\n1. Deep Learning for Natural Language Processing from University of Oxford and DeepMind\n\t- [Course homepage](https://www.cs.ox.ac.uk/teaching/courses/2016-2017/dl/)\n\t- [Course slides](https://github.com/oxford-cs-deepnlp-2017/lectures)\n\t- [Course lectures](https://www.youtube.com/playlist?list=PL613dYIGMXoZBtZhbyiBqb0QtgK6oJbpm)\n\nBooks\n-----\n1. [Deep Learning with Text: Natural Language Processing (Almost) from Scratch with Python and spaCy](https://www.amazon.com/Deep-Learning-Text-Approach-Processing/dp/1491984414) by Patrick Harrison and Matthew Honnibal \n1. [Neural Network Methods in Natural Language Processing](https://www.amazon.com/gp/product/1627052984) by Yoav Goldberg and Graeme Hirst\n1. [Deep Learning in Natural Language Processing](http://www.springer.com/us/book/9789811052088) by Li Deng and Yang Liu\n1. [Natural Language Processing in Action](https://www.manning.com/books/natural-language-processing-in-action) by Hobson Lane, Cole Howard, and Hannes Hapke\n1. Deep Learning: Natural Language Processing in Python by The LazyProgrammer (Kindle only)\n\t1. [Word2Vec and Word Embeddings in Python and Theano](https://www.amazon.com/Deep-Learning-Language-Processing-Embeddings-ebook/dp/B01KQ0ZN0A)\n\t1. [From Word2Vec to GLoVe in Python and Theano](https://www.amazon.com/Deep-Learning-Language-Processing-Word2Vec-ebook/dp/B01KRBOO4Y/)\n\t1. [Recursive Neural Networks: Recursive Neural (Tensor) Networks in Theano](https://www.amazon.com/Deep-Learning-Language-Processing-Recursive-ebook/dp/B01KS5AEXO)\n1. [Applied Natural Language Processing with Python](https://www.amazon.ca/Applied-Natural-Language-Processing-Python/dp/1484237323) by  Taweh Beysolow II\n1. [Deep Learning Cookbook](https://www.amazon.ca/Deep-Learning-Cookbook-Practical-Recipes/dp/149199584X) by Douwe Osinga\n1. [Deep Learning for Natural Language Processing: Creating Neural Networks with Python](https://www.amazon.ca/Deep-Learning-Natural-Language-Processing/dp/148423684X) by Palash Goyal, Sumit Pandey, Karan Jain\n1. [Machine Learning for Text](https://www.amazon.ca/Machine-Learning-Text-Charu-Aggarwal/dp/3319735306) by Charu C. Aggarwal\n1. [Natural Language Processing with TensorFlow](https://www.amazon.ca/Natural-Language-Processing-TensorFlow-language-ebook/dp/B077Q3VZFR) by Thushan Ganegedara\n1. [fastText Quick Start Guide: Get started with Facebook's library for text representation and classification](https://www.amazon.ca/fastText-Quick-Start-Guide-representation/dp/1789130999)\n1. [Hands-On Natural Language Processing with Python](https://www.amazon.ca/Hands-Natural-Language-Processing-Python/dp/178913949X)\n1. [Natural Language Processing in Action, Seond Edition](https://www.manning.com/books/natural-language-processing-in-action-second-edition) by Hobson Lane and Maria Dyshel\n1. [Getting Started with Natural Language Processing in Action](https://www.manning.com/books/getting-started-with-natural-language-processing) by Ekaterina Kochmar\n2. [Deep Learning for Natural Language Processing in Action](https://www.manning.com/books/deep-learning-for-natural-language-processing) by Stephan Raaijmakers\n\nTutorials\n-----\n\n1. [Text classification guide from Google](https://developers.google.com/machine-learning/guides/text-classification/)\n1. [Deep Learning for NLP with PyTorch](https://pytorch.org/tutorials/beginner/deep_learning_nlp_tutorial.html)\n\nTalks\n----\n1. [Deep Learning for Natural Language Processing (without Magic)](http://www.socher.org/index.php/DeepLearningTutorial/DeepLearningTutorial)\n1. [A Primer on Neural Network Models for Natural Language Processing](https://arxiv.org/abs/1510.00726)\n1. [Deep Learning for Natural Language Processing: Theory and Practice (Tutorial)](https://www.microsoft.com/en-us/research/publication/deep-learning-for-natural-language-processing-theory-and-practice-tutorial/)\n1. [TensorFlow Tutorials](https://www.tensorflow.org/tutorials/mandelbrot)\n1. [Practical Neural Networks for NLP](https://github.com/clab/dynet_tutorial_examples) from EMNLP 2016 using DyNet framework\n1. [Recurrent Neural Networks with Word Embeddings](http://deeplearning.net/tutorial/rnnslu.html)\n1. [LSTM Networks for Sentiment Analysis](http://deeplearning.net/tutorial/lstm.html)\n1. [TensorFlow demo using the Large Movie Review Dataset](http://ai.stanford.edu/~amaas/data/sentiment/)\n1. [LSTMVis: Visual Analysis for Recurrent Neural Networks](http://lstm.seas.harvard.edu/client/index.html)\n1. Using deep learning in natural language processing by Rob Romijnders from PyData Amsterdam 2017\n\t- [video](https://www.youtube.com/watch?v=HVdPWoZ_swY)\n\t- [slides](https://github.com/RobRomijnders/talks/blob/master/pydata_DL_NLP.pdf)\n1. [Richard Socher's talk on sentiment analysis, question answering, and sentence-image embeddings](https://www.youtube.com/watch?v=tdLmf8t4oqM)\n1. [Deep Learning, an interactive introduction for NLP-ers](http://www.slideshare.net/roelofp/220115dlmeetup)\n1. [Deep Natural Language Understanding](http://videolectures.net/deeplearning2016_cho_language_understanding/)\n1. [Deep Learning Summer School, Montreal 2016](http://videolectures.net/deeplearning2016_montreal/) Includes state-of-art language modeling.\n1. Tackling the Limits of Deep Learning for NLP by Richard Socher\n\t- [video](https://www.youtube.com/watch?v=JYwNmSe4HqE)\n\t- [slides](https://berkeley-deep-learning.github.io/cs294-131-s17/slides/socher-talk.pdf)\n\nFrameworks\n----\n1. [Overview of DL frameworks for NLP](https://medium.com/@datamonsters/13-deep-learning-frameworks-for-natural-language-processing-in-python-2b84a6b6cd98)\n1. General Frameworks\n\t1. [Keras](https://keras.io/) - _The Python Deep Learning library_ Emphasis on user friendliness, modularity, easy extensibility, and Pythonic.\n\t1. [TensorFlow](https://www.tensorflow.org/) - A cross-platform, general purpose Machine Intelligence library with Python and C++ API.\n\t1. [PyTorch](http://pytorch.org/) - PyTorch is a deep learning framework that puts Python first. \"Tensors and Dynamic neural networks in Python with strong GPU acceleration.\"\n\n1. Specific Frameworks\n\t1. [SpaCy](https://spacy.io/) - A Python package designed for speed, getting things dones, and interoperates with other Deep Learning frameworks\n\t1. [Genism: Topic modeling for humans](https://pypi.python.org/pypi/gensim) - A Python package that includes word2vec and doc2vec implementations.\n\t1. [fasttext](https://github.com/facebookresearch/fastText) Facebook's library for fast text representation and classification.\n\t1. Built on TensorFlow\n\t\t1. [SyntaxNet](https://github.com/tensorflow/models/tree/master/research/syntaxnet) - A toolkit for natural language understanding (NLU).\n\t\t1. [textsum](https://github.com/tensorflow/models/tree/master/research/textsum) - A Sequence-to-Sequence with Attention Model for Text Summarization.\n\t\t1. [Skip-Thought Vectors](https://github.com/tensorflow/models/tree/master/research/skip_thoughts) implementation in TensorFlow.\n\t\t1. [ActiveQA: Active Question Answering](https://github.com/google/active-qa) - Using reinforcement learning to train artificial agents for question answering\n\t\t1. [BERT](https://github.com/google-research/bert) - Bidirectional Encoder Representations from Transformers for pre-trained models\n\t1. Built on PyTorch\n\t\t1. [PyText](https://github.com/facebookresearch/pytext) - A deep-learning based NLP modeling framework by Facebook\n\t\t1. [AllenNLP](https://allennlp.org/) - An open-source NLP research library\n\t\t1. [Flair](https://github.com/zalandoresearch/flair) - A very simple framework for state-of-the-art NLP\n\t\t1. [fairseq](https://github.com/pytorch/fairseq) - A Sequence-to-Sequence Toolkit\n\t\t1. [fastai](http://docs.fast.ai/text.html) - Simplifies training fast and accurate neural nets using modern best practices\n\t\t1. [Transformer model](http://nlp.seas.harvard.edu/2018/04/03/attention.html) - Annotated notebook implementation\n\t1. [Deeplearning4j’s NLP framework](http://deeplearning4j.org/nlp) - Java implementation.\n\t1. [DyNet](https://github.com/clab/dynet) - _The Dynamic Neural Network Toolkit_ \"work well with networks that have dynamic structures that change for every training instance\".\n\t1. [deepnl](https://github.com/attardi/deepnl) - A Python library for NLP based on Deep Learning neural network architecture.\n\nPapers\n----\n1. [Deep or shallow, NLP is breaking out](http://dl.acm.org/citation.cfm?id=2874915) - General overview of how Deep Learning is impacting NLP.\n1. [Natural Language Processing from Research at Google](http://research.google.com/pubs/NaturalLanguageProcessing.html) - Not all Deep Learning (but mostly).\n1. [Context Dependent Recurrent Neural Network Language Model](http://www.msr-waypoint.com/pubs/176926/rnn_ctxt.pdf)\n1. [Translation Modeling with Bidirectional Recurrent Neural Networks](https://www-i6.informatik.rwth-aachen.de/publications/download/936/SundermeyerMartinAlkhouliTamerWuebkerJoernNeyHermann--TranslationModelingwithBidirectionalRecurrentNeuralNetworks--2014.pdf)\n1. [Contextual LSTM (CLSTM) models for Large scale NLP tasks](https://arxiv.org/abs/1602.06291)\n1. [LSTM Neural Networks for Language Modeling](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.248.4448\u0026rep=rep1\u0026type=pdf)\n1. [Exploring the Limits of Language Modeling](http://arxiv.org/pdf/1602.02410.pdf)\n1. [Conversational Contextual Cues](https://arxiv.org/abs/1606.00372) - Models context and participants in conversations.\n1. [Sequence to sequence learning with neural networks](http://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf)\n1. [Efficient Estimation of Word Representations in Vector Space](http://arxiv.org/pdf/1301.3781.pdf)\n1. [Learning Character-level Representations for Part-of-Speech Tagging](http://jmlr.org/proceedings/papers/v32/santos14.pdf)\n1. [Representation Learning for Text-level Discourse Parsing](http://www.cc.gatech.edu/~jeisenst/papers/ji-acl-2014.pdf)\n1. [Fast and Robust Neural Network Joint Models for Statistical Machine Translation](http://acl2014.org/acl2014/P14-1/pdf/P14-1129.pdf)\n1. [Parsing With Compositional Vector Grammars](http://www.socher.org/index.php/Main/ParsingWithCompositionalVectorGrammars)\n1. [Smart Reply: Automated Response Suggestion for Email](https://arxiv.org/abs/1606.04870)\n1. [Neural Architectures for Named Entity Recognition](https://arxiv.org/abs/1603.01360) - State-of-the-art performance in NER with bidirectional LSTM with a sequential conditional random layer and transition-based parsing with stack LSTMs.\n1. [Grammar as a Foreign Language](https://arxiv.org/abs/1412.7449) - State-of-the-art syntactic constituency parsing using generic sequence-to-sequence approach.\n\nBlog Posts\n----\n\n1. [Natural Language Processing (NLP) progress](https://nlpprogress.com/) Tracking the most common NLP tasks, including the datasets and the current state-of-the-art \n1. [A Review of the Recent History of Natural Language Processing](http://blog.aylien.com/a-review-of-the-recent-history-of-natural-language-processing/)\n1. [Deep Learning, NLP, and Representations](http://colah.github.io/posts/2014-07-NLP-RNNs-Representations/)\n1. [The Unreasonable Effectiveness of Recurrent Neural Networks](http://karpathy.github.io/2015/05/21/rnn-effectiveness/)\n1. [Neural Language Modeling From Scratch](http://ofir.io/Neural-Language-Modeling-From-Scratch/?a=1)\n1. [Machine Learning for Emoji Trends](http://instagram-engineering.tumblr.com/post/117889701472/emojineering-part-1-machine-learning-for-emoji)\n1. [Teaching Robots to Feel: Emoji \u0026 Deep Learning](http://getdango.com/emoji-and-deep-learning.html)\n1. [Computational Linguistics and Deep Learning](http://www.mitpressjournals.org/doi/pdf/10.1162/COLI_a_00239) - Opinion piece on how Deep Learning fits into the broader picture of text processing.\n1. [Deep Learning NLP Best Practices](http://ruder.io/deep-learning-nlp-best-practices/index.html)\n1. [7 types of Artificial Neural Networks for Natural Language Processing](https://medium.com/@datamonsters/artificial-neural-networks-for-natural-language-processing-part-1-64ca9ebfa3b2)\n1. [How to solve 90% of NLP problems: a step-by-step guide](https://blog.insightdatascience.com/how-to-solve-90-of-nlp-problems-a-step-by-step-guide-fda605278e4e)\n2. [7 Applications of Deep Learning for Natural Language Processing](https://machinelearningmastery.com/applications-of-deep-learning-for-natural-language-processing/)\n\nDatasets\n----\n1. [Dataset from \"One Billion Word Language Modeling Benchmark\"](http://www.statmt.org/lm-benchmark/1-billion-word-language-modeling-benchmark-r13output.tar.gz) - Almost 1B words, already pre-processed text.\n1. [Stanford Sentiment Treebank](https://nlp.stanford.edu/sentiment/treebank.html) - Fine grained sentiment labels for 215,154 phrases in the parse trees of 11,855 sentences.\n1. [Chatbot data from Kaggle](https://www.kaggle.com/samdeeplearning/deepnlp)\n1. [A list of text datasets that are free/public domain in alphabetical order](https://github.com/niderhoff/nlp-datasets)\n1. [Another list of text datasets that are free/public domain in reverse chronological order](https://github.com/karthikncode/nlp-datasets)\n1. Question Answering datasets\n\t1. [Quora's Question Pairs Dataset](https://data.quora.com/First-Quora-Dataset-Release-Question-Pairs) - Identify question pairs that have the same intent.\n\t1. [CMU's Wikipedia Factoid Question Answers](https://www.cs.cmu.edu/~ark/QA-data/)\n\t1. [DeepMind's Algebra Question Answering](https://github.com/deepmind/AQuA)\n\t1. [DeepMind's from CNN \u0026 DailyMail Question Answering](https://github.com/deepmind/rc-data)\n\t1. [Microsoft's WikiQA Open Domain Question Answering](https://www.microsoft.com/en-us/research/publication/wikiqa-a-challenge-dataset-for-open-domain-question-answering/)\n\t1. [Stanford Question Answering Dataset (SQuAD)](https://rajpurkar.github.io/SQuAD-explorer/) - covering reading comprehension\n\nWord Embeddings and friends\n----\n1. [The amazing power of word vectors](https://blog.acolyer.org/2016/04/21/the-amazing-power-of-word-vectors/) from The Morning Paper blog\n1. [Distributed Representations of Words and Phrases and their Compositionality](https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf) - The original word2vec paper.\n1. [word2vec Parameter Learning Explained](https://arxiv.org/abs/1411.2738) An elucidating explanation of word2vec training\n1. [Word embeddings in 2017: Trends and future directions](http://ruder.io/word-embeddings-2017/)\n1. [Learning Word Vectors for 157 Languages](https://arxiv.org/abs/1802.06893)\n1. [GloVe: Global Vectors for Word Representation](http://www-nlp.stanford.edu/pubs/glove.pdf) - A \"count-based\"/co-occurrence model to learn word embeddings.\n1.  Doc2Vec\n\t- [A gentle introduction to Doc2Vec](https://medium.com/scaleabout/a-gentle-introduction-to-doc2vec-db3e8c0cce5e)\n\t- [Distributed Representations of Sentences and Documents](https://cs.stanford.edu/~quocle/paragraph_vector.pdf)\n1. [Dynamic word embeddings for evolving semantic discovery](https://blog.acolyer.org/2018/02/22/dynamic-word-embeddings-for-evolving-semantic-discovery/) from The Morning Paper blog\n1. Ali Ghodsi's lecture on word2vec: \n\t- [part 1](https://www.youtube.com/watch?v=TsEGsdVJjuA)\n\t- [part 2](https://www.youtube.com/watch?v=nuirUEmbaJU)\n1. [word2vec analogy demo](http://deeplearner.fz-qqq.net/)\n1. [TensorFlow Embedding Projector of word vectors](http://projector.tensorflow.org/)\n1. Skip-Thought Vectors - \"unsupervised learning of a generic, distributed sentence encoder\"\n    - [Paper](http://arxiv.org/abs/1506.06726)\n    - [Code](https://github.com/ryankiros/skip-thoughts)\n\n-----\nContributing\n----\nHave anything in mind that you think is awesome and would fit in this list? Feel free to send me a pull request!\n\n-----\nLicense\n----\n\n[![CC0](http://i.creativecommons.org/p/zero/1.0/88x31.png)](http://creativecommons.org/publicdomain/zero/1.0/)\n\nTo the extent possible under law, [Dr. Brian J. Spiering](http://www.linkedin.com/in/brianspiering/) has waived all copyright and related or neighboring rights to this work.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrianspiering%2Fawesome-dl4nlp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbrianspiering%2Fawesome-dl4nlp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrianspiering%2Fawesome-dl4nlp/lists"}