{"id":13811217,"url":"https://github.com/DanAnastasyev/DeepNLP-Course","last_synced_at":"2025-05-14T15:31:40.358Z","repository":{"id":97239508,"uuid":"122381314","full_name":"DanAnastasyev/DeepNLP-Course","owner":"DanAnastasyev","description":"Deep NLP Course","archived":false,"fork":false,"pushed_at":"2019-07-20T20:24:49.000Z","size":1974,"stargazers_count":627,"open_issues_count":4,"forks_count":165,"subscribers_count":25,"default_branch":"master","last_synced_at":"2024-11-19T05:56:09.155Z","etag":null,"topics":["colab-notebook","deep-learning","keras","nlp","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DanAnastasyev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-02-21T19:15:38.000Z","updated_at":"2024-10-12T16:50:43.000Z","dependencies_parsed_at":"2023-04-26T03:48:51.063Z","dependency_job_id":null,"html_url":"https://github.com/DanAnastasyev/DeepNLP-Course","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DanAnastasyev%2FDeepNLP-Course","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DanAnastasyev%2FDeepNLP-Course/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DanAnastasyev%2FDeepNLP-Course/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DanAnastasyev%2FDeepNLP-Course/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DanAnastasyev","download_url":"https://codeload.github.com/DanAnastasyev/DeepNLP-Course/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254171889,"owners_count":22026534,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["colab-notebook","deep-learning","keras","nlp","pytorch"],"created_at":"2024-08-04T03:00:42.629Z","updated_at":"2025-05-14T15:31:35.343Z","avatar_url":"https://github.com/DanAnastasyev.png","language":"Jupyter Notebook","funding_links":[],"categories":["Course and Tutorial"],"sub_categories":[],"readme":"# Deep NLP Course at ABBYY\n\nDeep learning for NLP crash course at ABBYY.\n\nSuggested textbook: [Neural Network Methods in Natural Language Processing by Yoav Goldberg](https://www.amazon.com/Language-Processing-Synthesis-Lectures-Technologies/dp/1627052984)\n\n*I'm gradually updating and translating the notebooks right now. Stay in touch.*\n\n## Materials\n### Week 1: *Introduction*  \nSentiment analysis on the IMDB movie review dataset: a short overview of classical machine learning for NLP + indecently brief intro to keras.   \n\nRussian version: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/12nrEX3JXTxsHWC-HpuwkTWyJybjmkZu-#forceEdit=true\u0026offline=true\u0026sandboxMode=true)\n\nUpdated English version: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1eW-mN3gEdLluYe1W1unQ-7-71by8eauz#scrollTo=OlqOAQmQGXOL\u0026forceEdit=true\u0026offline=true\u0026sandboxMode=true)\n\n### Week 2: *Word Embeddings: Part 1*  \nMeet the Word Embeddings: an unsupervised method to capture some fun relationships between words.  \nPhrases similarity with word embeddings model + word based machine translation without parallel data (with MUSE word embeddings).  \n\nRussian version: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1o65wrq6RYgWyyMvNP8r9ZknXBniDoXrn#forceEdit=true\u0026offline=true\u0026sandboxMode=true)\n\nUpdated English version: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1ey9NARKvk-c4vfQGdvOkPjp5wNGmxd5o#forceEdit=true\u0026offline=true\u0026sandboxMode=true)\n\n### Week 3: *Word Embeddings: Part 2*  \nIntroduction to PyTorch. Implementation of pet linear regression on pure numpy and pytorch. Implementations of CBoW, skip-gram, negative sampling and structured Word2vec models.  \n\nRussian version: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1YruNhE5aEJfLpaCZSKGIaZ1hOQR5qoIG#forceEdit=true\u0026offline=true\u0026sandboxMode=true)\n\nUpdated English version: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1Cgdg_jUIbhMBZiL3DtUQMN6xqFUYjKlg#forceEdit=true\u0026offline=true\u0026sandboxMode=true)\n\n### Week 4: *Convolutional Neural Networks*  \nIntroduction to convolutional networks. Relations between convolutions and n-grams. Simple surname detector on character-level convolutions + fun visualizations.\n\nRussian version: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1Vo_yuiA7xLjavUA_7ayLeosGJyMsyDAt#forceEdit=true\u0026offline=true\u0026sandboxMode=true)\n\nUpdated English version: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1iwhfaHp_L2loxjvbqW9DhO9BaWlIWzpB#forceEdit=true\u0026offline=true\u0026sandboxMode=true)\n\n### Week 5: *RNNs: Part 1*  \nRNNs for text classification. Simple RNN implementation + memorization test. Surname detector in multilingual setup: character-level LSTM classifier.\n\nRussian version: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1-FoMnf7s-BYNM7jT9UF3u9m63h7dSq3_#forceEdit=true\u0026offline=true\u0026sandboxMode=true)\n\nUpdated English version: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1WA9YA30m7xFYfLyptuW7lHROvVGbZWZo#forceEdit=true\u0026offline=true\u0026sandboxMode=true)\n\n### Week 6: *RNNs: Part 2*  \nRNNs for sequence labelling. Part-of-speech tagger implementations based on word embeddings and character-level word embeddings.\n\nRussian version: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1A7dbNANHg8srCemnwFI8WB1wLhvmuJp0#forceEdit=true\u0026offline=true\u0026sandboxMode=true)\n\n### Week 7: *Language Models: Part 1*  \nCharacter-level language model for Russian troll tweets generation: fixed-window model via convolutions and RNN model.  \nSimple conditional language model: surname generation given source language.  \nAnd Toxic Comment Classification Challenge - to apply your skills to a real-world problem.\n\nRussian version: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1W5uaNpKFoaq1gV9N9FpIAEDyrsGGRBBi#forceEdit=true\u0026offline=true\u0026sandboxMode=true)\n\n### Week 8: *Language Models: Part 2*  \nWord-level language model for poetry generation. Pet examples of transfer learning and multi-task learning applied to language models.\n\nRussian version: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1lUlBsdvAYJc5rLHwkOICyFhvns5Ssp1X#forceEdit=true\u0026offline=true\u0026sandboxMode=true)\n\n### Week 9: *Seq2seq*  \nSeq2seq for machine translation and image captioning. Byte-pair encoding, beam search and other usefull stuff for machine translation.\n\nRussian version: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1jSYWuEGwik2lnnvGSU_PyXTFtbRKSyz_#forceEdit=true\u0026offline=true\u0026sandboxMode=true)\n\n### Week 10: *Seq2seq with Attention*  \nSeq2seq with attention for machine translation and image captioning.  \n\nRussian version: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1xZed_YAQf20fYacr9anE7T4EsdC_R0Oy#forceEdit=true\u0026offline=true\u0026sandboxMode=true)\n\n### Week 11: *Transformers \u0026 Text Summarization*  \nImplementation of Transformer model for text summarization. Discussion of Pointer-Generator Networks for text summarization.\n\nRussian version: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1wy5BDHZVEm-vSeH8U4Xh0Sm3bArwVWGU#forceEdit=true\u0026offline=true\u0026sandboxMode=true)\n\n### Week 12: *Dialogue Systems: Part 1*  \nGoal-orientied dialogue systems. Implemention of the multi-task model: intent classifier and token tagger for dialogue manager.\n\nRussian version: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1lNhbHboRYVb-caV7Ktj9cWJnHW7DPD9-#forceEdit=true\u0026offline=true\u0026sandboxMode=true)\n\n### Week 13: *Dialogue Systems: Part 2*  \nGeneral conversation dialogue systems and DSSMs. Implementation of question answering model on SQuAD dataset and chit-chat model on OpenSubtitles dataset.\n\nRussian version: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/19kQoxDWhv9VOfXxZCHcB39qIM30gziCR#forceEdit=true\u0026offline=true\u0026sandboxMode=true)\n\n### Week 14: *Pretrained Models*  \nPretrained models for various tasks: Universal Sentence Encoder for sentence similarity, ELMo for sequence tagging (with a bit of CRF), BERT for SWAG - reasoning about possible continuation.\n\nRussian version: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1WMspBJe-m0mJHb7SbDTj64W-5-3AxW7v#forceEdit=true\u0026offline=true\u0026sandboxMode=true)\n\n### Final Presentation\n[NLP Summary](https://drive.google.com/open?id=16GV-jSGtMAQPJgO_B6q1gLYXL10vM8Ev) - summary of cool stuff that appeared and didn't in the course.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FDanAnastasyev%2FDeepNLP-Course","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FDanAnastasyev%2FDeepNLP-Course","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FDanAnastasyev%2FDeepNLP-Course/lists"}