{"id":13684716,"url":"https://github.com/ElizaLo/Deep-Learning","last_synced_at":"2025-05-01T00:32:37.278Z","repository":{"id":108180191,"uuid":"260496757","full_name":"ElizaLo/Deep-Learning","owner":"ElizaLo","description":"Implementation of Deep Learning Algorithms and useful information (courses, books, videos, etc.)","archived":false,"fork":false,"pushed_at":"2021-12-12T18:24:36.000Z","size":149,"stargazers_count":23,"open_issues_count":0,"forks_count":3,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-04-18T19:20:52.921Z","etag":null,"topics":["awesome","awesome-ai","awesome-deep-learning","awesome-dl","awesome-list","awesome-lists","deep-learning","deep-learning-algorithms","deep-learning-book","deep-neural-networks","deeplearning"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ElizaLo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-05-01T15:44:38.000Z","updated_at":"2025-04-08T05:44:05.000Z","dependencies_parsed_at":null,"dependency_job_id":"9b35d360-ee73-450d-94f8-f784dafd7311","html_url":"https://github.com/ElizaLo/Deep-Learning","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ElizaLo%2FDeep-Learning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ElizaLo%2FDeep-Learning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ElizaLo%2FDeep-Learning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ElizaLo%2FDeep-Learning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ElizaLo","download_url":"https://codeload.github.com/ElizaLo/Deep-Learning/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251697000,"owners_count":21629326,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["awesome","awesome-ai","awesome-deep-learning","awesome-dl","awesome-list","awesome-lists","deep-learning","deep-learning-algorithms","deep-learning-book","deep-neural-networks","deeplearning"],"created_at":"2024-08-02T14:00:37.627Z","updated_at":"2025-05-01T00:32:37.235Z","avatar_url":"https://github.com/ElizaLo.png","language":null,"funding_links":[],"categories":["Awesome lists for learning AI","Other Lists"],"sub_categories":["TeX Lists"],"readme":"\u003cimg src=\"https://github.com/ElizaLo/Deep-Learning/blob/master/img/Banner_DL.png\" width=\"900\" height=\"100\"\u003e\n\n[![Hits](https://hits.seeyoufarm.com/api/count/incr/badge.svg?url=https%3A%2F%2Fgithub.com%2FElizaLo%2FDeep-Learning\u0026count_bg=%2304749C\u0026title_bg=%23B5B5BC\u0026icon=python.svg\u0026icon_color=%23E7E7E7\u0026title=Repository+Views\u0026edge_flat=false)](https://hits.seeyoufarm.com)\n\nThis repository contains examples of deep learning algorithms implemented in Python with mathematics behind them being explained.\n\n\u003e - [ ] For **Machine Learning** algorithms please check [Machine Learning](https://github.com/ElizaLo/Machine-Learning) repository.\n\n\u003e - [ ] For **Natural Language Processing** (NLU = NLP + NLG) please check [Natural Language Processing](https://github.com/ElizaLo/NLP-Natural-Language-Processing) repository.\n\n\u003e - [ ] For **Computer Vision** please check [Computer Vision](https://github.com/ElizaLo/Computer-Vision) repository.\n\n\n## 🎓 University Courses \n\n- [ ] [CS 231N: Convolutional Neural Networks for Visual Recognition, Stanford](https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv)\n- [ ] [CS 224N: Natural Language Processing with Deep Learning, Stanford](https://www.youtube.com/playlist?list=PLoROMvodv4rOhcuXMZkNm7j3fVwBBY42z)\n- [ ] [Machine Learning Crash Course](https://techdevguide.withgoogle.com/paths/machine-learning/featured/ml-crash-course#)\n- [ ] [fast.ai: Practical Deep Learning for Coders](https://course.fast.ai)\"\n- [ ] [CS 285: Deep Reinforcement Learning, UC Berkeley](http://rail.eecs.berkeley.edu/deeprlcourse/)\n- [ ] [CSC 2541: Differentiable Inference and Generative Models](http://www.cs.toronto.edu/~duvenaud/courses/csc2541/index.html)\n- [ ] [MIT 6.S191: Introduction to Deep Learning](http://introtodeeplearning.com)\n  - [MIT 6.S191: Introduction to Deep Learning ](https://www.youtube.com/playlist?list=PLtBw6njQRU-rwp5__7C0oIVt26ZgjG9NI)\n- [ ] [Frontiers of Deep Learning (Simons Institute)](https://www.youtube.com/playlist?list=PLgKuh-lKre11ekU7g-Z_qsvjDD8cT-hi9)\n  - [Course website](https://simons.berkeley.edu/workshops/dl2019-1)\n- [ ] [New Deep Learning Techniques](https://www.youtube.com/playlist?list=PLHyI3Fbmv0SdM0zXj31HWjG9t9Q0v2xYN)\n  - [Course website](http://www.ipam.ucla.edu/programs/workshops/new-deep-learning-techniques/?tab=overview)\n- [ ] [Geometry of Deep Learning (Microsoft Research)](https://www.youtube.com/playlist?list=PLD7HFcN7LXRe30qq36It2XCljxc340O_d)\n  - [Course website](https://www.microsoft.com/en-us/research/event/ai-institute-2019/)\n- [ ] [Deep Multi-Task and Meta Learning (Stanford CS330)](https://www.youtube.com/playlist?list=PLoROMvodv4rMC6zfYmnD7UG3LVvwaITY5)\n  - [Course Website](http://cs330.stanford.edu/)\n- [ ] [Advanced Deep Learning \u0026 Reinforcement Learning 2020 (DeepMind / UCL)](https://www.youtube.com/playlist?list=PLqYmG7hTraZDNJre23vqCGIVpfZ_K2RZs)\n- [ ] [Deep Reinforcment Learning, Decision Making and Control (UC Berkeley CS285)](https://www.youtube.com/playlist?list=PLkFD6_40KJIwhWJpGazJ9VSj9CFMkb79A)\n- [ ] [Full Stack Deep Learning 2019](https://www.youtube.com/playlist?list=PL1T8fO7ArWlcf3Hc4VMEVBlH8HZm_NbeB)\n- [ ] [Emerging Challenges in Deep Learning](https://www.youtube.com/playlist?list=PLgKuh-lKre10BpafDrv0fg2VNUweWXWVd)\n- [ ] [Deep|Bayes 2019 Summer School](https://www.youtube.com/playlist?list=PLe5rNUydzV9QHe8VDStpU0o8Yp63OecdW)\n- [ ] [Workshop on Theory of Deep Learning: Where next (Institure for Advanced Study)](https://www.youtube.com/playlist?list=PLdDZb3TwJPZ5dqqg_S-rgJqSFeH4DQqFQ)\n- [ ] [Deep Learning: Alchemy or Science? (Institure for Advanced Study)](https://www.youtube.com/playlist?list=PLdDZb3TwJPZ7aAxhIHALBoh8l6-UxmMNP)\n\n## 🔹 Coursera Courses\n\n\u003cdetails\u003e\n \u003csummary\u003e List of Coursera Courses \u003c/summary\u003e \n\u003cul style=\"list-style-type:circle;\"\u003e\n  \u003cli\u003e \u003ca href=\"https://www.coursera.org/specializations/deep-learning\"\u003e Spesialization: Deep Learning Specialization, Andrew Ng \u003c/a\u003e \u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"\"\u003e \u003c/a\u003e \u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"\"\u003e \u003c/a\u003e \u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"\"\u003e \u003c/a\u003e \u003c/li\u003e\n\u003c/ul\u003e\n\u003c/details\u003e\n\n\n## 📚 Books\n\u003cdetails\u003e\n \u003csummary\u003e List of Books \u003c/summary\u003e \n\u003cul style=\"list-style-type:circle;\"\u003e\n  \u003cli\u003e\u003ca href=\"https://www.litmir.me/bd/?b=643725\u0026p=1\"\u003eГлубокое обучение. Погружение в мир нейронных сетей, Сергей Николенко \u003c/a\u003e \u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"http://www.deeplearningbook.org/front_matter.pdf\"\u003e Deep Learning, Goodfellow \u003c/a\u003e \u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://d2l.ai/index.html\"\u003e Dive into Deep Learning\u003c/a\u003e \u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"\"\u003e \u003c/a\u003e \u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"\"\u003e \u003c/a\u003e \u003c/li\u003e\n\u003c/ul\u003e\n\u003c/details\u003e\n\n## Papers\n\n| Title | Description, Information |\n| :---:         |          :--- |\n|[Deep Learning Papers Reading Roadmap](https://github.com/floodsung/Deep-Learning-Papers-Reading-Roadmap)|Deep Learning papers reading roadmap for anyone who are eager to learn this amazing tech!|\n\n\n## Videos\n\u003cdetails\u003e\n \u003csummary\u003e Other useful links \u003c/summary\u003e \n\u003cul style=\"list-style-type:circle;\"\u003e\n  \u003cli\u003e\u003ca href=\"https://www.youtube.com/channel/UCdeSxuESqLOxuuwXNnqqbrA\"\u003e Семинары по машинному обучению JetBrains Research\u003c/a\u003e \u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://www.youtube.com/channel/UCeq6ZIlvC9SVsfhfKnSvM9w\"\u003e ML Trainings\u003c/a\u003e \u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"\"\u003e \u003c/a\u003e \u003c/li\u003e\n\u003c/ul\u003e\n\u003c/details\u003e\n\n\n## :octocat: GitHub Repositories\n\n| Title | Description, Information |\n| :---:         |          :--- |\n|[NVIDIA Deep Learning Examples for Tensor Cores](https://github.com/NVIDIA/DeepLearningExamples)|Deep Learning Examples|\n\n## Contests\n\u003cdetails\u003e\n \u003csummary\u003e Other useful links \u003c/summary\u003e \n\u003cul style=\"list-style-type:circle;\"\u003e\n  \u003cli\u003e\u003ca href=\"https://www.kaggle.com\"\u003e Kaggle \u003c/a\u003e \u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://boosters.pro\"\u003e Boosters \u003c/a\u003e \u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://mlcontests.com\"\u003e Machine Learning Contests \u003c/a\u003e \u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"\"\u003e \u003c/a\u003e \u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"\"\u003e \u003c/a\u003e \u003c/li\u003e\n\u003c/ul\u003e\n\u003c/details\u003e\n\n## 📌 Other\n\u003cdetails\u003e\n \u003csummary\u003e Other useful links \u003c/summary\u003e \n\u003cul style=\"list-style-type:circle;\"\u003e\n  \u003cli\u003e\u003ca href=\"https://ods.ai\"\u003e Open Data Science\u003c/a\u003e \u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://dyakonov.org\"\u003e Блог Александра Дьяконова\u003c/a\u003e \u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"\"\u003e \u003c/a\u003e \u003c/li\u003e\n\u003c/ul\u003e\n\u003c/details\u003e\n\n* [Caffe](https://github.com/weiliu89/caffe) – a fast open framework for deep learning;\n* [Deep Learning](http://www.deeplearningbook.org) - Ian Goodfellow, Yoshua Bengio, and Aaron Courville (2016);\n* [Deep Learning](https://www.udacity.com/course/deep-learning--ud730) от Google — короткий курс для продвинутых. Основное внимание уделяется библиотеке для глубинного обучения TensorFlow;\n* [Deep Learning at Oxford](https://www.youtube.com/playlist?list=PLE6Wd9FR--EfW8dtjAuPoTuPcqmOV53Fu) (2015) – a YouTube playlist with lectures ([read more](http://www.cs.ox.ac.uk/teaching/courses/2014-2015/ml/));\n* [awesome-deep-vision](https://github.com/kjw0612/awesome-deep-vision) – a curated list of deep learning resources for computer vision;\n* [awesome-deep-learning-papers](https://github.com/terryum/awesome-deep-learning-papers) – a curated list of the most cited deep learning papers (since 2010); \n* [Deep Learning Tutorials](https://github.com/subokita/DeepLearningTutorials) – notes and code;\n* [dl-docker](https://github.com/saiprashanths/dl-docker) – an all-in-one Docker image for deep learning. Contains all the popular DL frameworks (TensorFlow, Theano, Torch, Caffe, etc.);\n* [Self-Study Courses for Deep Learning](https://developer.nvidia.com/deep-learning-courses) от NVDIA — self-paced classes for deep learning that feature interactive lectures, hands-on exercises, and recordings of the office hours Q\u0026A with instructors. You’ll learn everything you need to design, train, and integrate neural network-powered artificial intelligence into your applications with widely used open-source frameworks and NVIDIA software. During the hands-on exercises, you will use GPUs and deep learning software in the cloud;\n* [deep-rl-tensorflow](https://github.com/carpedm20/deep-rl-tensorflow) - ensorFlow implementation of Deep Reinforcement Learning papers;\n* [TensorFlow 101](https://github.com/sjchoi86/Tensorflow-101) – Tensorflow tutorials;\n* [Introduction to Deep Learning for Image Recognition](https://github.com/rouseguy/scipyUS2016_dl-image) – this notebook accompanies the Introduction to Deep Learning for Image Recognition workshop to explain the core concepts of deep learning with emphasis on classifying images as the application;\n\n## Main skills required by the Deep Learning Engineer / Deep Learning Research Engineer\n\n\u003e The [research](https://apps.ucu.edu.ua/en/articles-and-research/data-science-job-market-2020-1/data-scientists-skills-2020-1/) made by **Faculty of Applied Sciences at UCU**. Link on main [article](https://apps.ucu.edu.ua/en/articles-and-research/data-science-job-market-2020-1/).\n\n### Deep Learning Engineer / Deep Learning Research Engineer\n\n1. Python3: numpy, scikit-learn, pandas, scipy.\n2. Statistics (regression, properties of distributions, statistical tests, and proper usage, etc.) and probability theory.\n3. Deep learning frameworks:  Tensorflow, PyTorch; MxNet, Caffe, Keras.\n4. Deep learning architectures: VGG, ResNet, Inception, MobileNet.\n5. Deepnets, hyperparameter optimization, visualization, interpretation.\n6. Machine learning models.\n\n### Python for Deep Learning and Research\n\n- Basic algorithms and common tasks\n- Classical algorithms\n- Computational complexity\n- Useful Libraries and Frameworks\n- CPU vs GPU parallelization\n- Cloud and GPU Integration\n- Data Visualization\n- Vectors and Vectorization\n- Image Processing\n- Language Processing\n\n### Mathematics for Deep Learning\n\n- Common Notation and Core Ideas\n- Linear Algebra\n- N-dim Spaces\n- Vectors, Matrices and Operators\n- Mathematical and Function Analysis calculus\n- Derivative and Partial derivative\n- Chain Rule\n- Probability theory\n- Introduction to Statistics\n\n### Linear, Polynomial and Multivariate Regression\n\n- Price prediction Task\n- Linear Regression\n- Least square method\n- Loss Function\n- Optimization Task\n- Gradient Descent\n- MLE — Maximum Likelihood Estimation\n- Data Preprocessing\n- Model Visualization\n- Data Normalization\n- Polynomial Regression\n- Multivariate Regression\n\n### Introduction Computer Vision\n\n- Basic idea of Computer Vision\n- Classical Computer Vision\n- Deep Learning and CV\n- Core Idea of Semantic Gap\n- Classification Task\n- N-dim Spaces and Metrics\n- Common datasets\n- Mnist and Fashion-Mnist\n- Cifar10 and Cifar100\n- Cats vs Dogs\n- ImageNet and MS COCO\n- Euclidean Distance\n- Nearest Neighbour\n\n### Classification and Computer Vision\n\n- Image Classification\n- Cosine Similarity\n- Manhattan distance\n- KNN\n- Train / Val / Test data split\n- Logistic Regression\n- Logistic Regression and Maximum Likelihood Estimation\n- Loss function and Cross Entropy\n- Accuracy and Metrics\n- Precision, Recall and F1\n\n### Neural Networks\n\n- Rosenblatt’s Perceptron\n- Artificial Neuron\n- Warren McCulloch and Walter Pitts Neuron\n- Fully Connected (Linear, Dense, Affine) Layer\n- Activation Layers\n- BackPropagation Algorithm\n- Stochastic Gradient Descent\n- Biological Neuron and Analogy\n\n### Computation graphs and Deep Learning Frameworks\n\n- Computational graphs\n- Differentiable graphs\n- Deep Learning Frameworks\n- Custom Framework Realization\n- Linear operations and Activation Realizations\n- Main Blocks Of Deep Learning FrameWorks\n- Custom Model and Train\n- Optimizator realization\n- TensorFlow\n- Keras\n- PyTorch\n\n### Deep Learning\n- Neural Networks Problems\n- Activation Functions\n- Weights Initialization\n- Initialization Techniks\n- Overfitting and Underfitting\n- Regularization Methods\n- L1 and L2 Regularization\n- Ensemble of Models\n- Dropout\n- Hyper Parameters Search\n- Optimizations behind SGD\n- Momentum and Nesterov Momentum\n- Adagrad, RMSprop\n- Adam, Nadam\n- Batch-Normalization\n\n### Unsupervised Learning\n\n- Dimensionality reduction\n- Feature Learning\n- Vector Representation\n- Embeddings\n- Kernel Method\n- Clusterization\n- k-means Clusterization\n- Hierarchical Clusterization\n- Neural Networks and Unsupervised Learning\n- Autoencoders\n- Autoencoders architectures\n- Tasks for Autoencoders\n- Problem of Image Generation\n- Image Denoising Task\n\n### Introduction to Deep Learning in Computer Vision\n\n- Problems of Fully Connected Neural Networks\n- Towards Convolution Neural Network\n- CNN as feature extractor\n- Computer Vision tasks\n- Transfer Learning\n- Transfer Learning in Practice\n- What Next (breath: CNN Architectures, Image Detection, Segmentation, GANs)\n\n### Introduction to Natural Language Processing\n\n- Introduction to Natural Language Processing\n- Text classification\n- Words Preprocessing and Representation\n- Part-of-Speech tagging (PoS tagging)\n- Tokenization, Lemmatization and Stemming\n- Bag of Words\n- TF-IDF\n- Distributive semantics\n- Vector Semantics\n- Term-document matrix\n- Word context matrix\n- Dense Vectors and Embeddings\n- Word2Vec\n- What Next (breath: RNN, Seq2Seq, Attention, Transformers, Modern Language Models)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FElizaLo%2FDeep-Learning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FElizaLo%2FDeep-Learning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FElizaLo%2FDeep-Learning/lists"}