Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/ElizaLo/Deep-Learning

Implementation of Deep Learning Algorithms and useful information (courses, books, videos, etc.)
https://github.com/ElizaLo/Deep-Learning

List: Deep-Learning

awesome awesome-ai awesome-deep-learning awesome-dl awesome-list awesome-lists deep-learning deep-learning-algorithms deep-learning-book deep-neural-networks deeplearning

Last synced: 3 months ago
JSON representation

Implementation of Deep Learning Algorithms and useful information (courses, books, videos, etc.)

Awesome Lists containing this project

README 

        



[![Hits](https://hits.seeyoufarm.com/api/count/incr/badge.svg?url=https%3A%2F%2Fgithub.com%2FElizaLo%2FDeep-Learning&count_bg=%2304749C&title_bg=%23B5B5BC&icon=python.svg&icon_color=%23E7E7E7&title=Repository+Views&edge_flat=false)](https://hits.seeyoufarm.com)



This repository contains examples of deep learning algorithms implemented in Python with mathematics behind them being explained.



> - [ ] For **Machine Learning** algorithms please check [Machine Learning](https://github.com/ElizaLo/Machine-Learning) repository.



> - [ ] For **Natural Language Processing** (NLU = NLP + NLG) please check [Natural Language Processing](https://github.com/ElizaLo/NLP-Natural-Language-Processing) repository.



> - [ ] For **Computer Vision** please check [Computer Vision](https://github.com/ElizaLo/Computer-Vision) repository.



## 🎓 University Courses 



- [ ] [CS 231N: Convolutional Neural Networks for Visual Recognition, Stanford](https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv)

- [ ] [CS 224N: Natural Language Processing with Deep Learning, Stanford](https://www.youtube.com/playlist?list=PLoROMvodv4rOhcuXMZkNm7j3fVwBBY42z)

- [ ] [Machine Learning Crash Course](https://techdevguide.withgoogle.com/paths/machine-learning/featured/ml-crash-course#)

- [ ] [fast.ai: Practical Deep Learning for Coders](https://course.fast.ai)"

- [ ] [CS 285: Deep Reinforcement Learning, UC Berkeley](http://rail.eecs.berkeley.edu/deeprlcourse/)

- [ ] [CSC 2541: Differentiable Inference and Generative Models](http://www.cs.toronto.edu/~duvenaud/courses/csc2541/index.html)

- [ ] [MIT 6.S191: Introduction to Deep Learning](http://introtodeeplearning.com)

  - [MIT 6.S191: Introduction to Deep Learning ](https://www.youtube.com/playlist?list=PLtBw6njQRU-rwp5__7C0oIVt26ZgjG9NI)

- [ ] [Frontiers of Deep Learning (Simons Institute)](https://www.youtube.com/playlist?list=PLgKuh-lKre11ekU7g-Z_qsvjDD8cT-hi9)

  - [Course website](https://simons.berkeley.edu/workshops/dl2019-1)

- [ ] [New Deep Learning Techniques](https://www.youtube.com/playlist?list=PLHyI3Fbmv0SdM0zXj31HWjG9t9Q0v2xYN)

  - [Course website](http://www.ipam.ucla.edu/programs/workshops/new-deep-learning-techniques/?tab=overview)

- [ ] [Geometry of Deep Learning (Microsoft Research)](https://www.youtube.com/playlist?list=PLD7HFcN7LXRe30qq36It2XCljxc340O_d)

  - [Course website](https://www.microsoft.com/en-us/research/event/ai-institute-2019/)

- [ ] [Deep Multi-Task and Meta Learning (Stanford CS330)](https://www.youtube.com/playlist?list=PLoROMvodv4rMC6zfYmnD7UG3LVvwaITY5)

  - [Course Website](http://cs330.stanford.edu/)

- [ ] [Advanced Deep Learning & Reinforcement Learning 2020 (DeepMind / UCL)](https://www.youtube.com/playlist?list=PLqYmG7hTraZDNJre23vqCGIVpfZ_K2RZs)

- [ ] [Deep Reinforcment Learning, Decision Making and Control (UC Berkeley CS285)](https://www.youtube.com/playlist?list=PLkFD6_40KJIwhWJpGazJ9VSj9CFMkb79A)

- [ ] [Full Stack Deep Learning 2019](https://www.youtube.com/playlist?list=PL1T8fO7ArWlcf3Hc4VMEVBlH8HZm_NbeB)

- [ ] [Emerging Challenges in Deep Learning](https://www.youtube.com/playlist?list=PLgKuh-lKre10BpafDrv0fg2VNUweWXWVd)

- [ ] [Deep|Bayes 2019 Summer School](https://www.youtube.com/playlist?list=PLe5rNUydzV9QHe8VDStpU0o8Yp63OecdW)

- [ ] [Workshop on Theory of Deep Learning: Where next (Institure for Advanced Study)](https://www.youtube.com/playlist?list=PLdDZb3TwJPZ5dqqg_S-rgJqSFeH4DQqFQ)

- [ ] [Deep Learning: Alchemy or Science? (Institure for Advanced Study)](https://www.youtube.com/playlist?list=PLdDZb3TwJPZ7aAxhIHALBoh8l6-UxmMNP)



## 🔹 Coursera Courses



  List of Coursera Courses  




## 📚 Books



  List of Books  




## Papers



| Title | Description, Information |

| :---:         |          :--- |

|[Deep Learning Papers Reading Roadmap](https://github.com/floodsung/Deep-Learning-Papers-Reading-Roadmap)|Deep Learning papers reading roadmap for anyone who are eager to learn this amazing tech!|



## Videos



  Other useful links  




## :octocat: GitHub Repositories



| Title | Description, Information |

| :---:         |          :--- |

|[NVIDIA Deep Learning Examples for Tensor Cores](https://github.com/NVIDIA/DeepLearningExamples)|Deep Learning Examples|



## Contests



  Other useful links  




## 📌 Other



  Other useful links  




* [Caffe](https://github.com/weiliu89/caffe) – a fast open framework for deep learning;

* [Deep Learning](http://www.deeplearningbook.org) - Ian Goodfellow, Yoshua Bengio, and Aaron Courville (2016);

* [Deep Learning](https://www.udacity.com/course/deep-learning--ud730) от Google — короткий курс для продвинутых. Основное внимание уделяется библиотеке для глубинного обучения TensorFlow;

* [Deep Learning at Oxford](https://www.youtube.com/playlist?list=PLE6Wd9FR--EfW8dtjAuPoTuPcqmOV53Fu) (2015) – a YouTube playlist with lectures ([read more](http://www.cs.ox.ac.uk/teaching/courses/2014-2015/ml/));

* [awesome-deep-vision](https://github.com/kjw0612/awesome-deep-vision) – a curated list of deep learning resources for computer vision;

* [awesome-deep-learning-papers](https://github.com/terryum/awesome-deep-learning-papers) – a curated list of the most cited deep learning papers (since 2010); 

* [Deep Learning Tutorials](https://github.com/subokita/DeepLearningTutorials) – notes and code;

* [dl-docker](https://github.com/saiprashanths/dl-docker) – an all-in-one Docker image for deep learning. Contains all the popular DL frameworks (TensorFlow, Theano, Torch, Caffe, etc.);

* [Self-Study Courses for Deep Learning](https://developer.nvidia.com/deep-learning-courses) от NVDIA — self-paced classes for deep learning that feature interactive lectures, hands-on exercises, and recordings of the office hours Q&A with instructors. You’ll learn everything you need to design, train, and integrate neural network-powered artificial intelligence into your applications with widely used open-source frameworks and NVIDIA software. During the hands-on exercises, you will use GPUs and deep learning software in the cloud;

* [deep-rl-tensorflow](https://github.com/carpedm20/deep-rl-tensorflow) - ensorFlow implementation of Deep Reinforcement Learning papers;

* [TensorFlow 101](https://github.com/sjchoi86/Tensorflow-101) – Tensorflow tutorials;

* [Introduction to Deep Learning for Image Recognition](https://github.com/rouseguy/scipyUS2016_dl-image) – this notebook accompanies the Introduction to Deep Learning for Image Recognition workshop to explain the core concepts of deep learning with emphasis on classifying images as the application;



## Main skills required by the Deep Learning Engineer / Deep Learning Research Engineer



> The [research](https://apps.ucu.edu.ua/en/articles-and-research/data-science-job-market-2020-1/data-scientists-skills-2020-1/) made by **Faculty of Applied Sciences at UCU**. Link on main [article](https://apps.ucu.edu.ua/en/articles-and-research/data-science-job-market-2020-1/).



### Deep Learning Engineer / Deep Learning Research Engineer



1. Python3: numpy, scikit-learn, pandas, scipy.

2. Statistics (regression, properties of distributions, statistical tests, and proper usage, etc.) and probability theory.

3. Deep learning frameworks:  Tensorflow, PyTorch; MxNet, Caffe, Keras.

4. Deep learning architectures: VGG, ResNet, Inception, MobileNet.

5. Deepnets, hyperparameter optimization, visualization, interpretation.

6. Machine learning models.



### Python for Deep Learning and Research



- Basic algorithms and common tasks

- Classical algorithms

- Computational complexity

- Useful Libraries and Frameworks

- CPU vs GPU parallelization

- Cloud and GPU Integration

- Data Visualization

- Vectors and Vectorization

- Image Processing

- Language Processing



### Mathematics for Deep Learning



- Common Notation and Core Ideas

- Linear Algebra

- N-dim Spaces

- Vectors, Matrices and Operators

- Mathematical and Function Analysis calculus

- Derivative and Partial derivative

- Chain Rule

- Probability theory

- Introduction to Statistics



### Linear, Polynomial and Multivariate Regression



- Price prediction Task

- Linear Regression

- Least square method

- Loss Function

- Optimization Task

- Gradient Descent

- MLE — Maximum Likelihood Estimation

- Data Preprocessing

- Model Visualization

- Data Normalization

- Polynomial Regression

- Multivariate Regression



### Introduction Computer Vision



- Basic idea of Computer Vision

- Classical Computer Vision

- Deep Learning and CV

- Core Idea of Semantic Gap

- Classification Task

- N-dim Spaces and Metrics

- Common datasets

- Mnist and Fashion-Mnist

- Cifar10 and Cifar100

- Cats vs Dogs

- ImageNet and MS COCO

- Euclidean Distance

- Nearest Neighbour



### Classification and Computer Vision



- Image Classification

- Cosine Similarity

- Manhattan distance

- KNN

- Train / Val / Test data split

- Logistic Regression

- Logistic Regression and Maximum Likelihood Estimation

- Loss function and Cross Entropy

- Accuracy and Metrics

- Precision, Recall and F1



### Neural Networks



- Rosenblatt’s Perceptron

- Artificial Neuron

- Warren McCulloch and Walter Pitts Neuron

- Fully Connected (Linear, Dense, Affine) Layer

- Activation Layers

- BackPropagation Algorithm

- Stochastic Gradient Descent

- Biological Neuron and Analogy



### Computation graphs and Deep Learning Frameworks



- Computational graphs

- Differentiable graphs

- Deep Learning Frameworks

- Custom Framework Realization

- Linear operations and Activation Realizations

- Main Blocks Of Deep Learning FrameWorks

- Custom Model and Train

- Optimizator realization

- TensorFlow

- Keras

- PyTorch



### Deep Learning

- Neural Networks Problems

- Activation Functions

- Weights Initialization

- Initialization Techniks

- Overfitting and Underfitting

- Regularization Methods

- L1 and L2 Regularization

- Ensemble of Models

- Dropout

- Hyper Parameters Search

- Optimizations behind SGD

- Momentum and Nesterov Momentum

- Adagrad, RMSprop

- Adam, Nadam

- Batch-Normalization



### Unsupervised Learning



- Dimensionality reduction

- Feature Learning

- Vector Representation

- Embeddings

- Kernel Method

- Clusterization

- k-means Clusterization

- Hierarchical Clusterization

- Neural Networks and Unsupervised Learning

- Autoencoders

- Autoencoders architectures

- Tasks for Autoencoders

- Problem of Image Generation

- Image Denoising Task



### Introduction to Deep Learning in Computer Vision



- Problems of Fully Connected Neural Networks

- Towards Convolution Neural Network

- CNN as feature extractor

- Computer Vision tasks

- Transfer Learning

- Transfer Learning in Practice

- What Next (breath: CNN Architectures, Image Detection, Segmentation, GANs)



### Introduction to Natural Language Processing



- Introduction to Natural Language Processing

- Text classification

- Words Preprocessing and Representation

- Part-of-Speech tagging (PoS tagging)

- Tokenization, Lemmatization and Stemming

- Bag of Words

- TF-IDF

- Distributive semantics

- Vector Semantics

- Term-document matrix

- Word context matrix

- Dense Vectors and Embeddings

- Word2Vec

- What Next (breath: RNN, Seq2Seq, Attention, Transformers, Modern Language Models)