{"id":15654028,"url":"https://github.com/graykode/deeplearning-study","last_synced_at":"2025-04-30T22:24:45.441Z","repository":{"id":110172933,"uuid":"169181154","full_name":"graykode/DeepLearning-Study","owner":"graykode","description":"This is repository for DeepLearning Study in Kyung Hee University","archived":false,"fork":false,"pushed_at":"2019-04-10T07:36:21.000Z","size":1641,"stargazers_count":28,"open_issues_count":0,"forks_count":10,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-30T20:33:45.133Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/graykode.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-05T02:33:29.000Z","updated_at":"2024-07-15T18:25:42.000Z","dependencies_parsed_at":"2023-04-22T16:45:23.286Z","dependency_job_id":null,"html_url":"https://github.com/graykode/DeepLearning-Study","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/graykode%2FDeepLearning-Study","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/graykode%2FDeepLearning-Study/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/graykode%2FDeepLearning-Study/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/graykode%2FDeepLearning-Study/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/graykode","download_url":"https://codeload.github.com/graykode/DeepLearning-Study/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251790583,"owners_count":21644241,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-03T12:49:05.526Z","updated_at":"2025-04-30T22:24:45.410Z","avatar_url":"https://github.com/graykode.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## DeepLearning Basic Study\n\n\u003cp align=\"center\"\u003e\n    \u003cimg width=\"100\" src=\"https://media-thumbs.golden.com/OLqzmrmwAzY1P7Sl29k2T9WjJdM=/200x200/smart/golden-storage-production.s3.amazonaws.com/topic_images/e08914afa10a4179893eeb07cb5e4713.png\" /\u003e\n    \u003cimg width=\"100\" src=\"http://dcomstudy.com/image/header.png\" /\u003e\n    \u003cimg width=\"100\" src=\"https://avatars2.githubusercontent.com/u/37439249?s=200\u0026v=4\" /\u003e\n\u003c/p\u003e\n\n\nThis is repository for Deep Learning Study in Kyung Hee University Computer Engineering Club `D.COM`.\n\n#### Recommend this study to those who want to review the Machine Learning concept again and to those who have just learned Python.\n\n- I've created a course material that will be accessible to the first person to start Python.\n- [Tae Hwan Jung(@graykode)](https://github.com/graykode) will lead this Study with **Pytorch** for DeepLearning Framework. But I will implement Tensorflow, Pytorch, Keras for beginner. \n- We deal with **basic mathematical theory** and **basic models in Deep Learning** such as ` DNN, CNN, RNN, LSTM ` in 1st Study **All of Code were implemented with less than 30 lines.** \n- We will use `Google Colaboratory` GPU for memory resource so you can run easily in Colab link.(Thank for Google!)\n- First, I made lecture with page link material in Korean, only wrote Contents in English\n\n\n### Contribution Guide\nIf you find English link or helpful link irrespective of language, Please give me contribution in README, Markdown like this.\n\u003e Linear Regression([Eng[(your contribution link), Kor)\n\n\n## Curriculum \nPlease see down Contents.\n- 1 Weeks\n  - Basic Probability Review\n  - Supervisor Learning vs. Un-supervisor Learning\n  - Linear Regression, Logistic Regression `manual` Gradient Descent implementation using `pure python`\n- 2 Weeks\n  - method using Google Colaboratory.\n  - Linear Regression, Logistic Regression Review, Convert `manual` to `auto` implementation using `Pytorch`\n- 3 Weeks \n  - Classification with DNN(Deep Neural Network) in `Pytorch`\n  - apply Regularization(DropOut) concept to DNN\n  - Optimization function in `Pytorch`, mini-batch, SGD, Adagrad, RMSProp, AdaDelta, Adam optimizer\n- 4 Weeks\n  - Basic Convolution Neural Network\n  - load dataset and use data loader with `torchvision`\n  - apply Machine Learning Diagnostic(Train Set, Cross Validation Set, Test Set) concept to DNN\n  - Implementation MNIST Classification using CNN\n- 5 Weeks\n  - Basic RNN(Recurrent Neural Network) and LSTM in Pytorch\n  - Teacher Forcing vs. No Teacher Forcing\n  - Practice : Predict Next word using RNN or LSTM\n- 6 Weeks - Hackathon\n  - Topic1 : Classification Cat , Dog Image, [Dataset](https://github.com/ardamavi/Dog-Cat-Classifier/tree/master/Data/Train_Data)\n  - Topic2 : Classification Positive or Negative Korean Naver Movie, [Dataset](https://github.com/e9t/nsmc)\n\n\n## Contents\n\n#### 0. Review Basic mathematical Theory with pure `Python`\n- **Supervisor Learning vs. Unsupervisor Learning : In this Study, We will deal with only supervisor concept model.**\n\n- Basic Probability Review\n  - Bayers Theorem(Eng, [Kor](https://taeoh-kim.github.io/blog/bayes-theorem%EA%B3%BC-sigmoid%EC%99%80-softmax%EC%82%AC%EC%9D%B4%EC%9D%98-%EA%B4%80%EA%B3%84/)), Bayesian inference in Generative Model(Eng, [Kor](https://statkclee.github.io/statistics/bayesian-ab-testing-in-practice.html))\n  - Generative Model vs. Discriminative Model(Eng, [Kor](http://sanghyukchun.github.io/61/))\n  - Maximum Likelihood vs. Maximum A Posteriori(Eng, [Kor](https://darkpgmr.tistory.com/62))\n  - Maximizing Likelihood is Minimizing Cross-Entropy(Eng, [Kor](https://taeoh-kim.github.io/blog/cross-entropy%EC%9D%98-%EC%A0%95%ED%99%95%ED%95%9C-%ED%99%95%EB%A5%A0%EC%A0%81-%EC%9D%98%EB%AF%B8/))\n- Linear Regression(Eng, [Kor](https://wikidocs.net/4212))\n  - Univariate Linear Regression(Eng, [Kor](https://wikidocs.net/4213)) vs. Multivariate Linear Regression(Eng, [Kor](https://wikidocs.net/7639))\n  - **loss function and activation function** in Linear Regression\n    - activation function : identity map(Eng, [Kor](https://ko.wikipedia.org/wiki/%ED%95%AD%EB%93%B1_%ED%95%A8%EC%88%98))\n    - loss function : MSE function([Eng, Kor](https://en.wikipedia.org/wiki/Mean_squared_error))\n  - Gradient Descent in Linear Regression\n    - manual : [0.LinearRegression(manual)-Univariate.py](https://github.com/graykode/DeepLearning-Study/blob/master/0.LinearRegression(manual)-Univariate.py)\n    - manual : [0.LinearRegression(manual)-Multivariate.py](https://github.com/graykode/DeepLearning-Study/blob/master/0.LinearRegression(manual)-Multivariate.py)\n  - Problem : XOR\n\n- Logistic Regression\n  - What is different with Linear Regression?(Eng, [Kor](https://wikidocs.net/4267))\n  - **loss function and activation function** in Logistic Regression\n    - activation function : [sigmoid ](https://en.wikipedia.org/wiki/Sigmoid_function)vs. [tanh](https://en.wikipedia.org/wiki/Hyperbolic_function) vs. [ReLu](https://en.wikipedia.org/wiki/Rectifier_(neural_networks)) vs. [Softmax](https://en.wikipedia.org/wiki/Softmax_function)\n    - loss function : Maximizing Likelihood is Minimizing Cross-Entropy(Eng, [Kor](https://taeoh-kim.github.io/blog/cross-entropy%EC%9D%98-%EC%A0%95%ED%99%95%ED%95%9C-%ED%99%95%EB%A5%A0%EC%A0%81-%EC%9D%98%EB%AF%B8/))\n  - Gradient Descent in Logistic Regression\n    - manual : [0.LogisticRegression(manual)-Binary.py](https://github.com/graykode/DeepLearning-Study/blob/master/0.LogisticRegression(manual)-Binary.py)\n    - manual : [0.LogisticRegression(manual)-Softmax.py](https://github.com/graykode/DeepLearning-Study/blob/master/0.LogisticRegression(manual)-Softmax.py)\n  - different with binary classification and multi classification(sigmoid vs. Softmax)(Eng, [Kor1](https://wikidocs.net/4291), [Kor2](https://taeoh-kim.github.io/blog/bayes-theorem%EA%B3%BC-sigmoid%EC%99%80-softmax%EC%82%AC%EC%9D%B4%EC%9D%98-%EA%B4%80%EA%B3%84/))\n  - different with Multi-Classification and Multi-labels Classification([Eng](https://stats.stackexchange.com/questions/11859/what-is-the-difference-between-multiclass-and-multilabel-problem), Kor)\n\n- Optimizing\n  - What is batch and mini-batch?(Eng, [Kor](http://shuuki4.github.io/deep%20learning/2016/05/20/Gradient-Descent-Algorithm-Overview.html))\n  - role of Momentum(Eng, [Kor](http://shuuki4.github.io/deep%20learning/2016/05/20/Gradient-Descent-Algorithm-Overview.html))\n  - SGD, Adagrad, RMSProp, AdaDelta, Adam optimizer([Eng](http://ruder.io/optimizing-gradient-descent/?fbclid=IwAR3-EUWRXxLwNlGIEBaETVeVU9VOnDH8hIlp1PJvMG0StbM72gEKMpWA_VA), [Kor](http://shuuki4.github.io/deep%20learning/2016/05/20/Gradient-Descent-Algorithm-Overview.html)) : [2.DNN-Optimization.py](https://github.com/graykode/DeepLearning-Study/blob/master/2.DNN-Optimization.py)\n\n- Regularization\n  - What is Overfitting?(Eng, [Kor](https://wikidocs.net/4269))\n  - Regularization : weight decay\n    - weight decay : Linear Regression(Eng, [Kor](https://wikidocs.net/4330))\n    - weight decay : Logistic Regression(Eng, [Kor](https://wikidocs.net/4331))\n  - Regularization : dropout(Eng, [Kor](https://pythonkim.tistory.com/42))\n\n- Machine Learning Diagnostic\n  - Train Set, Cross Validation Set, Test Set(Eng, [Kor](https://wikidocs.net/4656))\n  - Bias vs. Variance(Eng, [Kor](https://wikidocs.net/4657))\n  - Learning Curves(Eng, [Kor](https://wikidocs.net/4658))\n\n\n#### 1.DeepLearning FrameWork Basic\n- Abstract Model using Pytorch Class : [1.Pytorch-Basic.py](https://github.com/graykode/DeepLearning-Study/blob/master/1.Pytorch-Basic.py)\n- method using Google Colaboratory\n- Convert `manual gradient descent` to `auto graident descent`\n  - [1.LinearRegression(auto)-Univariate.py](https://github.com/graykode/DeepLearning-Study/blob/master/1.LinearRegression(auto)-Univariate.py)\n  - [1.LinearRegression(auto)-Multivariate.py](https://github.com/graykode/DeepLearning-Study/blob/master/1.LinearRegression(auto)-Multivariate.py)\n  - [1.LogisticRegression(auto)-Binary.py](https://github.com/graykode/DeepLearning-Study/blob/master/1.LogisticRegression(auto)-Binary.py)\n  - [1.LogisticRegression(auto)-Softmax.py](https://github.com/graykode/DeepLearning-Study/blob/master/1.LogisticRegression(auto)-Softmax.py)\n\n\n#### 2.DNN(Deep Neural Network)\n- Mathematical Back Propagation in Deep Neural Network(Eng, [Kor1](https://wikidocs.net/4262), [Kor2](https://wikidocs.net/4279))\n- Basic Classification using Deep Neural Network\n  - ~~Classification  : Linear Regression in Deep Neural Network~~\n  - Classification  : Logistic Regression in Deep Neural Network\n    - 1 Layer Classification : [2.DNN-LinearRegression1.py](https://github.com/graykode/DeepLearning-Study/blob/master/2.DNN-LinearRegression1.py)\n    - 2 Layers Classification : [2.DNN-LinearRegression2.py](https://github.com/graykode/DeepLearning-Study/blob/master/2.DNN-LinearRegression2.py)\n- Dropout in Deep Neural Network : [2.DNN-Dropout.py](https://github.com/graykode/DeepLearning-Study/blob/master/2.DNN-Dropout.py)\n\n\n#### 3.DataLoader and basic Dataset and Image handler\n- MNIST : [3.DataLoader-MNIST.py](https://github.com/graykode/DeepLearning-Study/blob/master/3.DataLoader-MNIST.py)\n- Cifar10 : [3.DataLoader-Cifar10.py](https://github.com/graykode/DeepLearning-Study/blob/master/3.DataLoader-Cifar10.py)\n- Cifar100 : [3.DataLoader-Cifar100.py](https://github.com/graykode/DeepLearning-Study/blob/master/3.DataLoader-Cifar100.py)\n- Image Folder : [3.DataLoader-ImageFolder.py](https://github.com/graykode/DeepLearning-Study/blob/master/3.DataLoader-ImageFolder.py)\n\n\n#### 4.CNN(Convolution Neural Network)\n- [awesome lecture](https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-convolutional-neural-networks?fbclid=IwAR21k7YvRmCC1RqAJznzLjDPEf8EaZ2jBGeevX4GkiXruocr1akBAIX9-4U)\n- Structure of CNN\n  - [4.CNN-Introduce.py](https://github.com/graykode/DeepLearning-Study/blob/master/4.CNN-Introduce.py)\n  - Convolutional Layer\n    - Role of filter(=kernel) vs. receptive fields\n    - Role of Padding\n    - Weight sharing in Convolutional Layer\n  - Role of Channel, Reason using Multi Channel\n  - Weight sharing in CNN\n  - Pooling Layer\n    - Max Pooling\n    - Average Pooling\n- FeedForward in Convolution Neural Network\n- Mathematical Back Propagation in Convolution Neural Network\n- Practice : Classification MNIST\n\n\n#### 5.RNN(Recurrent Neural Network)\n- [awesome lecture](https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-recurrent-neural-networks?fbclid=IwAR0rE5QoMJ3l005fhvqoer0Jo_6GiXAF8XM86iWCXD78e3Ud_nDtw_NGzzY)\n- Structure of RNN\n  - [5.RNN-Introduce.py](https://github.com/graykode/DeepLearning-Study/blob/master/5.RNN-Introduce.py)\n  - One-to-one vs. One-to-many vs. Many-to-one vs. Many-to-many\n  - Hidden State\n  - Output Layer\n  - Weight sharing in RNN\n- [Teacher Forcing vs. No Teacher Forcing](https://www.quora.com/What-is-the-teacher-forcing-in-RNN?awc=15748_1550222926_22685c53296b51dfb8cb6be25b6ce096\u0026uiv=6\u0026txtv=8\u0026source=awin\u0026medium=ad\u0026campaign=uad_mkt_en_acq_us_awin\u0026set=awin)\n- FeedForward in Recurrent Neural Network(Eng, [Kor](https://ratsgo.github.io/natural%20language%20processing/2017/03/09/rnnlstm/))\n- Mathematical Back Propagation in Recurrent Neural Network(Eng, [Kor](https://ratsgo.github.io/natural%20language%20processing/2017/03/09/rnnlstm/))\n- Practice : [Predict Next word using RNN](https://github.com/graykode/DeepLearning-Study/blob/master/5.RNN-PredictWord.py)\n\n\n#### 6.LSTM(Long Short Term Memory)\n- Structure of LSTM\n  - [6.LSTM-Introduce.py](https://github.com/graykode/DeepLearning-Study/blob/master/6.LSTM-Introduce.py)\n  - Hidden State, Cell State\n  - Different of RNN with LSTM\n  - Output Layer\n  - Weight sharing in RNN\n- FeedForward in LSTM(Eng, [Kor](https://ratsgo.github.io/natural%20language%20processing/2017/03/09/rnnlstm/))\n- Mathematical Back Propagation in LSTM(Eng, [Kor](https://ratsgo.github.io/natural%20language%20processing/2017/03/09/rnnlstm/))\n- Bi-directional LSTM(BiLSTM)(Eng, [Kor](https://ratsgo.github.io/natural%20language%20processing/2017/10/22/manning/))\n- Practice : [LSTM-AutoComplete with LSTM](https://github.com/graykode/DeepLearning-Study/blob/master/6.LSTM-AutoComplete.py)\n\n\n#### 7. Application Level\n- Vision : Cat or Dog Image Classification.\n- Natural Language Processing : Positive or Negative Classification with Naver Movie Review.\n\n\n## Reference\n- Andrew NG - Machine Learning Lecture\n- Korean Andrew Ng NoteBook : [WikiBook](https://wikidocs.net/book/587)\n\n\n## Author\n- Tae Hwan Jung(Jeff Jung) @graykode\n- Author Email : [nlkey2022@gmail.com](mailto:nlkey2022@gmail.com)\n\n\n## License\n\u003cp xmlns:dct=\"http://purl.org/dc/terms/\"\u003e\n  \u003ca rel=\"license\"\n     href=\"https://creativecommons.org/licenses/by-nc-sa/2.0/kr/\"\u003e\n    \u003cimg src=\"https://wikidocs.net/static/img/by-nc-sa.png\" style=\"border-style: none;\" alt=\"CC0\" /\u003e\n  \u003c/a\u003e\n\u003c/p\u003e","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgraykode%2Fdeeplearning-study","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgraykode%2Fdeeplearning-study","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgraykode%2Fdeeplearning-study/lists"}