https://github.com/zellyn/deeplearning-class-2011
Code for Deep Learning class at Google
https://github.com/zellyn/deeplearning-class-2011
Last synced: 3 months ago
JSON representation
Code for Deep Learning class at Google
- Host: GitHub
- URL: https://github.com/zellyn/deeplearning-class-2011
- Owner: zellyn
- Created: 2011-05-24T22:18:28.000Z (about 14 years ago)
- Default Branch: master
- Last Pushed: 2011-09-01T16:47:42.000Z (almost 14 years ago)
- Last Synced: 2025-04-01T19:03:24.650Z (3 months ago)
- Language: Matlab
- Homepage:
- Size: 52.6 MB
- Stars: 73
- Watchers: 11
- Forks: 69
- Open Issues: 0
-
Metadata Files:
- Readme: README.org
Awesome Lists containing this project
README
* Overview
My code for the Deep Learning class exercises. There should be nothing
proprietary in here. For the earlier exercises, I tried to create
parallel implementations in Octave and NumPy. Later on, class-supplied
helper code necessitated the use of Matlab (for now).
* Materials
- [[http://openclassroom.stanford.edu/MainFolder/CoursePage.php?course=DeepLearning][OpenClassroom Regression Tutorial]]
- [[http://ufldl.stanford.edu/wiki/index.php/UFLDL_Tutorial][UFLDF Tutorial Wiki]]
* Note
- The L-BFGS Matlab code is licensed by Stanford under a Creative Commons,
Attribute, Non-Commercial license. Please read the
[[http://ufldl.stanford.edu/wiki/index.php/Exercise:Sparse_Autoencoder#Sparse_autoencoder_implementation][details on the UFLDL wiki]].
- The MNIST digit data comes from [[http://yann.lecun.com/exdb/mnist/]].
* Questions
** UFLDL
"Since J(W,b) is a non-convex function, gradient descent is
susceptible to local optima; however, in practice gradient descent
usually works fairly well." - [[http://ufldl.stanford.edu/wiki/index.php/Backpropagation_Algorithm][UFLDL/Backpropagation]]
Why? Is it *almost* convex? Are the local optima all of a similar
quality? Are any of the variations (squared error / squared error +
weight decay / squared error + weight decay + sparsity constraints)
convex?
* Tasks
** Python
*** TODO Get [[file:ufldf/stackedae_exercise/stackedae_exercise.py][stackedae_exercise.py]] to work.
*** TODO Implement [[file:ufldf/linear_decoder_exercise][linear_decoder_exercise]].
*** TODO Implement [[file:ufldf/cnn_exercise][cnn_exercise]].