https://github.com/roatienza/ml
Foundations of Machine Learning Course Materials.
https://github.com/roatienza/ml
gmm linear-algebra linear-regression machine-learning optimization pca probability svm vector-calculus
Last synced: 3 months ago
JSON representation
Foundations of Machine Learning Course Materials.
- Host: GitHub
- URL: https://github.com/roatienza/ml
- Owner: roatienza
- Created: 2020-08-20T07:15:44.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2022-12-07T00:32:13.000Z (over 2 years ago)
- Last Synced: 2025-03-25T16:12:06.665Z (3 months ago)
- Topics: gmm, linear-algebra, linear-regression, machine-learning, optimization, pca, probability, svm, vector-calculus
- Language: Jupyter Notebook
- Homepage:
- Size: 69.7 MB
- Stars: 25
- Watchers: 3
- Forks: 20
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Foundations of Machine Learning Course Materials.
This is the repository of my course in **Foundations of Machine Learning** (EE 298M/CoE 197M). Similar to my previous course in [Deep Learning](https://github.com/roatienza/Deep-Learning-Experiments), I would like to strike a balance between theory and practical implementation of concepts. The course materials are still under development. Please expect occasional errors, missing parts, bugs, etc. Apologies in advance.
With some modifications, the theory part is based on Mathematics for Machine Learning book. It is freely available at this [link](https://mml-book.github.io/).
**Thank you!** I greatly appreciate the authors for having the book available online. This is helpful for students who have limited resources.
As much as possible, code examples are written to better understand key concepts. The code examples in this course are in [Jupyter Notebook](https://jupyter.org/). I tried using [Google Colab](https://colab.research.google.com/) but encountered errors in saving my notebooks. So, I switched to Jupyter Notebook. In theory, you can upload the notebooks here to Google Colab.
## Roadmap
0. **Why Machine Learning** - Importance of Foundations of Machine Learning, Course Outline
- [Slides](https://github.com/roatienza/ml/blob/master/intro_to_ml/Intro_to_ML.pdf)
1. **ML Toolkit** - Environment, Code Editor, Python, Numpy, Matplotlib, etc
- [Slides](https://github.com/roatienza/ml/blob/master/toolkit/Env_Editor_Python_Numpy.pdf)
2. **Linear Algebra** - Tensors, Operations, Basis, Rank, Spaces/Subspaces, Groups, Linear Mappings
- [Slides](https://github.com/roatienza/ml/blob/master/linear_algebra/Linear_Algebra.pdf)
- Notebook (Code Examples):
- [Introduction to Numpy](https://github.com/roatienza/ml/blob/master/linear_algebra/notebook/numpy_intro.ipynb)
- [Linear Regression](https://github.com/roatienza/ml/blob/master/linear_algebra/notebook/scikit-learn-linear-regression.ipynb)
- [Basis Vectors Transformation](https://github.com/roatienza/ml/blob/master/linear_algebra/notebook/transform.ipynb)
3. **Analytic Geometry** - Distance, Metric, Norm, Inner Product, Basis, Projection, Gram-Schmidt, Rotation
- [Slides](https://github.com/roatienza/ml/blob/master/analytic_geometry/Analytic_Geometry.pdf)
4. **Matrix Decomposition** - Eigenvalues, Eigenvector, Eigendecomposition, Spectral Theorem, Singular-Value Decomposition, Matrix Approximation
- [Slides](https://github.com/roatienza/ml/blob/master/matrix_decompositions/Matrix_Decompositions.pdf)
- Notebook (Code Example):
- [Matrix Approximation](https://github.com/roatienza/ml/blob/master/matrix_decompositions/notebook/matrix_approximation.ipynb)
5. **Vector Calculus** - Learning, Taylor Series, Gradients, Jacobian, Backpropagation, Hessian
- [Slides](https://github.com/roatienza/ml/blob/master/vector_calculus/Vector_Calculus.pdf)
6. **Probability Distributions** - Distributions, Gaussian, Bayes, Sufficient Statistics, Exponential Family, Conjugacy, Transformation
- [Slides](https://github.com/roatienza/ml/blob/master/probability/Probability.pdf)
7. **Optimization** - Gradient Descent, Stochastic Gradient Descent, Convex Optimization, Linear & Quadratic Programming, Convex Conjugate
- [Slides](https://github.com/roatienza/ml/blob/master/optimization/Optimization.pdf)
8. **Machine Learning Principles** - Empirical Risk Minimization (ERM), Maximum Likelihood Estimation (MLE), Maximum A Posteriori (MAP), Intro to Information Theory, Directed Graphical Models
- [Slides](https://github.com/roatienza/ml/blob/master/ml_principles/ML_Principles.pdf)
9. **Linear Regression** - MLE, MAP
- [Slides](https://github.com/roatienza/ml/blob/master/linear_regression/Linear_Regression.pdf)
- Notebook (Code Example):
- [MLE and MAP](https://github.com/roatienza/ml/blob/master/linear_regression/notebook/mle_and_map_poly.ipynb)
10. **Principal Component Analysis** - Low-dimensional Representation, Latent Variable Model
- [Slides](https://github.com/roatienza/ml/blob/master/pca/PCA.pdf)
- Notebook (Code Example):
- [MNIST PCA](https://github.com/roatienza/ml/blob/master/pca/notebook/pca.ipynb)
11. **Gaussian Mixture Models** Responsibilities, Parameter Updates, Expectation Maximization
- [Slides](https://github.com/roatienza/ml/blob/master/gmm/GMM.pdf)
- Notebook (Code Example):
- [GMM 2-component 2D](https://github.com/roatienza/ml/blob/master/gmm/notebook/gmm.ipynb)
12. **Support Vector Machines**
## Cheat Sheets
0. [Numpy](https://github.com/roatienza/ml/blob/master/intro_to_ml/image/numpy_cheatsheet.jpeg)
1. [Scikit-Learn](https://github.com/roatienza/ml/blob/master/intro_to_ml/image/sckit-learn_cheatsheet.jpeg)
2. [dplyr and tidyr](https://github.com/roatienza/ml/blob/master/intro_to_ml/image/dplyr_cheatsheet.jpeg)
3. [Neural Networks](https://github.com/roatienza/ml/blob/master/intro_to_ml/image/neural_networks_cheatsheet.jpeg)
## Appreciation
If you find the materials in this repo useful, please give it a star or fork it.
## Citation
If you find this work useful, please cite:
```
@misc{atienza2020ml,
title={Foundations of Machine Learning},
author={Atienza, Rowel},
year={2020},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/roatienza/ml}},
}
```