Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/dair-ai/Mathematics-for-ML

🧮 A collection of resources to learn mathematics for machine learning
https://github.com/dair-ai/Mathematics-for-ML

ai deep-learning machine-learning mathematics ml

Last synced: 12 days ago
JSON representation

🧮 A collection of resources to learn mathematics for machine learning

Awesome Lists containing this project

README 

        
# Mathematics for Machine Learning



A collection of resources to learn and review mathematics for machine learning.



# :book: Books



### Algebra, Topology, Differential Calculus, and Optimization Theory For Computer Science and Machine Learning

*by Jean Gallier and Jocelyn Quaintance*



Includes mathematical concepts for machine learning and computer science. 



Book: https://www.cis.upenn.edu/~jean/math-deep.pdf



### Applied Math and Machine Learning Basics

*by Ian Goodfellow and Yoshua Bengio and Aaron Courville*



This includes the math basics for deep learning from the [Deep Learning](https://www.deeplearningbook.org/) book. 



Chapter: https://www.deeplearningbook.org/contents/part_basics.html



### Mathematics for Machine Learning

*by Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong*



This is probably the place you want to start. Start slowly and work on some examples. Pay close attention to the notation and get comfortable with it.



Book: https://mml-book.github.io



### Probabilistic Machine Learning: An Introduction

*by Kevin Patrick Murphy*



This book contains a comprehensive overview of classical machine learning methods and the principles explaining them. 



Book: https://probml.github.io/pml-book/book1.html



### Mathematics for Deep Learning

*by Brent Werness, Rachel Hu et al.*



This reference contains some mathematical concepts to help build a better understanding of deep learning.



Chapter: https://d2l.ai/chapter_appendix-mathematics-for-deep-learning/index.html 



### The Mathematical Engineering of Deep Learning

*by Benoit Liquet, Sarat Moka and Yoni Nazarathy*



This book provides a complete and concise overview of the mathematical engineering of deep learning. In addition to overviewing deep learning foundations, the treatment includes convolutional neural networks, recurrent neural networks, transformers, generative adversarial networks, reinforcement learning, and multiple tricks of the trade. The focus is on the basic mathematical description of deep learning models, algorithms and methods.



Book: https://deeplearningmath.org



### Bayes Rules! An Introduction to Applied Bayesian Modeling

*by Alicia A. Johnson, Miles Q. Ott, Mine Dogucu*



Great online book covering Bayesian approaches. 



Book: https://www.bayesrulesbook.com/index.html



# 📄 Papers



### The Matrix Calculus You Need For Deep Learning

*by Terence Parr & Jeremy Howard*



In deep learning, you need to understand a bunch of fundamental matrix operations. If you want to dive deep into the math of matrix calculus this is your guide.



Paper: https://arxiv.org/abs/1802.01528



### The Mathematics of AI

*by Gitta Kutyniok*



An article summarising the importance of mathematics in deep learning research and how it’s helping to advance the field.



Paper: https://arxiv.org/pdf/2203.08890.pdf



# 🎥 Video Lectures



### Multivariate Calculus by Imperial College London

*by Dr. Sam Cooper & Dr. David Dye*



Backpropagation is a key algorithm for training deep neural nets that rely on Calculus. Get familiar with concepts like chain rule, Jacobian, gradient descent.



Video Playlist: https://www.youtube.com/playlist?list=PLiiljHvN6z193BBzS0Ln8NnqQmzimTW23



### Mathematics for Machine Learning - Linear Algebra

*by Dr. Sam Cooper & Dr. David Dye*



A great companion to the previous video lectures. Neural networks perform transformations on data and you need linear algebra to get better intuitions of how that is done.



Video Playlist: https://www.youtube.com/playlist?list=PLiiljHvN6z1_o1ztXTKWPrShrMrBLo5P3



### CS229: Machine Learning

*by Anand Avati*



Lectures containing mathematical explanations to many concepts in machine learning. 



Course: https://www.youtube.com/playlist?list=PLoROMvodv4rNH7qL6-efu_q2_bPuy0adh



# 🧮 Math Basics



### The Elements of Statistical Learning

*by Jerome H. Friedman, Robert Tibshirani, and Trevor Hastie*



Machine learning deals with data and in turn uncertainty which is what statistics aims to teach. Get comfortable with topics like estimators, statistical significance, etc.



Book: https://hastie.su.domains/ElemStatLearn/



If you are interested in an introduction to statistical learning, then you might want to check out ["An Introduction to Statistical Learning"](https://www.statlearning.com/).



### Probability Theory: The Logic of Science

*by E. T. Jaynes*



In machine learning, we are interested in building probabilistic models and thus you will come across concepts from probability theory like conditional probability and different probability distributions.



Source: https://bayes.wustl.edu/etj/prob/book.pdf



### Information Theory, Inference and Learning Algorithms

*by David J. C. MacKay*



When you are applying machine learning you are dealing with information processing which in essence relies on ideas from information theory such as entropy and KL Divergence,...



Book: https://www.inference.org.uk/itprnn/book.html



### Statistics and probability

*by Khan Academy*



A complete overview of statistics and probability required for machine learning.



Course: https://www.khanacademy.org/math/statistics-probability



### Linear Algebra Done Right

*by Sheldon Axler*



Slides and video lectures on the popular linear algebra book Linear Algebra Done Right. 



Lecture and Slides: https://linear.axler.net/LADRvideos.html



### Linear Algebra

*by Khan Academy*



Vectors, matrices, operations on them, dot & cross product, matrix multiplication etc. is essential for the most basic understanding of ML maths.



Course: https://www.khanacademy.org/math/linear-algebra



## Calculus

*by Khan Academy*



Precalculus, Differential Calculus, Integral Calculus, Multivariate Calculus



Course: https://www.khanacademy.org/math/calculus-home



---

This collection is far from exhaustive but it should provide a good foundation to start learning some of the mathematical concepts used in machine learning. Reach out on [Twitter](https://twitter.com/omarsar0) if you have any questions.