{"id":21254788,"url":"https://github.com/mlelarge/icfp-ml","last_synced_at":"2025-07-11T02:31:29.604Z","repository":{"id":104970372,"uuid":"596292360","full_name":"mlelarge/icfp-ml","owner":"mlelarge","description":"machine learning course for ICFP","archived":false,"fork":false,"pushed_at":"2024-03-26T11:47:08.000Z","size":18203,"stargazers_count":4,"open_issues_count":0,"forks_count":7,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-03-26T12:57:18.797Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mlelarge.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-02-01T21:40:46.000Z","updated_at":"2024-02-28T14:14:59.000Z","dependencies_parsed_at":"2024-02-29T15:54:39.295Z","dependency_job_id":"c3901884-87ca-4d3a-a6e0-8cd29cb85f06","html_url":"https://github.com/mlelarge/icfp-ml","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlelarge%2Ficfp-ml","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlelarge%2Ficfp-ml/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlelarge%2Ficfp-ml/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlelarge%2Ficfp-ml/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mlelarge","download_url":"https://codeload.github.com/mlelarge/icfp-ml/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225669533,"owners_count":17505386,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-21T03:58:22.837Z","updated_at":"2025-07-11T02:31:29.598Z","avatar_url":"https://github.com/mlelarge.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Machine Learning - [Master ICFP](https://www.phys.ens.fr/en/formations/m2-icfp)\n\n**Prerequisites**:\n- Proficiency in Python: please use the tutorial [here](https://cs231n.github.io/python-numpy-tutorial/) for those who aren't as familiar with Python\n- Basic Calculus, Linear Algebra\n- Basic Probability and Statistics\n\n![](images/planning.png)\n\n## 1. Fundamentals of predictions and supervised learning (16/01)\n\n### Fundamentals of predictions\n- Minimizing errors\n- Modeling knowledge\n- Prediction via optimization\n- Types of errors and successes\n- Properties of ROC curves\n\n### Ref\n- [Fundamentals of prediction](https://mlstory.org/prediction.html) from Patterns, Predictions, and Actions (A story about machine learning) by Moritz Hardt and Benjamin Recht\n\n### practicals\n- Exact ROC curves for Gaussian mixtures: https://github.com/mlelarge/icfp-ml/blob/main/Exact_ROC_GM.ipynb\n\n### supervised learning\n- Sample versus Population\n- A first learning algorithm: the perceptron\n- Connection to empirical risk minimization\n- Formal guarantees for the perceptron\n\n### Ref: \n- [Supervised learning](https://mlstory.org/supervised.html)  from Patterns, Predictions, and Actions (A story about machine learning) by Moritz Hardt and Benjamin Recht\n\n### practicals\n- Naive Bayes and logistic regression: https://github.com/mlelarge/icfp-ml/blob/main/01_NaivesBayes_Logistic_empty.ipynb\n\n## 2. Pytorch basics and autodiff (23/01)\n\n[Module 2a - Pytorch tensors](https://dataflowr.github.io/website/modules/2a-pytorch-tensors/)\n\n[Module 2b - Automatic differentiation](https://dataflowr.github.io/website/modules/2b-automatic-differentiation/)\n\n## 3. Optimization for machine learning (30/01)\n\n- gradient descent\n- SGD\n- over-parameterized models:https://hackmd.io/@mlelarge/S1y5bEAhj\n\n### Ref: \n- In Chapter 5 (Sections 5.2.1 and 5.4) of [Learning Theory from First Principles](https://www.di.ens.fr/~fbach/ltfp_book.pdf) by Francis Bach\n\n\n### practicals\n- [Module 5 - Stacking layers](https://dataflowr.github.io/website/modules/5-stacking-layers/)\n\n- Heavy Ball Method: https://github.com/mlelarge/icfp-ml/blob/main/HeavyBall_empty.ipynb\n\n## 4. Kernels (06/02)\n\n- Local averaging methods\n    - partitions estimators\n    - k-nearest neighbors\n    - kernel smoothing\n- Positive-definite kernel methods\n    - representer theorem\n    - kernel trick\n\n### Ref: \n- Chapters 6 and 7 of [Learning Theory from First Principles](https://www.di.ens.fr/~fbach/ltfp_book.pdf) by Francis Bach\n\n### practicals\n- Kernel with [random Fourier features](https://github.com/mlelarge/icfp-ml/blob/main/03_kernel_random_fourier_empty.ipynb)\n\n## 5. Unsupervised Learning (13/02)\n\n- K-means clustering\n- Mixtures of Gaussian\n- Expectation-Maximization for GMM\n\n### Ref:\n- [Expectation-Maximization for the Gaussian Mixture Model](https://perso.telecom-paristech.fr/bonald/documents/gmm.pdf) by Thomas Bonald\n- [The Expectation Maximization Algorithm - A short tutorial](https://www.seanborman.com/publications/EM_algorithm.pdf) by Sean Borman\n\n### practicals\n- SVD\n- [Eigenfaces](https://github.com/mlelarge/icfp-ml/blob/main/02_SVD_Eigenfaces_empty.ipynb)\n\n## 6. Bayesian and Variational Inference (20/02 06-13/03)\n\n- Gaussian\n- Linear regression\n- Logistic regression\n- Laplace method\n- Variational inference\n\n### Ref:\n- Chapters 2.3 - 3.3 - 4.4 - 4.5 - 10 [Pattern Recognition and Machine Learning](https://www.microsoft.com/en-us/research/wp-content/uploads/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf) by Christopher Bishop\n \n## 7. Deep generative models: flows and diffusions (20/03)\n\n- [Normalizing flows](https://dataflowr.github.io/website/modules/9c-flows/)\n- [Denoising Diffusion Probabilistic Models](https://dataflowr.github.io/website/modules/18a-diffusion/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmlelarge%2Ficfp-ml","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmlelarge%2Ficfp-ml","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmlelarge%2Ficfp-ml/lists"}