{"id":15038271,"url":"https://github.com/epfml/optml_course","last_synced_at":"2025-05-14T13:06:48.260Z","repository":{"id":37405395,"uuid":"122380371","full_name":"epfml/OptML_course","owner":"epfml","description":"EPFL Course - Optimization for Machine Learning - CS-439","archived":false,"fork":false,"pushed_at":"2025-05-14T07:22:24.000Z","size":418745,"stargazers_count":1265,"open_issues_count":6,"forks_count":330,"subscribers_count":73,"default_branch":"master","last_synced_at":"2025-05-14T08:31:29.690Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/epfml.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-02-21T19:06:55.000Z","updated_at":"2025-05-14T07:22:28.000Z","dependencies_parsed_at":"2022-07-08T17:47:52.448Z","dependency_job_id":"fcf1aefe-01b1-4be8-a904-7b1a34ff678f","html_url":"https://github.com/epfml/OptML_course","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2FOptML_course","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2FOptML_course/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2FOptML_course/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2FOptML_course/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/epfml","download_url":"https://codeload.github.com/epfml/OptML_course/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254149955,"owners_count":22022851,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-24T20:37:45.949Z","updated_at":"2025-05-14T13:06:43.247Z","avatar_url":"https://github.com/epfml.png","language":"Jupyter Notebook","readme":"# EPFL Course - Optimization for Machine Learning - CS-439\n\n[Official coursebook information](http://edu.epfl.ch/coursebook/en/optimization-for-machine-learning-CS-439)\n\n`Lectures:` Fri 13:15-15:00 in [CO2](https://plan.epfl.ch/?room==CO%202)\n\n`Exercises:` Fri 15:15-17:00 in [BC01](https://plan.epfl.ch/?room==BC%2001)\n\nThis course teaches an overview of modern mathematical optimization methods, for applications in machine learning and data science. In particular, scalability of algorithms to large datasets will be discussed in theory and in implementation.\n\n### Team\n - Instructors: \n   - Nicolas Flammarion [nicolas.flammarion@epfl.ch](mailto:nicolas.flammarion@epfl.ch)\n - Assistants:\n   - Aditya Varre [aditya.varre@epfl.ch](mailto:aditya.varre@epfl.ch)\n   - Oguz Kaan Yüksel [oguz.yuksel@epfl.ch](mailto:oguz.yuksel@epfl.ch)\n   - Thomas Weinberger [thomas.weinberger@epfl.ch](mailto:thomas.weinberger@epfl.ch)\n   - Yitao Xu [yitao.xu@epfl.ch](mailto:yitao.xu@epfl.ch)\n\n \n\n   \n`Contents:`\n\nConvexity, Gradient Methods, Proximal algorithms, Subgradient Methods, Stochastic and Online Variants of mentioned methods, Coordinate Descent, Frank-Wolfe, Accelerated Methods, Primal-Dual context and certificates, Lagrange and Fenchel Duality, Second-Order Methods including Quasi-Newton Methods, Derivative-Free Optimization.\n\n*Advanced Contents:*\n\nParallel and Distributed Optimization Algorithms, Federated Learning\n\nComputational Trade-Offs (Time vs Data vs Accuracy), Lower Bounds\n\nNon-Convex Optimization: Convergence to Critical Points, Alternating minimization, Neural network training\n\n### Program:\n| Nr | Date  | Topic                                                 | Materials                                                                                                  | Exercises                             |\n| -- | ----- | ----------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | ------------------------------------- |\n| 1  | 21.2. | Introduction, Convexity                               |  [slides](../../raw/master/slides/lecture01.pdf)| [lab00](../../raw/master/labs/ex00/exercise00.pdf)|\n| 2  | 28.2.  | Gradient Descent                                      |  [slides](../../raw/master/slides/lecture02.pdf)| [lab01](../../raw/master/labs/ex01/exercise01.pdf) |\n| 3  | 7.3.  | Projected Gradient Descent                            |   [slides](../../raw/master/slides/lecture03.pdf) | [lab02](../../raw/master/labs/ex02/exercise02.pdf)|\n| 4  | 14.3. | Proximal and Subgradient Descent                      |  [slides](../../raw/master/slides/lecture04.pdf) |  [lab03](../../raw/master/labs/ex03/exercise03.pdf) |\n| 5  | 21.3. | Stochastic Gradient Descent, Non-Convex Optimization  |  [slides](../../raw/master/slides/lecture05.pdf)  | [lab04](../../raw/master/labs/ex04/exercise04.pdf) |\n| 6  | 28.3. | Non-Convex Optimization                               | [slides](../../raw/master/slides/lecture06.pdf)  | [lab05](../../raw/master/labs/ex05/exercise05.pdf) |\n| 7  | 4.4. | Newton's Method \u0026 Quasi-Newton                        | [slides](../../raw/master/slides/lecture07.pdf)  | [lab06](../../raw/master/labs/ex06/exercise06.pdf) |\n| 8  | 11.4. | Coordinate Descent                                    |  [slides](../../raw/master/slides/lecture07.pdf)  | lab07 |\n| .  | 18.4. | `easter vacation`                                     |                                                                                                            | -                                     |\n| .  | 25.4.  | `easter vacation`                                     |                                                                                                            | -                                     |\n| 9  |  2.5. | Frank-Wolfe                                           |  slides | [lab08](../../raw/master/labs/ex08/exercise08.pdf) |\n| 10 | 9.5. | Accelerated Gradient, Gradient-free, adaptive methods |  slides    | lab09|\n| 11 | 16.5.  | Opt for ML in Practice                                |  slides                                                    | lab10 | \n| 12 | 23.5.  | Opt for ML in Practice                                |  slides                                                     | Q\u0026A Projects                          |\n| 13 | 30.5. | `Mini-Project week`                                   |                                                                                                            | -                                     |\n\n### Lecture Notes:\nThe course is based on the following [lecture notes](../../raw/master/lecture_notes/lecture-notes.pdf).\n\n### Videos:\nThe [videos](https://mediaspace.epfl.ch/channel/CS-439+Optimization+for+machine+learning/31980) of the lectures for each week will be available.\n\n### Exercises:\nThe [weekly exercises](../../tree/master/labs/) consist of a mix of theoretical and practical `Python` exercises for the corresponding topic each week (starting week 2). Solutions to exercises are available in the lab folder.\n\n### Forum:\n[Discussion forum](https://edstem.org/eu/courses/2015/discussion/) (EPFL internal)\n\n### Project:\nA `mini-project` will focus on the practical implementation: Here we encourage students to investigate the real-world performance of one of the studied optimization algorithms or variants, helping to provide solid empirical evidence for some behaviour aspects on a real machine-learning task. The project is mandatory and done in groups of 3 students. It will count 30% to the final grade. Project reports (3 page PDF) are due June 14th. Here is a [detailed project description](../../raw/master/labs/mini-project/miniproject_description.pdf).\n\n### Assessment:\nSession Exam. Format: Closed book. Theoretical questions similar to exercises. You are allowed to bring one cheat sheet (A4 size paper, both sides can be used).\n\nFor practice: \n- exams [2023](../../raw/master/exams/exam2023.pdf), [2022](../../raw/master/exams/exam2022.pdf), [2021](../../raw/master/exams/exam2021.pdf), [2020](../../raw/master/exams/exam2020.pdf), [2019](../../raw/master/exams/exam2019.pdf), [2018](../../raw/master/exams/exam2018.pdf)\n- solutions [2023](../../raw/master/exams/exam2023solutions.pdf), [2022](../../raw/master/exams/exam2022solutions.pdf), [2021](../../raw/master/exams/exam2021solutions.pdf), [2020](../../raw/master/exams/exam2020solutions.pdf), [2019](../../raw/master/exams/exam2019solutions.pdf), [2018](../../raw/master/exams/exam2018solutions.pdf).\n\n### Links to related courses and materials \n - [CMU 10-725](https://www.stat.cmu.edu/~ryantibs/convexopt-F18/)\n - [Berkeley EE-227C](https://ee227c.github.io/)\n \n### Recommended Books\n - [Convex Optimization: Algorithms and Complexity](https://arxiv.org/pdf/1405.4980.pdf), by Sébastien Bubeck (free online)\n - [Convex Optimization](http://stanford.edu/~boyd/cvxbook/), Stephen Boyd and Lieven Vandenberghe (free online)\n - [Introductory Lectures on Convex Optimization](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.693.855\u0026rep=rep1\u0026type=pdf), Yurii Nesterov (free online)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepfml%2Foptml_course","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fepfml%2Foptml_course","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepfml%2Foptml_course/lists"}