{"id":13835740,"url":"https://github.com/saarbk/Introduction-to-Machine-Learning","last_synced_at":"2025-07-10T12:32:43.516Z","repository":{"id":134660907,"uuid":"488326321","full_name":"saarbk/Introduction-to-Machine-Learning","owner":"saarbk","description":"Sharing both theoretical and programing ideas, that I came across at Introduction to Machine Learning course. notes, homework solution and python assignment","archived":false,"fork":false,"pushed_at":"2023-10-16T18:02:25.000Z","size":1642,"stargazers_count":4,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-04-12T16:33:43.445Z","etag":null,"topics":["homework-assignments","knn-algorithm","machine-learning","neural-network","sgd-classifier","student-project","svm"],"latest_commit_sha":null,"homepage":"https://saarbk.github.io/iml","language":"TeX","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/saarbk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-05-03T18:47:07.000Z","updated_at":"2023-02-17T11:26:47.000Z","dependencies_parsed_at":null,"dependency_job_id":"c024ceba-7964-45cb-870d-18f4acecb5be","html_url":"https://github.com/saarbk/Introduction-to-Machine-Learning","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/saarbk%2FIntroduction-to-Machine-Learning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/saarbk%2FIntroduction-to-Machine-Learning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/saarbk%2FIntroduction-to-Machine-Learning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/saarbk%2FIntroduction-to-Machine-Learning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/saarbk","download_url":"https://codeload.github.com/saarbk/Introduction-to-Machine-Learning/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":213924053,"owners_count":15657934,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["homework-assignments","knn-algorithm","machine-learning","neural-network","sgd-classifier","student-project","svm"],"created_at":"2024-08-04T15:00:18.343Z","updated_at":"2024-08-04T15:00:54.127Z","avatar_url":"https://github.com/saarbk.png","language":"TeX","funding_links":[],"categories":["Computer Science"],"sub_categories":[],"readme":"[![Display](https://img.shields.io/badge/Display-Website-blue.svg)](https://saarbk.github.io/iml)\n\n# Introduction-to-Machine-Learning\nRepository of notebooks and conceptual insights from my \"Introduction to Machine Learning\" course. Each section contains corresponding PDF solutions,  ![equation](https://latex.codecogs.com/svg.image?\\text{\\TeX}), and Python scripts.\n\n\n## ![equation](https://latex.codecogs.com/svg.image?%5Cinline%20%5CLARGE%20%5Ctextbf%7BSection%201%7D)\n(Warm-up)\n\n![equation](https://latex.codecogs.com/svg.image?\\textbf{Theory\u0026space;Part}\u0026space;)\n\\\n  [1.1](Section1.0/section_1.pdf) Linear Algebra\n  \\\n  [1.2](Section1.0/section_1.pdf) Calculus and Probability\n  \\\n  [1.3](Section1.0/section_1.pdf) Optimal Classifiers and Decision Rules\n  \\\n  [1.4](Section1.0/section_1.pdf)  Multivariate normal (or Gaussian) distribution\n  \n  \n![equation](https://latex.codecogs.com/svg.image?\\textbf{Programming\u0026space;Part}\u0026space;)\n\\\n[Visualizing the Hoeffding bound.](Section1.0/plot1.png)\n[k-NN algorithm.](Section1.0/KNN.py)\n\n## [![equation](https://latex.codecogs.com/svg.image?%5Cinline%20%5CLARGE%20%5Ctextbf%7BSection%202%7D)](https://github.com/saarbk/Introduction-to-Machine-Learning/blob/main/EX2/Section_2.pdf)\n![equation](https://latex.codecogs.com/svg.image?\\textbf{Theory\u0026space;Part}\u0026space;)\n\\\n[2.1](Section2.0/Section2.pdf) PAC learnability of ℓ2-balls around the origin\n\\\n[2.2](Section2.0/Section2.pdf) PAC in Expectation\n\\\n[2.3](Section2.0/Section2.pdf) Union Of Intervals \n\\\n[2.4](Section2.0/Section2.pdf) Prediction by polynomials\n\\\n[2.5](Section2.0/Section2.pdf) Structural Risk Minimization\n\n![equation](https://latex.codecogs.com/svg.image?\\textbf{Programming\u0026space;Part}\u0026space;)\n\n[Union Of Intervals.](EX2/union_of_intervals.py)\nStudy the hypothesis class of a finite\nunion of disjoint intervals, and the properties of the ERM algorithm for this class.\nTo review, let the sample space be ![equation](https://latex.codecogs.com/svg.image?X\u0026space;=\u0026space;[0,\u0026space;1]) and assume we study a binary classification problem,i.e. ![equation](https://latex.codecogs.com/svg.image?Y\u0026space;=\u0026space;0,\u0026space;1).\nWe will try to learn using an hypothesis class that consists of k disjoint intervals. \ndefine the corresponding hypothesis as  \n   \n   ![equation](https://latex.codecogs.com/svg.image?%5Cinline%20h_I(x)=%5Cbegin%7Bcases%7D1%20\u0026%5Ctext%7Bif%20%7D%20x%5Cin%20%5Bl_1,u_1%5D%5Ccup%20%5Cdots%20%5Ccup%20%5Bl_k,u_k%5D%20%5C%5C1%20\u0026%5Ctext%7Botherwise%7D%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%5Cend%7Bcases%7D)\n## ![equation](https://latex.codecogs.com/svg.image?%5Cinline%20%5CLARGE%20%5Ctextbf%7BSection%203%7D)\n![equation](https://latex.codecogs.com/svg.image?\\textbf{Theory\u0026space;Part}\u0026space;)\n\\\n[3.1](Section3.0/section3.pdf) Step-size Perceptron\n\\\n[3.2](Section3.0/section3.pdf) Convex functions\n\\\n[3.3](Section3.0/section3.pdf) GD with projection\n\\\n[3.4](Section3.0/section3.pdf) Gradient Descent on Smooth Functions\n\n![equation](https://latex.codecogs.com/svg.image?\\textbf{Programming\u0026space;Part}\u0026space;)\n\n[SGD for Hinge loss.](Section3.0/sgd.py)\nIn the file skeleton sgd.py there is an helper function. The function reads the examples labelled 0, 8 \nand returns them with the labels −1/+1. In case you are unable to\nread the MNIST data with the provided script, you can download the file from [ Here](https://github.com/amplab/datasciencesp14/blob/master/lab7/mldata/mnist-original.mat). \n\n![equation](https://latex.codecogs.com/svg.image?\\inline\u0026space;\\large\u0026space;\\bg{red}\\ell(y)_{hinge}=\\max\u0026space;(0,1-\\mathbf{x}_i\u0026space;y_i))\n\n\n[SGD for log-loss.](Section3.0/sgd.py)\nIn this exercise we will optimize the log loss defined\nas follows:\n\n![equation](https://latex.codecogs.com/svg.image?\\ell_{log}(\\mathbf{w},x,y)\u0026space;=\u0026space;\\log(1\u0026plus;e^{-y\\mathbf{w}\\cdot\u0026space;x}))\n## ![equation](https://latex.codecogs.com/svg.image?%5Cinline%20%5CLARGE%20%5Ctextbf%7BSection%204%7D)\n![equation](https://latex.codecogs.com/svg.image?\\textbf{Theory\u0026space;Part}\u0026space;)\n\\\n[4.1](Section4.0/section_4.pdf) SVM with multiple classes\n\\\n[4.2](Section4.0/section_4.pdf) Soft-SVM bound using hard-SVM\n\\\n[4.3](Section4.0/section_4.pdf) Separability using polynomial kernel\n\\\n[4.4](Section4.0/section_4.pdf) Expressivity of ReLU networks\n\\\n[4.5](Section4.0/section_4.pdf) Implementing boolean functions using ReLU networks. \n\n![equation](https://latex.codecogs.com/svg.image?\\textbf{Programming\u0026space;Part}\u0026space;)\n\n[SVM](Section4.0/svm.py)\nExploring different polynomial kernel degrees for\nSVM. We will use an existing implementation of SVM, the SVC class from `sklearn.svm.`\n\n\n[Neural Networks](Section4.0/svm.py)\nwe will implement the back-propagation\nalgorithm for training a neural network. We will work with the MNIST data set that consists\nof 60000 28x28 gray scale images with values of 0 to 1.\nDefine the log-loss on a single example\n\n![equation](https://latex.codecogs.com/svg.image?%5Cinline%20%5Cell_%7B(%5Cmathbf%7Bx,y%7D)%7D(W)=-%5Cmathbf%7By%7D%5Clog%5Cmathbf%7Bz%7D_L(%5Cmathbf%7Bx;%5Cmathcal%7BW%7D%7D))\n\nAnd the loss we want to minimize is\n\n![equation](https://latex.codecogs.com/svg.image?%5Cinline%20%5Cell(%5Cmathcal%7BW%7D)=%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bi=1%7D%5E%7Bn%7D%5Cell%20(%5Cmathbf%7Bx%7D_i,%5Cmathbf%7By%7D_i)(%5Cmathcal%7BW%7D)=%5Cfrac%7B1%7D%7Bn%7D%5Csum_%7Bi=1%7D%5E%7Bn%7D-%5Cmathbf%7By%7D_i%5Cast%20%5Clog%20%5Cmathbf%7Bz%7D_L(%5Cmathbf%7Bx%7D_i;%5Cmathcal%7BW%7D))\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsaarbk%2FIntroduction-to-Machine-Learning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsaarbk%2FIntroduction-to-Machine-Learning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsaarbk%2FIntroduction-to-Machine-Learning/lists"}