{"id":25450117,"url":"https://github.com/rajnandinithopte/machine-learning_supervised-semisupervised-unsupervised-activelearning-svm","last_synced_at":"2025-11-02T03:30:25.884Z","repository":{"id":275660770,"uuid":"926786811","full_name":"rajnandinithopte/Machine-Learning_Supervised-SemiSupervised-Unsupervised-ActiveLearning-SVM","owner":"rajnandinithopte","description":"This project applies Support Vector Machines (SVM) and K-Means Clustering for multi-class and multi-label classification. It explores kernel methods, clustering evaluation, and hyperparameter tuning to improve model performance.","archived":false,"fork":false,"pushed_at":"2025-02-03T21:33:31.000Z","size":441,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-03T22:26:20.830Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rajnandinithopte.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-03T21:18:18.000Z","updated_at":"2025-02-03T21:33:34.000Z","dependencies_parsed_at":"2025-02-03T22:27:02.286Z","dependency_job_id":"80279fb1-40d8-4bce-8c80-fdc7bb48d31c","html_url":"https://github.com/rajnandinithopte/Machine-Learning_Supervised-SemiSupervised-Unsupervised-ActiveLearning-SVM","commit_stats":null,"previous_names":["rajnandinithopte/machine-learning_svm-kmeans-classification"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rajnandinithopte%2FMachine-Learning_Supervised-SemiSupervised-Unsupervised-ActiveLearning-SVM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rajnandinithopte%2FMachine-Learning_Supervised-SemiSupervised-Unsupervised-ActiveLearning-SVM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rajnandinithopte%2FMachine-Learning_Supervised-SemiSupervised-Unsupervised-ActiveLearning-SVM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rajnandinithopte%2FMachine-Learning_Supervised-SemiSupervised-Unsupervised-ActiveLearning-SVM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rajnandinithopte","download_url":"https://codeload.github.com/rajnandinithopte/Machine-Learning_Supervised-SemiSupervised-Unsupervised-ActiveLearning-SVM/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239366376,"owners_count":19626685,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-17T21:22:13.358Z","updated_at":"2025-11-02T03:30:25.847Z","avatar_url":"https://github.com/rajnandinithopte.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🔷 Machine-Learning: Supervised-SemiSupervised-Unsupervised-ActiveLearning-SVM\n\n## 🔷 Supervised, Semi-Supervised, and Unsupervised Learning | Active Learning Using Support Vector Machines\n\n### 🔶 Overview\nThis project explores different learning paradigms: **Supervised Learning, Semi-Supervised Learning, and Unsupervised Learning**. It focuses on **Active Learning with Support Vector Machines (SVMs)** to efficiently label data for model training. Various clustering methods and decision boundary analyses are performed to evaluate learning strategies.\n\n---\n\n## 🔷 Datasets Used\n- **Breast Cancer Wisconsin (Diagnostic) Data Set**  \n  - Binary classification problem (Benign vs. Malignant).\n  - Used for supervised, semi-supervised, and unsupervised learning.\n  - **[Dataset Link](https://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/wdbc.data)**\n\n- **Banknote Authentication Data Set**  \n  - Binary classification problem for distinguishing genuine vs. forged banknotes.\n  - Used for **Active Learning experiments**.\n  - **[Dataset Link](https://archive.ics.uci.edu/ml/datasets/banknote+authentication)**\n\n---\n\n## 🔷 Libraries Used\n- `numpy` - Numerical computations.\n- `pandas` - Data manipulation.\n- `matplotlib` \u0026 `seaborn` - Data visualization.\n- `scikit-learn` - Machine learning models (SVM, k-means, spectral clustering).\n- `scipy` - Mathematical functions for clustering.\n- `imbalanced-learn` - Handling class imbalance.\n- `statsmodels` - Statistical analysis.\n- `tqdm` - Progress tracking for Monte Carlo simulations.\n\n---\n\n## 🔷 Steps Taken to Accomplish the Project\n\n### 🔶 1. Supervised Learning with L1-Penalized SVM\n- Trained an **L1-penalized SVM** on the **Breast Cancer dataset** for binary classification.\n- Used **5-fold cross-validation** to **tune the penalty parameter (C)**.\n- **Normalized** the dataset before training.\n- **Monte Carlo Simulation:** Repeated the training process **30 times**, averaging performance metrics.\n- Reported:\n  - Accuracy\n  - Precision\n  - Recall\n  - F1-score\n  - AUC (Area Under the Curve)\n  - Confusion matrices\n  - **ROC Curve** for visualization.\n\n---\n\n### 🔶 2. Semi-Supervised Learning (Self-Training)\n- Used **50% labeled data** from both classes, treating the rest as **unlabeled**.\n- Trained an **L1-penalized SVM** on the labeled subset.\n- Selected the **unlabeled point farthest from the decision boundary**, let the model label it, and **added it back** to training data.\n- Repeated until all unlabeled points were classified.\n- Evaluated final SVM on test data and computed the **same performance metrics as in supervised learning**.\n\n---\n\n### 🔶 3. Unsupervised Learning with K-Means Clustering\n- Applied **k-means clustering (k=2)** on the **entire training set** (ignoring labels).\n- Ran **multiple iterations** of k-means to avoid local minima.\n- Determined **cluster labels** by polling the **30 closest points** to each cluster center.\n- Compared k-means predicted labels against true labels.\n- Evaluated:\n  - Accuracy\n  - Precision\n  - Recall\n  - F1-score\n  - AUC\n  - **ROC Curve**\n\n---\n\n### 🔶 4. Spectral Clustering\n- Implemented **Spectral Clustering using RBF kernel**.\n- Found an **optimal gamma value** to maintain balance in clusters.\n- Assigned labels using **fit-predict method** instead of cluster proximity.\n- Compared results with k-means clustering and SVM.\n\n---\n\n### 🔶 5. Active Learning Using SVMs\n- **Dataset:** Banknote Authentication Data.\n- Compared **Active Learning vs. Passive Learning** using **90 different SVMs**.\n- **Passive Learning Approach:**\n  - Started with **10 random data points**.\n  - Incrementally added **10 more randomly selected points** at each step.\n  - Trained SVM iteratively until **all 900 training points** were used.\n\n- **Active Learning Approach:**\n  - Started with **10 random data points**.\n  - Chose the **10 closest points to the SVM decision boundary** and added them to the training set.\n  - Repeated until all training data was used.\n- **Final Comparison:**\n  - Computed test errors across **50 trials** for both passive and active learning.\n  - Plotted **test error vs. number of training samples** for both approaches.\n\n---\n## 📌 **Note**\nThis repository contains **Jupyter Notebooks** detailing each step, along with **results and visualizations**.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frajnandinithopte%2Fmachine-learning_supervised-semisupervised-unsupervised-activelearning-svm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frajnandinithopte%2Fmachine-learning_supervised-semisupervised-unsupervised-activelearning-svm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frajnandinithopte%2Fmachine-learning_supervised-semisupervised-unsupervised-activelearning-svm/lists"}