{"id":15916522,"url":"https://github.com/qinhanmin2014/sklearn-svm-guide","last_synced_at":"2025-04-03T05:43:22.810Z","repository":{"id":123466178,"uuid":"196211071","full_name":"qinhanmin2014/sklearn-svm-guide","owner":"qinhanmin2014","description":"Rapidly obtain acceptable results using SVM (based on scikit-learn)","archived":false,"fork":false,"pushed_at":"2019-07-23T10:15:17.000Z","size":35352,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-08T19:42:27.529Z","etag":null,"topics":["machine-learning","scikit-learn","svm"],"latest_commit_sha":null,"homepage":null,"language":"Terra","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/qinhanmin2014.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-07-10T13:32:31.000Z","updated_at":"2019-07-23T10:15:19.000Z","dependencies_parsed_at":null,"dependency_job_id":"5ed348e1-3fc6-4ff8-a8d9-50722b33d5ea","html_url":"https://github.com/qinhanmin2014/sklearn-svm-guide","commit_stats":{"total_commits":9,"total_committers":1,"mean_commits":9.0,"dds":0.0,"last_synced_commit":"a412e310d11ba4e2480cb88ab41ba4b560792af6"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qinhanmin2014%2Fsklearn-svm-guide","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qinhanmin2014%2Fsklearn-svm-guide/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qinhanmin2014%2Fsklearn-svm-guide/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qinhanmin2014%2Fsklearn-svm-guide/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/qinhanmin2014","download_url":"https://codeload.github.com/qinhanmin2014/sklearn-svm-guide/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246944384,"owners_count":20858773,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["machine-learning","scikit-learn","svm"],"created_at":"2024-10-06T18:02:06.791Z","updated_at":"2025-04-03T05:43:22.792Z","avatar_url":"https://github.com/qinhanmin2014.png","language":"Terra","funding_links":[],"categories":[],"sub_categories":[],"readme":"# sklearn-svm-guide\nRapidly obtain acceptable results using SVM (based on scikit-learn)\n\n## common procedure\n\n- Conduct simple scaling on the data\n  * sklearn.preprocessing.MinMaxScaler/StandardScaler\n- Consider the RBF kernel\n  * sklearn.svm.SVC default\n- Use cross-validation to find the best parameter C and gamma   \n  * sklearn.model_selection.GridSearchCV\n\n## common example\n\n```python\nimport numpy as np\nfrom sklearn.datasets import load_digits\nfrom sklearn.model_selection import cross_val_score, GridSearchCV\nfrom sklearn.preprocessing import MinMaxScaler\nfrom sklearn.svm import SVC\nX_train, y_train = load_digits(return_X_y=True)\nsc = MinMaxScaler(feature_range=(-1, 1))\nXt_train = sc.fit_transform(X_train)\nparams = {\"C\": np.logspace(-5, 15, num=11, base=2),\n          \"gamma\": np.logspace(3, -15, num=10, base=2)}\nclf = GridSearchCV(SVC(), params, n_jobs=-1)\nscores = cross_val_score(clf, Xt_train, y_train)\nprint(np.mean(scores), \"+/-\", np.std(scores))\n```\n\n## experiment A: Examples of the Proposed Procedure\n\n- Datasets 1: Astroparticle (from the reference)\n  * [jupyter notebook](https://nbviewer.jupyter.org/github/qinhanmin2014/sklearn-svm-guide/blob/master/A1_Astroparticle_Physics.ipynb)\n  *  evaluate using test set accuracy\n  *  default in libsvm and old default in scikit-learn: 66.93% (66.93% in the reference)\n  *  new default in scikit=learn: 96.25%\n  *  scale with MinMaxScaler: 96.15% (96.15% in the reference)\n  *  **scale with MinMaxScaler \u0026 tune the parameters: 96.93% (96.87% in the reference)**\n  *  scale with StandardScaler: 96.80%\n  *  scale with StandardScaler \u0026 tune the parameters: 96.68%\n\n- Datasets 2: Bioinformatics (from the reference)\n  * [jupyter notebook](https://nbviewer.jupyter.org/github/qinhanmin2014/sklearn-svm-guide/blob/master/A2_Bioinformatics.ipynb)\n  *  evaluate using cross validation accuracy\n  *  default in libsvm and old default in scikit-learn: 56.53% (56.52% in the reference)\n  *  new default in scikit=learn: 81.87%\n  *  scale with MinMaxScaler: 78.27% (78.52% in the reference)\n  *  **scale with MinMaxScaler \u0026 tune the parameters: 84.71% (85.17% in the reference)**\n  *  scale with StandardScaler: 56.53%\n  *  scale with StandardScaler \u0026 tune the parameters: 84.15%\n\n- Datasets 3: Astroparticle (from the reference)\n  * [jupyter notebook](https://nbviewer.jupyter.org/github/qinhanmin2014/sklearn-svm-guide/blob/master/A3_Vehicle.ipynb)\n  *  evaluate using test set accuracy\n  *  default in libsvm and old default in scikit-learn: 2.44% (2.44% in the reference)\n  *  new default in scikit=learn: 36.59%\n  *  scale with MinMaxScaler: 12.20% (12.20% in the reference)\n  *  **scale with MinMaxScaler \u0026 tune the parameters: 80.49% (87.80% in the reference)**\n  *  scale with StandardScaler: 65.85%\n  *  scale with StandardScaler \u0026 tune the parameters: 78.05%\n\n- Datasets 4: Breast Cancer (from sklearn.datasets.load_breast_cancer)\n  * [jupyter notebook](https://nbviewer.jupyter.org/github/qinhanmin2014/sklearn-svm-guide/blob/master/AX_Breast_Cancer.ipynb)\n  *  evaluate using cross validation accuracy\n  *  default in libsvm and old default in scikit-learn: 62.74%\n  *  new default in scikit=learn: 91.24%\n  *  scale with MinMaxScaler: 96.13%\n  *  scale with MinMaxScaler \u0026 tune the parameters: 97.54%\n  *  **scale with StandardScaler: 97.54%**\n  *  scale with StandardScaler \u0026 tune the parameters: 96.66%\n\n- Datasets 5: Digits (from sklearn.datasets.load_digits)\n  * [jupyter notebook](https://nbviewer.jupyter.org/github/qinhanmin2014/sklearn-svm-guide/blob/master/AX_Digits.ipynb)\n  *  evaluate using cross validation accuracy\n  *  default in libsvm and old default in scikit-learn: 44.88%\n  *  new default in scikit=learn: 96.38%\n  *  scale with MinMaxScaler: 95.72%\n  *  **scale with MinMaxScaler \u0026 tune the parameters: 97.33%**\n  *  scale with StandardScaler: 94.88%\n  *  scale with StandardScaler \u0026 tune the parameters: 94.77%\n\n- Datasets 6: Wine (from sklearn.datasets.load_wine)\n  * [jupyter notebook](https://nbviewer.jupyter.org/github/qinhanmin2014/sklearn-svm-guide/blob/master/AX_Wine.ipynb)\n  *  evaluate using cross validation accuracy\n  *  default in libsvm and old default in scikit-learn: 42.77%\n  *  new default in scikit=learn: 66.39%\n  *  scale with MinMaxScaler: 96.68%\n  *  scale with MinMaxScaler \u0026 tune the parameters: 96.68%\n  *  **scale with StandardScaler: 98.33%**\n  *  scale with StandardScaler \u0026 tune the parameters: 97.76%\n\n## experiment B: Common Mistakes in Scaling Training and Testing Data\n  * [jupyter notebook](https://nbviewer.jupyter.org/github/qinhanmin2014/sklearn-svm-guide/blob/master/B_Common_Mistakes_in_Scaling.ipynb)\n  *  evaluate using test set accuracy\n  * wrong way: use different scaler for training and testing sets (MinMaxScaler): 69.23% (69.23% in the reference)\n  * wrong way: use different scaler for training and testing sets (StandardScaler): 78.21%\n  * right way: use same scaler for training and testing sets (MinMaxScaler): 87.50% (89.42% in the reference)\n  * **right way: use same scaler for training and testing sets (StandardScaler): 89.42%**\n\n## experiment C: When to Use Linear but not RBF Kernel\n\n- Number of instances \u003c\u003c number of features\n  * suggestion: use linear kernel\n  * [jupyter notebook](https://nbviewer.jupyter.org/github/qinhanmin2014/sklearn-svm-guide/blob/master/C1_Linear_not_RBF_Kernel.ipynb)\n  * RBF kernel cross validation accuracy 92.85% (97.22% in the reference)\n  * linear kernel cross validation accuracy 92.85% (98.61% in the reference)\n\n- Both numbers of instances and features are large\n  * suggestion: use linear kernel\n  * [jupyter notebook](https://nbviewer.jupyter.org/github/qinhanmin2014/sklearn-svm-guide/blob/master/C2_Linear_not_RBF_Kernel.ipynb)\n  * RBF kernel, cross validation accuracy 97.17% (96.81% in the reference), wall time 15min 17s\n  * linear kernel, corss validation accuracy 96.63% (97.01% in the reference), wall time \u003c1s\n\n- Number of instances \u003e\u003e number of features\n  * suggestion: if linear kernel, set dual=False (default dual=True)\n  * [jupyter notebook](https://nbviewer.jupyter.org/github/qinhanmin2014/sklearn-svm-guide/blob/master/C3_Linear_not_RBF_Kernel.ipynb)\n  * dual=False, cross validation accuracy 68.51% (75.67% in the reference), wall time 35s\n  * dual=True, corss validation accuracy 68.51% (75.67% in the reference), wall time 10min 31s\n\n## experiment D: LIBLINEAR (LinearSVC)\n\n- In classification, large values in data may cause the following problems:\n  (1) Features in larger numeric ranges may dominate those in smaller ranges;\n  (2) Optimization methods for training may take longer time.\n  The typical remedy is to scale data feature-wisely.\n  However, for document data, often a simple instance-wise normalization is enough.\n  Each instance becomes a unit vector\n- Solvers in LIBLINEAR is not very sensitive to C. Once C is larger than certain value, the obtained models have similar performances.\n\n## reference\n\n- A Practical Guide to Support Vector Classification, Chih-Wei Hsu et al.\n- LIBLINEAR: A Library for Large Linear Classification, Rong-En Fan et al.\n- LIBSVM: A Library for Support Vector Machines, Chih-Chung Chang et al.\n\n## LIBLINEAR\n\n- https://www.csie.ntu.edu.tw/~cjlin/liblinear/\n- https://github.com/cjlin1/liblinear\n\n## LIBSVM\n\n- https://www.csie.ntu.edu.tw/~cjlin/libsvm/\n- https://github.com/cjlin1/libsvm\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqinhanmin2014%2Fsklearn-svm-guide","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fqinhanmin2014%2Fsklearn-svm-guide","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqinhanmin2014%2Fsklearn-svm-guide/lists"}