Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/ilias-ant/smo-from-scratch
Implementing Sequential Minimal Optimization algorithm from John C. Platt's 1998 paper.
https://github.com/ilias-ant/smo-from-scratch
optimization smo svm-classifier
Last synced: about 1 month ago
JSON representation
Implementing Sequential Minimal Optimization algorithm from John C. Platt's 1998 paper.
- Host: GitHub
- URL: https://github.com/ilias-ant/smo-from-scratch
- Owner: ilias-ant
- License: mit
- Created: 2022-06-17T16:16:40.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2022-06-25T20:41:16.000Z (over 2 years ago)
- Last Synced: 2024-10-14T00:53:47.929Z (2 months ago)
- Topics: optimization, smo, svm-classifier
- Language: Python
- Homepage:
- Size: 82 KB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# smo-from-scratch
[](https://www.python.org/)
[](https://github.com/psf/black)
Implementing Sequential Minimal Optimization algorithm from [John C. Platt's 1998 paper](https://www.microsoft.com/en-us/research/publication/sequential-minimal-optimization-a-fast-algorithm-for-training-support-vector-machines/).
### Setup
For demonstration purposes, we will make use of the [Gisette](https://archive.ics.uci.edu/ml/datasets/Gisette) dataset.
To populate the ``data/`` folder with the necessary data, simply run*:
```shell
sh datasets.sh
```
You should end up with the following data files:
```shell
data/
gisette_scale # extracted from gisette_scale.bz2
```
To enable reproducibility, [Poetry](https://python-poetry.org/) has been used as a dependency manager.
```shell
python3 -m pip install poetry
```
and then:
```shell
python3 -m poetry install --no-dev
```
to install all required project dependencies in a virtual environment.
### Usage
Spawn a shell within the created virtual environment with:
```shell
python3 -m poetry shell
```
From within the shell, the following:
```shell
python cli.py --help
```
will guide you through the available options:
```shell
Usage: cli.py [OPTIONS] COMMAND [ARGS]...
Options:
--help Show this message and exit.
Commands:
fit Perform a simple training of the SMO-based classifier, given a C.
tune Perform a hyperparameter tuning of the SMO-based classifier.
```
So, for example:
```
python cli.py fit --C=0.01
```
will train an SMO-based classifier with `C = 0.01`.
### Citation
```bibtex
@techreport{platt1998sequential,
author = {Platt, John},
title = {Sequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines},
institution = {Microsoft},
year = {1998},
month = {April},
abstract = {This paper proposes a new algorithm for training support vector machines: Sequential Minimal Optimization, or SMO. Training a support vector machine requires the solution of a very large quadratic programming (QP) optimization problem. SMO breaks this large QP problem into a series of smallest possible QP problems. These small QP problems are solved analytically, which avoids using a time-consuming numerical QP optimization as an inner loop. The amount of memory required for SMO is linear in the training set size, which allows SMO to handle very large training sets. Because matrix computation is avoided, SMO scales somewhere between linear and quadratic in the training set size for various test problems, while the standard chunking SVM algorithm scales somewhere between linear and cubic in the training set size. SMO's computation time is dominated by SVM evaluation, hence SMO is fastest for linear SVMs and sparse data sets. On real-world sparse data sets, SMO can be more than 1000 times faster than the chunking algorithm.},
url = {https://www.microsoft.com/en-us/research/publication/sequential-minimal-optimization-a-fast-algorithm-for-training-support-vector-machines/},
number = {MSR-TR-98-14},
}
```
---
*command ``sh datasets.sh`` works only on Linux-based systems - make necessary alterations depending on your OS.