https://github.com/zhenye-na/npca
📃 Exploration of Nonlinear Component Analysis as a Kernel Eigenvalue Problem
https://github.com/zhenye-na/npca
kernel-pca pca
Last synced: about 1 year ago
JSON representation
📃 Exploration of Nonlinear Component Analysis as a Kernel Eigenvalue Problem
- Host: GitHub
- URL: https://github.com/zhenye-na/npca
- Owner: Zhenye-Na
- License: mit
- Created: 2017-12-05T01:31:39.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2019-03-13T04:48:25.000Z (about 7 years ago)
- Last Synced: 2025-01-11T15:47:38.602Z (over 1 year ago)
- Topics: kernel-pca, pca
- Language: Jupyter Notebook
- Homepage:
- Size: 9.98 MB
- Stars: 0
- Watchers: 1
- Forks: 4
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Nonlinear Component Analysis as a Kernel Eigenvalue Problem
Course project for [IE529: Stats of Big data & Clustering](https://ise.illinois.edu/courses/profile/IE529-120168), 2017 Fall, UIUC
## Our team member
| Name | Guthub Homepage |
|----------------|---------------------------------------------------|
| Jvn Karthik | N/A |
| Naman Shukla | [namanUIUC](https://github.com/namanUIUC) |
| Shubham Bansal | [bansalshubh91](https://github.com/bansalshubh91) |
| Zhenye Na | [Zhenye-Na](https://github.com/Zhenye-Na) |
| Ziyu Zhou | [Ziyu0](https://github.com/Ziyu0) |
## Our primary work
We implemnt the experiments presented in the paper Nonlinear Component Analysis as a Kernel Eigenvalue Problem by Bernhard Schölkopf, Alexander Smola, Klaus-Robert Müller. Also, we write our own example on Kernel PCA. In this regard, you can read our report and our presentation slides.
## Dependencies
In order to run the experiments, make sure you have all dependencies installed
- matplotlib (>= 2.0.0)
- scipy (>=0.19.0)
- numpy (>=1.12.1)
- sklearn (>=0.0)
You can install them by typing
`pip3 install 'whatever you need'`
The prgramming languages we use are Python and MATLAB. If you do not have access to MATLAB on your laptop. We advise you to install `Octave` instead. You can refer to [this](https://www.gnu.org/software/octave/download.html) webpage for installing.
## Experiments in paper
In the paper, there are two major experiments:
- Toy example: 4-degree Polynomial Kernel PCA
- Character Recognition (USPS Dataset)
### Toy example: 4-degree Polynomial Kernel PCA
We implemented this part with MATLAB. The code can be found [here](https://github.com/Zhenye-Na/npca/blob/master/our_kpca/kpca_code.m).
## Our own implementation
### SVM and KPCA on Iris Dataset
Principal Component Analysis (PCA) is a dimensionality reduction technique that is used to transform and a high-dimensional dataset into a smaller dimen- sional subspace to give a directed impression of the dataset prior to running a machine learning algorithm on the data. The Iris dataset is in a 4 th dimensions (features) of three different iris ower species.
Related codes can be found [here](https://github.com/Zhenye-Na/npca/blob/master/application/SVM_on_Iris.ipynb).
### SVC on USPS Dataset
The dataset contains numeric data obtained from the scanning of handwritten digits from envelopes by the U.S. Postal Service. The original scanned digits are binary and of different sizes and orientations; the images here have been deslanted and size normalized, resulting in 16 × 16 grayscale. We will first extract features via Kernel PCA and apply that to a SVM classifier to train and test on the splitted USPS dataset.
Related codes can be found [here](https://github.com/Zhenye-Na/npca/blob/master/application/SVC_on_USPS.ipynb).