https://github.com/nghorbani/machinelearning

Some of experiences in Machine Learning field
https://github.com/nghorbani/machinelearning

bayesian bayesian-methods bayesian-optimization classification gaussian-processes hyperparameters matlab optimization regression simple

Last synced: 4 months ago
JSON representation

Some of experiences in Machine Learning field

• Host: GitHub
• URL: https://github.com/nghorbani/machinelearning
• Owner: nghorbani
• License: gpl-2.0
• Created: 2017-01-09T08:15:06.000Z (about 9 years ago)
• Default Branch: master
• Last Pushed: 2017-07-18T14:04:56.000Z (over 8 years ago)
• Last Synced: 2025-06-29T12:40:47.599Z (8 months ago)
• Topics: bayesian, bayesian-methods, bayesian-optimization, classification, gaussian-processes, hyperparameters, matlab, optimization, regression, simple
• Language: Jupyter Notebook
• Homepage:
• Size: 12.2 MB
• Stars: 1
• Watchers: 0
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: readme.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Some Machine Learning Experiences

## Bayesian Optimization with Gaussian Processes

[**BO in Python**](BayesianOptimization/BayesianOptimization.ipynb)

You will see an example of optimizing a simple Neural Network's Hyperparameters with Bayesian Optimization

Using Expected Improvement acquisition function:

![alt tag](images/OB1DSample.gif)

## Gaussian Process Regression and Classification

[**GPR in Python**](GaussianProcesses/GRP.ipynb)

[or in matlab](GaussianProcesses/GPR.m)

[**GPC in Python**](GaussianProcesses/GPC.ipynb)

[or in matlab](GaussianProcesses/GPC.m)

### Regression with Gaussian Processes

We have noisy sensor readings (indicated by errorbars).

first we will do a point prediction:

![alt tag](images/GRP-single.png)

Next we will predict 100 points

![alt tag](images/GPR-multi.png)

And we finally use MAP estimate of the hyperparameters:

![alt tag](images/GPR-optim.png)

### Classification with Gaussian Processes

We have generated training points and labels and then tried to compute labels for test points.

Each + is one trainig point and * is a test point:

![alt tag](images/GPC.png)

If the color of '*' and 'o' coincide then that point is correctly classified. The points without any circle around them are training points ('+').

## Linear Regression (LR) and Classification (LC)

### LR in high-dimensional feature space

[matlab code](LinearRegression/main.m)

![alt tag](images/LR_with_basis.png)

Here have projected input to a hd feature space with the help of basis functions and followed the rest of the standard linear model for regression. We see effect of different choices for basis function hyperparameters.

### LC with Linear and Quadratic Decision Boundaries

[matlab code](LinearClassification/linearclassification.m)

![alt tag](images/LC_LDA.png)

![alt tag](images/LC_QDA.png)

## Sampling Techinques

### Sampling from a cauchy distribution

[matlab code](Sampling/cauchysampling.m)

![alt tag](images/cauchysampling.png)

## Principle Component Analysis

### Basics of PCA

[matlab code](DimenstionalityReduction/pca_basics.m)

![alt tag](images/pca_basics.png)

in the folder you can also find code for PCA with isotropic noise.