Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/nikolaypavlov/MLPNeuralNet

Fast multilayer perceptron neural network library for iOS and Mac OS X
https://github.com/nikolaypavlov/MLPNeuralNet

Last synced: 19 days ago
JSON representation

Fast multilayer perceptron neural network library for iOS and Mac OS X

Lists

README 

        
#MLPNeuralNet

[![Build Status](https://travis-ci.org/nikolaypavlov/MLPNeuralNet.svg?branch=master)](https://travis-ci.org/nikolaypavlov/MLPNeuralNet)

[![Join the chat at https://gitter.im/nikolaypavlov/MLPNeuralNet](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/nikolaypavlov/MLPNeuralNet?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)



`MLPNeuralNet` is a fast [multilayer perceptron](http://en.wikipedia.org/wiki/Multilayer_perceptron) neural network library for iOS and Mac OS X. `MLPNeuralNet` predicts new examples through trained neural networks. It is built on top of Apple's [Accelerate Framework](https://developer.apple.com/library/ios/documentation/Accelerate/Reference/AccelerateFWRef/_index.html) using vectored operations and hardware acceleration (if available).



![Neural Network](http://nikolaypavlov.github.io/MLPNeuralNet/images/500px-Artificial_neural_network.png)



##Why would you use it?

Imagine that you have engineered a prediction model using Matlab (Python or R) and would like to use it in an iOS application. If that's the case, `MLPNeuralNet` is exactly what you need. `MLPNeuralNet` is designed to load and run models in [forward propagation](http://en.wikipedia.org/wiki/Backpropagation#Phase_1:_Propagation) mode only.



###Features

- [Classification](http://en.wikipedia.org/wiki/Binary_classification), [Multi-class classification](http://en.wikipedia.org/wiki/Multiclass_classification) and regression output

- Vectorised implementation

- Works with double precision

- Multiple hidden layers or none (in that case it's same as logistic/linear regression)



##Quick Example

Let's deploy a model for the AND function  ([conjunction](http://en.wikipedia.org/wiki/Logical_conjunction)) that works as follows: (of course, you do not need to use a neural network for this in the real world)



|X1 |X2 | Y |

|:-:|:-:|:-:|

| 0 | 0 | 0 |

| 1 | 0 | 0 |

| 0 | 1 | 0 |

| 1 | 1 | 1 |



Our model has the following weights and network configuration:



![AND Model Example](http://nikolaypavlov.github.io/MLPNeuralNet/images/network-arch.png)



```objective

// Use the designated initialiser to pass the network configuration and weights to the model.

// Note: You do not need to specify the biased units (+1 above) in the configuration.



NSArray *netConfig = @[@2, @1];

double wts[] = {-30, 20, 20};

NSData *weights = [NSData dataWithBytes:wts length:sizeof(wts)];



MLPNeuralNet *model = [[MLPNeuralNet alloc] initWithLayerConfig:netConfig

                                                        weights:weights

                                                     outputMode:MLPClassification];

// Predict output of the model for new sample

double sample[] = {0, 1};

NSData * vector = [NSData dataWithBytes:sample length:sizeof(sample)];

NSMutableData * prediction = [NSMutableData dataWithLength:sizeof(double)];

[model predictByFeatureVector:vector intoPredictionVector:prediction];



double * assessment = (double *)prediction.bytes;

NSLog(@"Model assessment is %f", assessment[0]);

```



##Extended Example

Let's say you trained a net using pybrain or even your own home brewed implementation.



![Extended Example](http://i.imgur.com/v2kMTUH.png)



```objective

// Use the designated initialiser to pass the network configuration and weights to the model.

// Note: You do not need to specify the biased units (+1 above) in the configuration.



NSArray *netConfig = @[@3, @2, @1];

double wts[] = {b1, w1, w2, w3, b2, w4, w5, w6, b3, w7, w8};

NSData *weights = [NSData dataWithBytes:wts length:sizeof(wts)];



MLPNeuralNet *model = [[MLPNeuralNet alloc] initWithLayerConfig:netConfig

                                                        weights:weights

                                                     outputMode:MLPClassification];

model.hiddenActivationFunction = MLPSigmoid;

model.outputActivationFunction = MLPNone;



// Predict output of the model for new sample

double sample[] = {0, 1, 2};

NSData * vector = [NSData dataWithBytes:sample length:sizeof(sample)];

NSMutableData * prediction = [NSMutableData dataWithLength:sizeof(double)];

[model predictByFeatureVector:vector intoPredictionVector:prediction];



double * assessment = (double *)prediction.bytes;

NSLog(@"Model assessment is %f", assessment[0]);

```



##Getting Started

The following instructions describe how to setup and install `MLPNeuralNet` using [CocoaPods](http://cocoapods.org/). It is written for Xcode 5 and the iOS 7.x(+) SDK. If you are not familiar with CocoaPods, just clone the repository and import `MLPNeuralNet` directly as a subproject.



##Installing through CocoaPods

Please add the following line to your *Podfile*.



```

pod 'MLPNeuralNet', '~> 1.0.0'

```



##Installing through Carthage

Please add the following line to your *Cartfile*.



```

github "nikolaypavlov/MLPNeuralNet" "master"

```



##Import `MLPNeuralNet.h`

Do not forget to add the following line to the top of your model:

```objectivec

#import "MLPNeuralNet.h"

```



##How many weights do I need to initialise network X->Y->Z?

Most of the popular libraries (including `MLPNeuralNet`) implicitly add biased units for each of the layers except the last one. Assuming these additional units, the total number of weights are `(X + 1) * Y + (Y + 1) * Z`.



##Importing weights from other libs.

You can do this for *some* of the neural network packages available.



###R nnet library:

 ```r

#Assuming nnet_model is a trained neural network

nnet_model$wts

```



###Python NeuroLab



```python

#Where net argument is an neurolab.core.Net object

import neurolab as nl

import numpy as np



def getweights(net):

     vec = []

     for layer in net.layers:

         b = layer.np['b']

         w = layer.np['w']

         newvec = np.ravel(np.concatenate((b, np.ravel(w,order='F'))).reshape((layer.ci+1, layer.cn)), order = 'F')

         [vec.append(nv) for nv in newvec]

     return np.array(vec)



```



###Python neon

```python

import numpy as np



def layer_names(params):

    layer_names = params.keys()

    layer_names.remove('epochs_complete')

    # Sort layers by their appearance in the model architecture

    # Since neon appands the index to the layer name we will use it to sort

    layer_names.sort(key=lambda x: int(x.split("_")[-1]))

    return layer_names



def getweights(file_name):

    vec = []

    # Load a stored model file from disk (should have extension prm)

    params = pkl.load(open(file_name, 'r'))

    layers = layer_names(params)

    

    for layer in layers:

        # Make sure our model has biases activated, otherwise add zeros here

        b = params[layer]['biases']

        w = params[layer]['weights']



        newvec = np.ravel(np.hstack((b,w)))

        [vec.append(nv) for nv in newvec]

    return vec



# An example call

getweights(expanduser('~/data/workout-dl/workout-ep100.prm'))

```



###Python keras

```python

import numpy as np



def get_weights_from_keras_model(model):

    vec = np.array([])

    for i in xrange(0, len(model.get_weights()), 2):

        bias = model.get_weights()[i + 1]

        weights_matrix = model.get_weights()[i]



        newvec = np.ravel(np.concatenate((bias.reshape(-1, 1), weights_matrix.T), axis=1))

        vec = np.append(vec, newvec)

    return np.array(vec)



```



## Performance benchmarks

In this test, the neural network has grown layer by layer from a `1 -> 1` configuration to a `200 -> 200 -> 200 -> 1` configuration. At each step, the output is calculated and benchmarked using random input vectorisation and weights. Total number of weights grow from 2 to 80601 accordingly. I understand that the test is quite synthetic, but I hope it illustrates the performance. I will be happy if you can propose a better one! :)



![MLPNeuralNet Performance Benchmark](http://nikolaypavlov.github.io/MLPNeuralNet/images/mlp-bench-regression-ios.png)



##Unit Testing

`MLPNeuralNet` includes a diverse suite of unit tests in the `/MLPNeuralNetTests` subdirectory. You can execute them using the ``MLPNeuralNet`` scheme within Xcode.



##Acknowledgements

`MLPNeuralNet` was inspired by:



- [Andrew Ng's course on Machine Learning](https://www.coursera.org/course/ml).

- [Jeff Leek course on Data Analysis](https://www.coursera.org/course/dataanalysis).



Credits:



- Neural Network image was taken from [Wikipedia Commons](http://en.wikipedia.org/wiki/File:Artificial_neural_network.svg).



##Contact Me

Maintainer: [Mykola Pavlov](http://github.com/nikolaypavlov/) ([email protected])



**Please let me know on how you use `MLPNeuralNet` for some real world problems.**



##Licensing

`MLPNeuralNet` is released under the BSD license. See the LICENSE file for more information.