Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/xmartlabs/Bender

Easily craft fast Neural Networks on iOS! Use TensorFlow models. Metal under the hood.
https://github.com/xmartlabs/Bender

apple convolutional-neural-networks deep-learning deep-neural-networks ios iphone machine-learning metal neural-networks residual-networks swift

Last synced: 6 days ago
JSON representation

Easily craft fast Neural Networks on iOS! Use TensorFlow models. Metal under the hood.

Awesome Lists containing this project

README 

        
# Bender





Build status

Platform iOS

Swift 4 compatible

CocoaPods compatible

Carthage compatible

License: MIT






![Bender](Documentation/Media/Artboard.png)



Bender is an abstraction layer over MetalPerformanceShaders useful for working with neural networks.



## Contents

* [Introduction](#introduction)

* [Why did we need Bender](#why)

* [Basic usage](#usage)

* [Requirements](#requirements)

* [Getting involved](#getting-involved)

* [Examples](#examples)

* [Installation](#installation)

* [Changelog](#change-log)



The documentation can be found under the `Documentation` folder:

* [API](Documentation/API.md) contains the most important information to get started.

* [Supported Layers] explains which layers are supported and how they map to TensorFlow ops.

* [Importing] explains how to import models from other frameworks such as TensorFlow. You can also find information on how to enhance this functionality for custom implementations.



## Introduction



Bender is an abstraction layer over MetalPerformanceShaders which is used to work with neural networks. It is of growing interest in the AI environment to execute neural networks on mobile devices even if the training process has been done previously. We want to make it easier for everyone to execute pretrained networks on iOS.



Bender allows you to easily define and run neural networks using the most common layers like Convolution, Pooling, FullyConnected and some normalizations among others. It is also flexible in the way it receives the parameters for these layers.



We also want to support loading models trained on other frameworks such as TensorFlow or Caffe2. Currently Bender includes an adapter for TensorFlow that loads a graph with variables and "translates" it to Bender layers. This feature supports a subset of TensorFlow's operations but we plan to enhance it to cover more cases.



## Why did we need Bender? 



At [Xmartlabs] we were about to start a Machine Learning project and investigated frameworks to use in iOS. We found MetalPerformanceShaders useful but not very user friendly and we saw ourselves repeating a lot of code and information. That is why we starting building a framework to handle that kind of stuff.



We also found ourselves creating scripts to translate the models we had from training with TensorFlow to iOS. This means transposing the weights to the MPSCNN format and also mapping the parameters of the different kinds of layers in TensorFlow to the parameters used by the MPSCNN kernels. TensorFlow can be compiled for iOS but currently it does not support running on GPU which we wanted to do. We also did not want to include TensorFlow's static library into our project. This is why we also started to work on an adapter that would parse a TF graph and translate it to our Bender layers.



## Usage



You can define your own network in Bender using our custom operator or you can load a model exported from TensorFlow. Defining a network and loading a model can be done like this:



```swift

import MetalBender



let url = Bundle.main.url(forResource: "myModel", withExtension: "pb")! // A TensorFlow model.

let network = Network.load(url: url, inputSize: LayerSize(h: 256, w: 256, f: 3))



network.run(input: /* ... */) { output in

    // ...

}

```



You can read more information about this in [Importing](Documentation/Importing.md).



If you want to define your network yourself you can do it like this:



```swift

let network = Network(inputSize: LayerSize(h: 256, w: 256, f: 3))



network.start

    ->> Convolution(convSize: ConvSize(outputChannels: 16, kernelSize: 3, stride: 2))

    ->> InstanceNorm()

    ->> Convolution(convSize: ConvSize(outputChannels: 32, kernelSize: 3, stride: 2), neuronType: .relu)

    ->> InstanceNorm()

    ->> FullyConnected(neurons: 128)

    ->> Neuron(type: .tanh)

    ->> FullyConnected(neurons: 10)

    ->> Softmax()

// ...

```



and once you're done with all your layers:



```swift

network.initialize()

```



To know more about this have a look at [API](Documentation/API.md).



## Requirements



* Xcode 9

* iOS 11.0+ (but deployment target is iOS 10.0, so iOS 10 is supported)



## Getting involved



* If you **want to contribute** please feel free to **submit pull requests**.

* If you **have a feature request** please **open an issue**.

* If you **found a bug** or **need help** please **check older issues, [FAQ](#faq) and threads on [StackOverflow](https://stackoverflow.com) before submitting an issue**.



Before contribute check the [CONTRIBUTING] file for more info.



If you use **Bender** in your app We would love to hear about it! Drop us a line on [Twitter](https://twitter.com/xmartlabs).



## Examples



Follow these steps to run the examples:

* Clone Bender repository (or download it).

* Run `carthage update --platform iOS` in the downloaded folder.

* Open Bender workspace and run the *Example* project.



> There is an Image recognition example which includes a MobileNet model in Bender and one in CoreML. It is also set up to run an Inception model but you will have to download it separately as it is almost 100 MB in size.

You can download it from http://download.tensorflow.org/models/inception_v3_2016_08_28.tar.gz but then you have to freeze it and add it to the 'Example' Xcode project as 'inception_v3.pb'.



## Installation



#### CocoaPods



To install Bender, simply add the following line to your Podfile:



```ruby

pod 'MetalBender', '~> 0.5'

```



> Remember that Bender compiles for iOS 10. So you must add `platform :ios, '10.0'` to your Podfile



#### Carthage



[Carthage](https://github.com/Carthage/Carthage) is a simple, decentralized dependency manager for Cocoa.



To install Bender, add the following line to your Cartfile:



```ogdl

github "xmartlabs/Bender" ~> 0.5

```



Then run:



```bash

carthage update --platform iOS

```



Finally, drag the built `.framework` binaries for `MetalBender`, `MetalPerformanceShadersProxy` and `SwiftProtobuf` to your application's Xcode project.



## Author



* [Xmartlabs SRL](https://github.com/xmartlabs) ([@xmartlabs](https://twitter.com/xmartlabs))



# Change Log



This can be found in the [CHANGELOG.md](CHANGELOG.md) file.



[Xmartlabs]: http://xmartlabs.com

[Importing]: Documentation/Importing.md

[CONTRIBUTING]: .github/CONTRIBUTING.md

[API]: Documentation/API.md

[Supported Layers]: Documentation/Supported_Layers.md



## License

[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fxmartlabs%2FBender.svg?type=large)](https://app.fossa.io/projects/git%2Bgithub.com%2Fxmartlabs%2FBender?ref=badge_large)



## Citation

If you use this code in your research please cite us:



```bibtex

@misc{xmartlabs-2017-bender,

  author = {Mathias Claassen and Santiago Castro},

  title = {Bender: Easily craft fast Neural Networks on {iOS}!},

  year = {2017},

  publisher = {GitHub},

  journal = {GitHub repository},

  howpublished = {\url{https://xmartlabs.github.io/Bender/}}

}

```