Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/crystal-data/num.cr

Scientific computing in pure Crystal
https://github.com/crystal-data/num.cr

autograd crystal gpu-accelerated-routines linear-algebra machine-learning tensors

Last synced: 8 days ago
JSON representation

Scientific computing in pure Crystal

Awesome Lists containing this project

README 

        
![num.cr](https://raw.githubusercontent.com/crystal-data/bottle/rename/static/numcr_logo.png)



[![Join the chat at https://gitter.im/crystal-data/bottle](https://badges.gitter.im/crystal-data/bottle.svg)](https://gitter.im/crystal-data/bottle?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)

![Crystal CI](https://github.com/crystal-data/num.cr/workflows/Crystal%20CI/badge.svg)



Num.cr is the core shard needed for scientific computing with Crystal



- **Website:** https://crystal-data.github.io/num.cr

- **API Documentation:** https://crystal-data.github.io/num.cr/

- **Source code:** https://github.com/crystal-data/num.cr

- **Bug reports:** https://github.com/crystal-data/num.cr/issues



It provides:



- An n-dimensional `Tensor` data structure

- Efficient `map`, `reduce` and `accumulate` routines

- GPU accelerated routines backed by `OpenCL`

- Linear algebra routines backed by `LAPACK` and `BLAS`



## Prerequisites



`Num.cr` aims to be a scientific computing library written in pure Crystal.

All standard operations and data structures are written in Crystal.  Certain

routines, primarily linear algebra routines, are instead provided by a

`BLAS` or `LAPACK` implementation.



Several implementations can be used, including `Cblas`, `Openblas`, and the

`Accelerate` framework on Darwin systems.  For GPU accelerated `BLAS` routines,

the `ClBlast` library is required.



`Num.cr` also supports `Tensor`s stored on a `GPU`.  This is currently limited

to `OpenCL`, and a valid `OpenCL` installation and device(s) are required.



## Installation



Add this to your applications `shard.yml`



```

dependencies:

  num:

    github: crystal-data/num.cr

```



Several third-party libraries are required to use certain features of `Num.cr`.

They are:



- BLAS

- LAPACK

- OpenCL

- ClBlast

- NNPACK



While not at all required, they provide additional functionality than is

provided by the basic library.



## Just show me the code



The core data structure implemented by `Num.cr` is the `Tensor`, an N-dimensional

data structure.  A `Tensor` supports slicing, mutation, permutation, reduction,

and accumulation.  A `Tensor` can be a view of another `Tensor`, and can support

either C-style or Fortran-style storage.



### Creation



There are many ways to initialize a `Tensor`.  Most creation methods can

allocate a `Tensor` backed by either `CPU` or `GPU` based storage.



```crystal

[1, 2, 3].to_tensor

Tensor.from_array [1, 2, 3]

Tensor(UInt8, CPU(UInt8)).zeros([3, 3, 2])

Tensor.random(0.0...1.0, [2, 2, 2])



Tensor(Float32, OCL(Float32)).zeros([3, 2, 2])

Tensor(Float64, OCL(Float64)).full([3, 4, 5], 3.8)

```



### Operations



A `Tensor` supports a wide variety of numerical operations.  Many of these

operations are provided by `Num.cr`, but any operation can be mapped across

one or more `Tensor`s using sophisticated broadcasted mapping routines.



```crystal

a = [1, 2, 3, 4].to_tensor

b = [[3, 4, 5, 6], [5, 6, 7, 8]].to_tensor



puts a + b



# a is broadcast to b's shape

# [[ 4,  6,  8, 10],

#  [ 6,  8, 10, 12]]

```



When operating on more than two `Tensor`s, it is recommended to use `map`

rather than builtin functions to avoid the allocation of intermediate

results.  All `map` operations support broadcasting.



```crystal

a = [1, 2, 3, 4].to_tensor

b = [[3, 4, 5, 6], [5, 6, 7, 8]].to_tensor

c = [3, 5, 7, 9].to_tensor



a.map(b, c) do |i, j, k|

  i + 2 / j + k * 3.5

end



# [[12.1667, 20     , 27.9   , 35.8333],

#  [11.9   , 19.8333, 27.7857, 35.75  ]]

```



### Mutation



`Tensor`s support flexible slicing and mutation operations.  Many of these

operations return views, not copies, so any changes made to the results might

also be reflected in the parent.



```crystal

a = Tensor.new([3, 2, 2]) { |i| i }



puts a.transpose



# [[[ 0,  4,  8],

#   [ 2,  6, 10]],

#

#  [[ 1,  5,  9],

#   [ 3,  7, 11]]]



puts a.reshape(6, 2)



# [[ 0,  1],

#  [ 2,  3],

#  [ 4,  5],

#  [ 6,  7],

#  [ 8,  9],

#  [10, 11]]



puts a[..., 1]



# [[ 2,  3],

#  [ 6,  7],

#  [10, 11]]



puts a[1..., {..., -1}]



# [[[ 6,  7],

#   [ 4,  5]],

#

#  [[10, 11],

#   [ 8,  9]]]



puts a[0, 1, 1].value



# 3

```



### Linear Algebra



`Tensor`s provide easy access to power Linear Algebra routines backed by

LAPACK and BLAS implementations, and ClBlast for GPU backed `Tensor`s.



```crystal

a = [[1, 2], [3, 4]].to_tensor.map &.to_f32



puts a.inv



# [[-2  , 1   ],

#  [1.5 , -0.5]]



puts a.eigvals



# [-0.372281, 5.37228  ]



puts a.matmul(a)



# [[7 , 10],

#  [15, 22]]

```



### Einstein Notation



For representing certain complex contractions of `Tensor`s, Einstein notation

can be used to simplify the operation.  For example, the following matrix

multiplication + summation operation:



```crystal

a = Tensor.new([30, 40, 50]) { |i| i * 1_f32 }

b = Tensor.new([40, 30, 20]) { |i| i * 1_f32 }



result = Float32Tensor.zeros([50, 20])

ny, nx = result.shape

b2 = b.swap_axes(0, 1)

ny.times do |k|

  nx.times do |l|

    result[k, l] = (a[..., ..., k] * b2[..., ..., l]).sum

  end

end

```



Can instead be represented in Einstein notiation as the following:



```crystal

Num::Einsum.einsum("ijk,jil->kl", a, b)

```



This can lead to performance improvements due to optimized contractions

on `Tensor`s.



```

einsum   2.22k   (450.41µs) (± 0.86%)   350kB/op        fastest

manual   117.52  (  8.51ms) (± 0.98%)  5.66MB/op  18.89× slower

```



### Machine Learning



`Num::Grad` provides a pure-crystal approach to find derivatives of

mathematical functions.  Use a `Num::Grad::Variable` with a `Num::Grad::Context`

to easily compute these derivatives.



```crystal

ctx = Num::Grad::Context(Tensor(Float64, CPU(Float64))).new



x = ctx.variable([3.0].to_tensor)

y = ctx.variable([2.0].to_tensor)



# f(x) = x ** y

f = x ** y

puts f # => [9]



f.backprop



# df/dx = y * x = 6.0

puts x.grad # => [6.0]

```



`Num::NN` contains an extension to `Num::Grad` that provides an easy-to-use

interface to assist in creating neural networks.  Designing and creating

a network is simple using Crystal's block syntax.



```crystal

ctx = Num::Grad::Context(Tensor(Float64, CPU(Float64))).new



x_train = [[0.0, 0.0], [1.0, 0.0], [0.0, 1.0], [1.0, 1.0]].to_tensor

y_train = [[0.0], [1.0], [1.0], [0.0]].to_tensor



x = ctx.variable(x_train)



net = Num::NN::Network.new(ctx) do

  input [2]

  # A basic network with a single hidden layer using

  # a ReLU activation function

  linear 3

  relu

  linear 1



  # SGD Optimizer

  sgd 0.7



  # Sigmoid Cross Entropy to calculate loss

  sigmoid_cross_entropy_loss

end



500.times do |epoch|

  y_pred = net.forward(x)

  loss = net.loss(y_pred, y_train)

  puts "Epoch: #{epoch} - Loss #{loss}"

  loss.backprop

  net.optimizer.update

end



# Clip results to make a prediction

puts net.forward(x).value.map { |el| el > 0 ? 1 : 0}



# [[0],

#  [1],

#  [1],

#  [0]]

```



Review the documentation for full implementation details, and if something is missing,

open an issue to add it!