Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/crystal-data/num.cr
Scientific computing in pure Crystal
https://github.com/crystal-data/num.cr
autograd crystal gpu-accelerated-routines linear-algebra machine-learning tensors
Last synced: 8 days ago
JSON representation
Scientific computing in pure Crystal
- Host: GitHub
- URL: https://github.com/crystal-data/num.cr
- Owner: crystal-data
- License: mit
- Created: 2019-10-10T18:15:21.000Z (about 5 years ago)
- Default Branch: master
- Last Pushed: 2024-06-20T18:34:53.000Z (5 months ago)
- Last Synced: 2024-10-25T01:22:48.489Z (15 days ago)
- Topics: autograd, crystal, gpu-accelerated-routines, linear-algebra, machine-learning, tensors
- Language: Crystal
- Homepage:
- Size: 5.27 MB
- Stars: 151
- Watchers: 10
- Forks: 12
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
- awesome-crystal - num.cr - Numerical computing library supporting N-Dimensional data (Science and Data analysis)
- awesome-crystal - num.cr - Numerical computing library supporting N-Dimensional data (Science and Data analysis)
README
![num.cr](https://raw.githubusercontent.com/crystal-data/bottle/rename/static/numcr_logo.png)
[![Join the chat at https://gitter.im/crystal-data/bottle](https://badges.gitter.im/crystal-data/bottle.svg)](https://gitter.im/crystal-data/bottle?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
![Crystal CI](https://github.com/crystal-data/num.cr/workflows/Crystal%20CI/badge.svg)Num.cr is the core shard needed for scientific computing with Crystal
- **Website:** https://crystal-data.github.io/num.cr
- **API Documentation:** https://crystal-data.github.io/num.cr/
- **Source code:** https://github.com/crystal-data/num.cr
- **Bug reports:** https://github.com/crystal-data/num.cr/issuesIt provides:
- An n-dimensional `Tensor` data structure
- Efficient `map`, `reduce` and `accumulate` routines
- GPU accelerated routines backed by `OpenCL`
- Linear algebra routines backed by `LAPACK` and `BLAS`## Prerequisites
`Num.cr` aims to be a scientific computing library written in pure Crystal.
All standard operations and data structures are written in Crystal. Certain
routines, primarily linear algebra routines, are instead provided by a
`BLAS` or `LAPACK` implementation.Several implementations can be used, including `Cblas`, `Openblas`, and the
`Accelerate` framework on Darwin systems. For GPU accelerated `BLAS` routines,
the `ClBlast` library is required.`Num.cr` also supports `Tensor`s stored on a `GPU`. This is currently limited
to `OpenCL`, and a valid `OpenCL` installation and device(s) are required.## Installation
Add this to your applications `shard.yml`
```
dependencies:
num:
github: crystal-data/num.cr
```Several third-party libraries are required to use certain features of `Num.cr`.
They are:- BLAS
- LAPACK
- OpenCL
- ClBlast
- NNPACKWhile not at all required, they provide additional functionality than is
provided by the basic library.## Just show me the code
The core data structure implemented by `Num.cr` is the `Tensor`, an N-dimensional
data structure. A `Tensor` supports slicing, mutation, permutation, reduction,
and accumulation. A `Tensor` can be a view of another `Tensor`, and can support
either C-style or Fortran-style storage.### Creation
There are many ways to initialize a `Tensor`. Most creation methods can
allocate a `Tensor` backed by either `CPU` or `GPU` based storage.```crystal
[1, 2, 3].to_tensor
Tensor.from_array [1, 2, 3]
Tensor(UInt8, CPU(UInt8)).zeros([3, 3, 2])
Tensor.random(0.0...1.0, [2, 2, 2])Tensor(Float32, OCL(Float32)).zeros([3, 2, 2])
Tensor(Float64, OCL(Float64)).full([3, 4, 5], 3.8)
```### Operations
A `Tensor` supports a wide variety of numerical operations. Many of these
operations are provided by `Num.cr`, but any operation can be mapped across
one or more `Tensor`s using sophisticated broadcasted mapping routines.```crystal
a = [1, 2, 3, 4].to_tensor
b = [[3, 4, 5, 6], [5, 6, 7, 8]].to_tensorputs a + b
# a is broadcast to b's shape
# [[ 4, 6, 8, 10],
# [ 6, 8, 10, 12]]
```When operating on more than two `Tensor`s, it is recommended to use `map`
rather than builtin functions to avoid the allocation of intermediate
results. All `map` operations support broadcasting.```crystal
a = [1, 2, 3, 4].to_tensor
b = [[3, 4, 5, 6], [5, 6, 7, 8]].to_tensor
c = [3, 5, 7, 9].to_tensora.map(b, c) do |i, j, k|
i + 2 / j + k * 3.5
end# [[12.1667, 20 , 27.9 , 35.8333],
# [11.9 , 19.8333, 27.7857, 35.75 ]]
```### Mutation
`Tensor`s support flexible slicing and mutation operations. Many of these
operations return views, not copies, so any changes made to the results might
also be reflected in the parent.```crystal
a = Tensor.new([3, 2, 2]) { |i| i }puts a.transpose
# [[[ 0, 4, 8],
# [ 2, 6, 10]],
#
# [[ 1, 5, 9],
# [ 3, 7, 11]]]puts a.reshape(6, 2)
# [[ 0, 1],
# [ 2, 3],
# [ 4, 5],
# [ 6, 7],
# [ 8, 9],
# [10, 11]]puts a[..., 1]
# [[ 2, 3],
# [ 6, 7],
# [10, 11]]puts a[1..., {..., -1}]
# [[[ 6, 7],
# [ 4, 5]],
#
# [[10, 11],
# [ 8, 9]]]puts a[0, 1, 1].value
# 3
```### Linear Algebra
`Tensor`s provide easy access to power Linear Algebra routines backed by
LAPACK and BLAS implementations, and ClBlast for GPU backed `Tensor`s.```crystal
a = [[1, 2], [3, 4]].to_tensor.map &.to_f32puts a.inv
# [[-2 , 1 ],
# [1.5 , -0.5]]puts a.eigvals
# [-0.372281, 5.37228 ]
puts a.matmul(a)
# [[7 , 10],
# [15, 22]]
```### Einstein Notation
For representing certain complex contractions of `Tensor`s, Einstein notation
can be used to simplify the operation. For example, the following matrix
multiplication + summation operation:```crystal
a = Tensor.new([30, 40, 50]) { |i| i * 1_f32 }
b = Tensor.new([40, 30, 20]) { |i| i * 1_f32 }result = Float32Tensor.zeros([50, 20])
ny, nx = result.shape
b2 = b.swap_axes(0, 1)
ny.times do |k|
nx.times do |l|
result[k, l] = (a[..., ..., k] * b2[..., ..., l]).sum
end
end
```Can instead be represented in Einstein notiation as the following:
```crystal
Num::Einsum.einsum("ijk,jil->kl", a, b)
```This can lead to performance improvements due to optimized contractions
on `Tensor`s.```
einsum 2.22k (450.41µs) (± 0.86%) 350kB/op fastest
manual 117.52 ( 8.51ms) (± 0.98%) 5.66MB/op 18.89× slower
```### Machine Learning
`Num::Grad` provides a pure-crystal approach to find derivatives of
mathematical functions. Use a `Num::Grad::Variable` with a `Num::Grad::Context`
to easily compute these derivatives.```crystal
ctx = Num::Grad::Context(Tensor(Float64, CPU(Float64))).newx = ctx.variable([3.0].to_tensor)
y = ctx.variable([2.0].to_tensor)# f(x) = x ** y
f = x ** y
puts f # => [9]f.backprop
# df/dx = y * x = 6.0
puts x.grad # => [6.0]
````Num::NN` contains an extension to `Num::Grad` that provides an easy-to-use
interface to assist in creating neural networks. Designing and creating
a network is simple using Crystal's block syntax.```crystal
ctx = Num::Grad::Context(Tensor(Float64, CPU(Float64))).newx_train = [[0.0, 0.0], [1.0, 0.0], [0.0, 1.0], [1.0, 1.0]].to_tensor
y_train = [[0.0], [1.0], [1.0], [0.0]].to_tensorx = ctx.variable(x_train)
net = Num::NN::Network.new(ctx) do
input [2]
# A basic network with a single hidden layer using
# a ReLU activation function
linear 3
relu
linear 1# SGD Optimizer
sgd 0.7# Sigmoid Cross Entropy to calculate loss
sigmoid_cross_entropy_loss
end500.times do |epoch|
y_pred = net.forward(x)
loss = net.loss(y_pred, y_train)
puts "Epoch: #{epoch} - Loss #{loss}"
loss.backprop
net.optimizer.update
end# Clip results to make a prediction
puts net.forward(x).value.map { |el| el > 0 ? 1 : 0}# [[0],
# [1],
# [1],
# [0]]
```Review the documentation for full implementation details, and if something is missing,
open an issue to add it!