Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/jatinchowdhury18/RTNeural
Real-time neural network inferencing
https://github.com/jatinchowdhury18/RTNeural
Last synced: 3 months ago
JSON representation
Real-time neural network inferencing
- Host: GitHub
- URL: https://github.com/jatinchowdhury18/RTNeural
- Owner: jatinchowdhury18
- License: bsd-3-clause
- Created: 2020-10-15T17:08:01.000Z (about 4 years ago)
- Default Branch: main
- Last Pushed: 2024-07-08T23:35:42.000Z (4 months ago)
- Last Synced: 2024-07-22T05:35:06.257Z (4 months ago)
- Language: C++
- Size: 4.2 MB
- Stars: 550
- Watchers: 25
- Forks: 58
- Open Issues: 20
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-musicdsp - RTNeural - A library designed for realtime inference of Neural Network models with audio in mind. Very small when compared to linking a library like OnnxRuntime or libtensorflow. (Machine Learning)
README
# RTNeural
[](https://github.com/jatinchowdhury18/RTNeural/actions/workflows/tests.yml)
[](https://github.com/jatinchowdhury18/RTNeural/actions/workflows/bench.yml)
[](https://github.com/jatinchowdhury18/RTNeural/actions/workflows/examples.yml)
[](https://github.com/jatinchowdhury18/RTNeural/actions/workflows/radsan.yml)
[](https://codecov.io/gh/jatinchowdhury18/RTNeural)
[](https://arxiv.org/abs/2106.03037)
[](https://opensource.org/licenses/BSD-3-Clause)
A lightweight neural network inferencing engine written in C++.
This library was designed with the intention of being used in
real-time systems, specifically real-time audio processing.
Currently supported layers:
- [x] Dense
- [x] GRU
- [x] LSTM
- [x] Conv1D
- [x] Conv2D
- [ ] MaxPooling
- [x] BatchNorm1D
- [x] BatchNorm2D
Currently supported activations:
- [x] tanh
- [x] ReLU
- [x] Sigmoid
- [x] SoftMax
- [x] ELu
- [x] PReLU
Additional resources:
- [RTNeural Discord](https://discord.gg/QMBBucKt4Q)
- [API Reference](https://ccrma.stanford.edu/~jatin/chowdsp/RTNeural)
- [Reference Paper](https://arxiv.org/abs/2106.03037)
- [Example Plugin](https://github.com/jatinchowdhury18/RTNeural-example)
- [Comparison Benchmarks](https://github.com/jatinchowdhury18/RTNeural-compare)
- [Experimental Extensions](https://github.com/jatinchowdhury18/RTNeural-Experimental)
## Citation
If you are using RTNeural as part of an academic work, please cite the library as follows:
```
@article{chowdhury2021rtneural,
title={RTNeural: Fast Neural Inferencing for Real-Time Systems},
author={Jatin Chowdhury},
year={2021},
journal={arXiv preprint arXiv:2106.03037}
}
```
## How To Use
`RTNeural` is capable of taking a neural network that
has already been trained, loading the weights from that
network, and running inference. Some simple examples
are available in the [`examples/`](./examples) directory.
### Exporting weights from a trained network
Neural networks are typically trained using `Python`
libraries including Tensorflow or PyTorch. Once you
have trained a neural network using one of these frameworks,
you can "export" the network weights to a json file,
so that `RTNeural` can read them. An implementation of
the export process for a "sequential" Tensorflow model is
provided in `python/model_utils.py`, and can be used as follows.
```python
# import dependencies
import tensorflow as tf
from tensorflow import keras
from model_utils import save_model
# create Tensrflow model
model = keras.Sequential()
...
# train model
model.train()
# export model weights
save_model(model, 'model_weights.json')
```
For an example of exporting a model from PyTorch,
see [this example script](./python/gru_torch.py).
### Creating a model
Next, you can create an inferencing engine in C++ directly
from the exported json file:
```cpp
#include
...
std::ifstream jsonStream("model_weights.json", std::ifstream::binary);
auto model = RTNeural::json_parser::parseJson(jsonStream);
```
### Running inference
Before running inference, it is recommended to "reset" the
state of your model (if the model has state).
```cpp
model->reset();
```
Then, you may run inference as follows:
```cpp
double input[] = { 1.0, 0.5, -0.1 }; // set up input vector
double output = model->forward(input); // compute output
```
### Compile-Time API
The code shown above will create the inferencing engine
dynamically at run-time. If the model architecture is
fixed at compile-time, it may be preferable to use RTNeural's
API for defining an inferencing engine type at compile-time,
which can significantly improve performance.
```cpp
// define model type
RTNeural::ModelT,
RTNeural::TanhActivationT,
RTNeural::DenseT
> modelT;
// load model weights from json
std::ifstream jsonStream("model_weights.json", std::ifstream::binary);
modelT.parseJson(jsonStream);
modelT.reset(); // reset state
double input[] = { 1.0, 0.5, -0.1 }; // set up input vector
double output = modelT.forward(input); // compute output
```
### Loading Layers from PyTorch
The above example code assumes that the trained model has
been exported from TensorFlow. For loading PyTorch models,
the RTNeural namespace `RTNeural::torch_helpers`, provides
helper functions for loading layers exported from PyTorch.
```cpp
// load model weights from json
std::ifstream jsonStream("model_weights.json", std::ifstream::binary);
nlohmann::json modelJson;
jsonStream >> modelJson;
// load a layer from a static model
RTNeural::ModelT> model;
RTNeural::torch_helpers::loadDense(modelJson, "name_of_layer.", model.get<0>());
```
For more examples, see the
[`examples/torch`](./examples/torch) directory.
## Building with CMake
`RTNeural` is built with CMake, and the easiest way to link
is to include `RTNeural` as a submodule:
```cmake
...
add_subdirectory(RTNeural)
target_link_libraries(MyCMakeProject LINK_PUBLIC RTNeural)
```
If you are trying to use RTNeural in a project that does not use
CMake, please see the [instructions below](#building-without-cmake).
### Choosing a Backend
`RTNeural` supports three backends,
[`Eigen`](http://eigen.tuxfamily.org/),
[`xsimd`](https://github.com/xtensor-stack/xsimd),
or the C++ STL. You can choose your backend by passing
either `-DRTNEURAL_EIGEN=ON`, `-DRTNEURAL_XSIMD=ON`,
or `-DRTNEURAL_STL=ON` to your CMake configuration. By
default, the `Eigen` backend will be used. Alternatively,
you may select your choice of backends in your CMake
configuration as follows:
```cmake
set(RTNEURAL_XSIMD ON CACHE BOOL "Use RTNeural with this backend" FORCE)
add_subdirectory(modules/RTNeural)
```
In general, the `Eigen` backend typically has the best
performance for larger networks, while smaller networks
may perform better with XSIMD. However, it is recommended
to measure the performance of your network with all the
backends that are available on your target platform
to ensure optimal performance. For more information see the
[benchmark results](https://github.com/jatinchowdhury18/RTNeural/actions?query=workflow%3ABench).
Note that you must abide by the licensing rules of whichever backend library you choose.
### Other configuration flags
If you would like to build RTNeural with the AVX SIMD extensions,
you may run CMake with the `-DRTNEURAL_USE_AVX=ON`. Note that
this flag will have no effect when compiling for platforms that
do not support AVX instructions.
### Building the test suite
To build RTNeural's test suite, run `cmake -Bbuild -DBUILD_TESTS=ON`, followed
by `cmake --build build`. To run the full testing suite, run `ctest` from the
`build` folder. For more information, see `tests/README.md`.
### Building the Performance Benchmarks
To build the performance benchmarks, run
`cmake -Bbuild -DBUILD_BENCH=ON`, followed by
`cmake --build build --config Release`. To run the layer benchmarks, run
`./build/rtneural_layer_bench `. To
run the model benchmark, run `./build/rtneural_model_bench`.
### Building the Examples
To build the RTNeural examples run:
```bash
cmake -Bbuild -DBUILD_EXAMPLES=ON
cmake --build build --config Release
```
The example programs will then be located in
`build/examples_out/`, and may be run from there.
An example of using RTNeural within a real-time
audio plugin can be found on GitHub
[here](https://github.com/jatinchowdhury18/RTNeural-example).
## Building without CMake
If you wish to use RTNeural in a project that doesn't use CMake,
RTNeural can be included as a header-only library, along with a few
extra steps.
1. Add a compile-time definition to define a default byte alignment for RTNeural.
For most cases this definition will be one of either:
- `RTNEURAL_DEFAULT_ALIGNMENT=16`
- `RTNEURAL_DEFAULT_ALIGNMENT=32`
2. Add a compile-time definition to [select a backend](#choosing-a-backend).
If you wish to use the STL backend, then no definition is required.
This definition should be one of the following:
- `RTNEURAL_USE_EIGEN=1`
- `RTNEURAL_USE_XSIMD=1`
4. Add the necessary include paths for your chosen backend. This path will be
one of either:
- `/modules/Eigen`
- `/modules/xsimd/include/xsimd`
It may also be worth checking out the
[example Makefile](./examples/hello_rtneural/Makefile).
## Contributing
Contributions to this project are most welcome!
Currently, there is a need for the
following improvements:
- Improved support for 2-dimensional input/output data.
- Improved support for "stateless" Conv1D layers.
- More robust support for exporting/loading models.
- Support for more activation layers.
- Any changes that improve overall performance.
General code maintenance and documentation is always
appreciated as well! Note that if you are implementing
a new layer type, it is not required to provide support
for all the backends, though it is recommended to at
least provide a "fallback" implementation using the STL
backend.
## Contributors
Please thank the following individuals for their important contributions:
- [wayne-chen](https://github.com/wayne-chen): Softmax activation layer and general API improvements.
- [hollance](https://github.com/hollance): RTNeural logo.
- [stepanmk](https://github.com/stepanmk): Eigen Conv1D layer optimization.
- [DamRsn](https://github.com/DamRsn): Eigen implementations for Conv2D and BatchNorm2D layers.
- [lHorvalds](https://github.com/IHorvalds): Eigen backend optimizations.
- [davidtrevelyan](https://github.com/davidtrevelyan): Testing framework upgrade.
- [purefunctor](https://github.com/purefunctor): Groups feature for Conv1D.
## Powered by RTNeural
RTNeural is currently being used by several audio plugins and other projects:
- [4000DB-NeuralAmp](https://github.com/EnrcDamn/4000DB-NeuralAmp): Neural emulation of the pre-amp section from the Akai 4000DB tape machine.
- [AIDA-X](https://github.com/AidaDSP/AIDA-X): An AU/CLAP/LV2/VST2/VST3 audio plugin that loads RTNeural models and cabinet IRs.
- [BYOD](https://github.com/Chowdhury-DSP/BYOD): A guitar distortion plugin containing several machine learning-based effects.
- [Chow Centaur](https://github.com/jatinchowdhury18/KlonCentaur): A guitar pedal emulation plugin, using a real-time recurrent neural network.
- [Chow Tape Model](https://github.com/jatinchowdhury18/AnalogTapeModel): An analog tape emulation, using a real-time dense neural network.
- [cppTimbreID](https://github.com/domenicostefani/cpp-timbreID): An audio feature extraction library.
- [guitarix](https://github.com/brummer10/guitarix): A guitarix effects suite, including neural network amplifier models.
- [GuitarML](https://guitarml.com/): GuitarML plugins use machine learning to model guitar amplifiers and effects.
- [MLTerror15](https://github.com/IHorvalds/MLTerror15): Deeply learned simulator for the Orange Tiny Terror with Recurrent Neural Networks.
- [NeuralNote](https://github.com/DamRsn/NeuralNote): An audio-to-MIDI transcription plugin using Spotify's [basic-pitch](https://github.com/spotify/basic-pitch) model.
- [rt-neural-lv2](https://github.com/AidaDSP/aidadsp-lv2): A headless lv2 plugin using RTNeural to model guitar pedals and amplifiers.
- Tone Empire plugins:
- [LVL - 01](https://tone-empire.com/shop/lvl-01/): An A.I./M.L.-based compressor effect.
- [TM700](https://tone-empire.com/shop/tm700/): A machine learning tape emulation effect.
- [Neural Q](https://tone-empire.com/shop/neuralq-v2/): An analog emulation 2-band EQ, using recurrent neural networks.
- [ToobAmp](https://github.com/rerdavies/ToobAmp): Guitar effect plugins for the Raspberry Pi.
If you are using RTNeural in one of your projects, let us know and we will add it to this list!
## License
RTNeural is open source, and is licensed under the
BSD 3-clause license.
Enjoy!