https://github.com/rwth-i6/returnn

The RWTH extensible training framework for universal recurrent neural networks
https://github.com/rwth-i6/returnn

deep-learning gpu recurrent-neural-networks tensorflow theano

Last synced: about 1 month ago
JSON representation

The RWTH extensible training framework for universal recurrent neural networks

• Host: GitHub
• URL: https://github.com/rwth-i6/returnn
• Owner: rwth-i6
• License: other
• Created: 2016-06-14T14:37:14.000Z (over 8 years ago)
• Default Branch: master
• Last Pushed: 2024-05-22T15:59:50.000Z (6 months ago)
• Last Synced: 2024-05-22T17:03:31.817Z (6 months ago)
• Topics: deep-learning, gpu, recurrent-neural-networks, tensorflow, theano
• Language: Python
• Homepage: http://returnn.readthedocs.io/
• Size: 25.7 MB
• Stars: 349
• Watchers: 31
• Forks: 130
• Open Issues: 160
• Metadata Files:
  • Readme: README.rst
  • Changelog: CHANGELOG.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE
  • Codeowners: CODEOWNERS

Awesome Lists containing this project

README

        
==================

Welcome to RETURNN

==================

`GitHub repository `__.

`RETURNN paper 2016 `_,

`RETURNN paper 2018 `_.

RETURNN - RWTH extensible training framework for universal recurrent neural networks,

is a Theano/TensorFlow-based implementation of modern recurrent neural network architectures.

It is optimized for fast and reliable training of recurrent neural networks in a multi-GPU environment.

The high-level features and goals of RETURNN are:

* **Simplicity**

  * Writing config / code is simple & straight-forward (setting up experiment, defining model)

  * Debugging in case of problems is simple

  * Reading config / code is simple (defined model, training, decoding all becomes clear)

* **Flexibility**

  * Allow for many different kinds of experiments / models

* **Efficiency**

  * Training speed

  * Decoding speed

All items are important for research, decoding speed is esp. important for production.

See our `Interspeech 2020 tutorial "Efficient and Flexible Implementation of Machine Learning for ASR and MT" video `__

(`slides `__)

with an introduction of the core concepts.

More specific features include:

- Mini-batch training of feed-forward neural networks

- Sequence-chunking based batch training for recurrent neural networks

- Long short-term memory recurrent neural networks

  including our own fast CUDA kernel

- Multidimensional LSTM (GPU only, there is no CPU version)

- Memory management for large data sets

- Work distribution across multiple devices

- Flexible and fast architecture which allows all kinds of encoder-attention-decoder models

See `documentation `__.

See `basic usage `__

and `technological overview `__.

`Here is the video recording of a RETURNN overview talk `_

(`slides `__,

`exercise sheet `__;

hosted by eBay).

There are `many example demos `_

which work on artificially generated data,

i.e. they should work as-is.

There are `some real-world examples `_

such as setups for speech recognition on the Switchboard or LibriSpeech corpus.

Some benchmark setups against other frameworks

can be found `here `_.

The results are in the `RETURNN paper 2016 `_.

Performance benchmarks of our LSTM kernel vs CuDNN and other TensorFlow kernels

are in `TensorFlow LSTM benchmark `__.

There is also `a wiki `_.

Questions can also be asked on

`StackOverflow using the RETURNN tag `_.

.. image:: https://github.com/rwth-i6/returnn/workflows/CI/badge.svg

    :target: https://github.com/rwth-i6/returnn/actions