https://github.com/lucidrains/light-recurrent-unit-pytorch

Implementation of a Light Recurrent Unit in Pytorch
https://github.com/lucidrains/light-recurrent-unit-pytorch

artificial-intelligence deep-learning recurrent-neural-networks

Last synced: about 10 hours ago
JSON representation

Implementation of a Light Recurrent Unit in Pytorch

• Host: GitHub
• URL: https://github.com/lucidrains/light-recurrent-unit-pytorch
• Owner: lucidrains
• License: mit
• Created: 2024-08-29T13:06:30.000Z (about 1 month ago)
• Default Branch: main
• Last Pushed: 2024-08-30T14:25:59.000Z (about 1 month ago)
• Last Synced: 2024-08-30T14:41:48.842Z (about 1 month ago)
• Topics: artificial-intelligence, deep-learning, recurrent-neural-networks
• Language: Python
• Homepage:
• Size: 980 KB
• Stars: 18
• Watchers: 2
• Forks: 1
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        


## Light Recurrent Unit - Pytorch

Implementation of the Light Recurrent Unit in Pytorch

## Install

```bash

$ pip install light-recurrent-unit-pytorch

```

## Usage

```python

import torch

from light_recurrent_unit_pytorch import LightRecurrentUnitCell

cell = LightRecurrentUnitCell(256)

x = torch.randn(2, 256)

hidden = torch.randn(2, 256)

next_hidden = cell(x, hidden) # (2, 256)

```

Single layer

```python

import torch

from light_recurrent_unit_pytorch import LightRecurrentUnitLayer

layer = LightRecurrentUnitLayer(256)

x = torch.randn(2, 1024, 256)

out = layer(x) # (2, 1024, 256)

assert out.shape == x.shape

```

Stacked

```python

import torch

from light_recurrent_unit_pytorch import LightRecurrentUnit

lru = LightRecurrentUnit(256, depth = 4)

x = torch.randn(2, 1024, 256)

out, layer_hiddens = lru(x) # (2, 1024, 256), List[(2, 256)]

assert out.shape == x.shape

```

## Citations

```bibtex

@Article{electronics13163204,

    AUTHOR = {Ye, Hong and Zhang, Yibing and Liu, Huizhou and Li, Xuannong and Chang, Jiaming and Zheng, Hui},

    TITLE = {Light Recurrent Unit: Towards an Interpretable Recurrent Neural Network for Modeling Long-Range Dependency},

    JOURNAL = {Electronics},

    VOLUME = {13},

    YEAR = {2024},

    NUMBER = {16},

    ARTICLE-NUMBER = {3204},

    URL = {https://www.mdpi.com/2079-9292/13/16/3204},

    ISSN = {2079-9292},

    DOI = {10.3390/electronics13163204}

}

```