{"id":14970732,"url":"https://github.com/nengo/keras-lmu","last_synced_at":"2025-10-25T20:33:59.028Z","repository":{"id":40400983,"uuid":"217137279","full_name":"nengo/keras-lmu","owner":"nengo","description":"Keras implementation of Legendre Memory Units","archived":false,"fork":false,"pushed_at":"2025-04-02T19:50:58.000Z","size":15876,"stargazers_count":213,"open_issues_count":18,"forks_count":34,"subscribers_count":19,"default_branch":"main","last_synced_at":"2025-04-12T02:59:45.459Z","etag":null,"topics":["keras","legendre","lmu","lstm","nengo","recurrent-neural-networks","tensorflow"],"latest_commit_sha":null,"homepage":"https://www.nengo.ai/keras-lmu/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nengo.png","metadata":{"files":{"readme":"README.rst","changelog":"CHANGES.rst","contributing":"CONTRIBUTING.rst","funding":null,"license":"LICENSE.rst","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2019-10-23T19:28:38.000Z","updated_at":"2025-04-06T17:29:26.000Z","dependencies_parsed_at":"2023-02-17T22:01:08.582Z","dependency_job_id":"f1602078-042f-474f-add1-66f940016ecb","html_url":"https://github.com/nengo/keras-lmu","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nengo%2Fkeras-lmu","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nengo%2Fkeras-lmu/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nengo%2Fkeras-lmu/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nengo%2Fkeras-lmu/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nengo","download_url":"https://codeload.github.com/nengo/keras-lmu/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248510008,"owners_count":21116131,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["keras","legendre","lmu","lstm","nengo","recurrent-neural-networks","tensorflow"],"created_at":"2024-09-24T13:44:03.527Z","updated_at":"2025-10-25T20:33:59.023Z","avatar_url":"https://github.com/nengo.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"KerasLMU: Recurrent neural networks using Legendre Memory Units\n---------------------------------------------------------------\n\n`Paper \u003chttps://papers.nips.cc/paper/9689-legendre-memory-units-continuous-time-representation-in-recurrent-neural-networks.pdf\u003e`_\n\nThis is a Keras-based implementation of the\nLegendre Memory Unit (LMU). The LMU is a novel memory cell for recurrent neural\nnetworks that dynamically maintains information across long windows of time using\nrelatively few resources. It has been shown to perform as well as standard LSTM or\nother RNN-based models in a variety of tasks, generally with fewer internal parameters\n(see `this paper\n\u003chttps://papers.nips.cc/paper/9689-legendre-memory-units-continuous-time-representation-in-recurrent-neural-networks.pdf\u003e`_ for more details). For the Permuted Sequential MNIST (psMNIST) task in particular, it has been demonstrated to outperform the current state-of-the-art results. See the note below for instructions on how to get access to this model.\n\nThe LMU is mathematically derived to orthogonalize its continuous-time history – doing\nso by solving *d* coupled ordinary differential equations (ODEs), whose phase space\nlinearly maps onto sliding windows of time via the Legendre polynomials up to degree\n*d* − 1 (the example for *d* = 12 is shown below).\n\n.. image:: https://i.imgur.com/Uvl6tj5.png\n   :target: https://i.imgur.com/Uvl6tj5.png\n   :alt: Legendre polynomials\n\nA single LMU cell expresses the following computational graph, which takes in an input\nsignal, **x**, and couples a optimal linear memory, **m**, with a nonlinear hidden\nstate, **h**. By default, this coupling is trained via backpropagation, while the\ndynamics of the memory remain fixed.\n\n.. image:: https://i.imgur.com/IJGUVg6.png\n   :target: https://i.imgur.com/IJGUVg6.png\n   :alt: Computational graph\n\nThe discretized **A** and **B** matrices are initialized according to the LMU's\nmathematical derivation with respect to some chosen window length, **θ**.\nBackpropagation can be used to learn this time-scale, or fine-tune **A** and **B**,\nif necessary.\n\nBoth the kernels, **W**, and the encoders, **e**, are learned. Intuitively, the kernels\nlearn to compute nonlinear functions across the memory, while the encoders learn to\nproject the relevant information into the memory (see `paper\n\u003chttps://papers.nips.cc/paper/9689-legendre-memory-units-continuous-time-representation-in-recurrent-neural-networks.pdf\u003e`_ for details).\n\n.. note::\n\n   The ``paper`` branch in the ``lmu`` GitHub repository includes a pre-trained\n   Keras/TensorFlow model, located at ``models/psMNIST-standard.hdf5``, which obtains\n   a psMNIST result of **97.15%**. Note that the network is using fewer internal\n   state-variables and neurons than there are pixels in the input sequence.\n   To reproduce the results from `this paper\n   \u003chttps://papers.nips.cc/paper/9689-legendre-memory-units-continuous-time-representation-in-recurrent-neural-networks.pdf\u003e`_,\n   run the notebooks in the ``experiments`` directory within the ``paper`` branch.\n\nNengo Examples\n--------------\n\n* `LMUs in Nengo (with online learning)\n  \u003chttps://www.nengo.ai/nengo/examples/learning/lmu.html\u003e`_\n* `Spiking LMUs in Nengo Loihi (with online learning)\n  \u003chttps://www.nengo.ai/nengo-loihi/examples/lmu.html\u003e`_\n* `LMUs in NengoDL (reproducing SotA on psMNIST)\n  \u003chttps://www.nengo.ai/nengo-dl/examples/lmu.html\u003e`_\n\nCitation\n--------\n\n.. code-block::\n\n   @inproceedings{voelker2019lmu,\n     title={Legendre Memory Units: Continuous-Time Representation in Recurrent Neural Networks},\n     author={Aaron R. Voelker and Ivana Kaji\\'c and Chris Eliasmith},\n     booktitle={Advances in Neural Information Processing Systems},\n     pages={15544--15553},\n     year={2019}\n   }\n\nPatent Notice\n-------------\n\nThis software is covered by one or more patents. For a full list, please visit:\n`https://www.appliedbrainresearch.com/leading-ai-chip-innovators-explore-our-patents-at-abr`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnengo%2Fkeras-lmu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnengo%2Fkeras-lmu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnengo%2Fkeras-lmu/lists"}