{"id":27329440,"url":"https://github.com/csirmaz/superloop","last_synced_at":"2026-05-09T04:31:42.227Z","repository":{"id":72744495,"uuid":"120684899","full_name":"csirmaz/superloop","owner":"csirmaz","description":"A framework to build RNNs with a \"superloop\" in TensorFlow/Keras (Python)","archived":false,"fork":false,"pushed_at":"2018-08-09T16:00:34.000Z","size":156,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-12T12:43:39.710Z","etag":null,"topics":["keras","keras-models","neural-network","rnn","tensorflow"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/csirmaz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-02-07T23:26:24.000Z","updated_at":"2023-07-29T19:51:12.000Z","dependencies_parsed_at":null,"dependency_job_id":"3bd0414b-67e1-438a-9122-b3bb080198e7","html_url":"https://github.com/csirmaz/superloop","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/csirmaz/superloop","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csirmaz%2Fsuperloop","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csirmaz%2Fsuperloop/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csirmaz%2Fsuperloop/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csirmaz%2Fsuperloop/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/csirmaz","download_url":"https://codeload.github.com/csirmaz/superloop/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csirmaz%2Fsuperloop/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32807168,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-08T08:22:46.396Z","status":"online","status_checked_at":"2026-05-09T02:00:06.633Z","response_time":123,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["keras","keras-models","neural-network","rnn","tensorflow"],"created_at":"2025-04-12T12:33:17.090Z","updated_at":"2026-05-09T04:31:42.222Z","avatar_url":"https://github.com/csirmaz.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# superloop\n\nAn implementation of \"superloops\" around recurrent neural networks (RNNs).\n\nA \"superloop\" is an extra loop that connects the output of the RNN to\nits input in the next timestep via one or more external systems (\"X\").\n\n![superloop overview](https://raw.githubusercontent.com/csirmaz/superloop/master/img/superloop1.png)\n\nThe external system can implement a special type of memory (derivable stack memory,\naddressable memory, etc.), an attention system, or other enhancement.\n\nThis project contains:\n\n- A framework to build RNNs with a \"superloop\" in TensorFlow/Keras\n- An implementation of a simple gated recurrent unit (\"SGU\") we use instead of LSTM or GRU cells in the RNN\n- An implementation of a register-based memory unit\n- An implementation of an attention system that allows the network to move freely along the input data\n\n## The superloop framework\n\nThe framework implements the RNN and the superloop by unrolling both.\nThe diagram below illustrates one RNN unit (LSTM, GRU, etc.) and how it is\nunrolled in time.\n\n![unrolling one unit](https://raw.githubusercontent.com/csirmaz/superloop/master/img/superloop2.png)\n\nWith two units (or, which is equivalent, layers of units) one after the other,\nunrolling looks like this:\n\n![unrolling two units](https://raw.githubusercontent.com/csirmaz/superloop/master/img/superloop3.png)\n\nIt is easy to see that the internal loops of the RNN and the superloop need to be unrolled\nat the same time:\n\n![unrolling the superloop](https://raw.githubusercontent.com/csirmaz/superloop/master/img/superloop4.png)\n\n**Builder** is a base class that allows building the same connections between layers\nagain and again, while reusing the layers themselves. This makes it possible to build\nan unrolled model easily. See `builder.py`.\n\nThe **Model** class is used by client code. It implements the (configurable) recipe for building the full model.\nSee `model.py`.\n\n## \"Simple Gated Recurrent Unit\" (SGU)\n\nWe use SGU cells in the RNN part of the model.\nThe SGU is implemented by a linear combination between the hidden state (internal input) and the (external) input, controlled\nby both inputs.\n\n![sgu](https://raw.githubusercontent.com/csirmaz/superloop/master/img/sgu1.png)\n\nWhere W1, W2, W3 are trainable weights. The switch represents a simple linear combination:\n\n![switch](https://raw.githubusercontent.com/csirmaz/superloop/master/img/sgu2.png)\n\nSee `sgu.py` for more.\n\n## Attention System\n\nA superloop extension that allows the system to focus on different parts of the input.\n    \nThe data the model operates on has N data locations, each of which is a 1D tensor.\nThe model maintains an index that can be modified by at most -1. or 1. in each step.\nIts output is the same size as the data at one datapoint, and it combines the datapoint\nthe index selects with the neighboring ones with a factor of 1/(1+d^2) where d\nis the distance between the index and the location of the datapoint.\n\nSee `attention.py` for more.\n\n## Register Memory\n\nA memory of multiple registers that can be used as an external system in a superloop.\n    \nThere are D (\"depth\") registers, each W (\"width\") wide. The RNN part of the model can\ncontrol writing and reading the registers, and receives the data read in the next timestep.\n\nSee `regmem.py` for more.\n\n## Usage\n\nThis example creates a model with an attention system.\nIts final output is the index at which the attention system finishes.\n\n```python\nCONFIG = { # See model.py for the description of the keys\n    'timesteps': 64,\n    'suppress_output': 0,\n    'model_name': 'Main',\n    'model_inputs': 2,\n    'model_outputs': 0,\n    'recurrent_model': 'SGU',\n    'recurrent_layers': 32,\n    'recurrent_units': 16,\n    'superloop_models': ['Attention'],\n    'printvalues': False,\n    'Attention': {\n        'datapoints': 16,\n        'outputs': 1,\n    }\n}\n\nslmodel = superloop.Model(CONFIG)\n(input, dummy_output) = slmodel.build_all()\n\nif load_model_file:\n    slmodel.load_weights(load_model_file)\n\nmodel = keras.models.Model(\n    inputs=[input, slmodel.superloop_models[0].data],\n    outputs=slmodel.superloop_models[0].position\n)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcsirmaz%2Fsuperloop","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcsirmaz%2Fsuperloop","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcsirmaz%2Fsuperloop/lists"}