{"id":13444647,"url":"https://github.com/raminmh/liquid-s4","last_synced_at":"2025-04-05T20:06:59.805Z","repository":{"id":71912850,"uuid":"490904794","full_name":"raminmh/liquid-s4","owner":"raminmh","description":"Liquid Structural State-Space Models","archived":false,"fork":false,"pushed_at":"2024-02-01T16:01:49.000Z","size":2220,"stargazers_count":343,"open_issues_count":6,"forks_count":59,"subscribers_count":15,"default_branch":"main","last_synced_at":"2025-03-29T19:08:25.132Z","etag":null,"topics":["deep-learning","liquid-neural-networks","neural-networks","recurrent-neural-networks","sequence-modeling","state-space-models","time-series"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2209.12951","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/raminmh.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-05-11T00:36:27.000Z","updated_at":"2025-03-19T12:56:05.000Z","dependencies_parsed_at":"2024-10-28T06:52:17.004Z","dependency_job_id":"bc82f54c-1da0-461c-90cc-4eefadd1b65a","html_url":"https://github.com/raminmh/liquid-s4","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/raminmh%2Fliquid-s4","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/raminmh%2Fliquid-s4/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/raminmh%2Fliquid-s4/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/raminmh%2Fliquid-s4/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/raminmh","download_url":"https://codeload.github.com/raminmh/liquid-s4/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247393569,"owners_count":20931812,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","liquid-neural-networks","neural-networks","recurrent-neural-networks","sequence-modeling","state-space-models","time-series"],"created_at":"2024-07-31T04:00:32.853Z","updated_at":"2025-04-05T20:06:59.782Z","avatar_url":"https://github.com/raminmh.png","language":"Python","funding_links":[],"categories":["Before 2023"],"sub_categories":[],"readme":"\t\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/liquid-structural-state-space-models/spo2-estimation-on-bidmc)](https://paperswithcode.com/sota/spo2-estimation-on-bidmc?p=liquid-structural-state-space-models)\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/liquid-structural-state-space-models/heart-rate-estimation-on-bidmc)](https://paperswithcode.com/sota/heart-rate-estimation-on-bidmc?p=liquid-structural-state-space-models)\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/liquid-structural-state-space-models/speech-recognition-on-speech-commands-2)](https://paperswithcode.com/sota/speech-recognition-on-speech-commands-2?p=liquid-structural-state-space-models)\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/liquid-structural-state-space-models/long-range-modeling-on-lra)](https://paperswithcode.com/sota/long-range-modeling-on-lra?p=liquid-structural-state-space-models)\n\n\n# Liquid State Space Models ([Paper](https://arxiv.org/abs/2209.12951))\n\nThis repository provides the implementation of Liquid S4 state-space models. Liquid-S4 takes state-spaces models to another level by utilizing a linearized version of [liquid neural networks](https://github.com/raminmh/liquid_time_constant_networks) at its core. Read the preprint for more details:\nhttps://arxiv.org/abs/2209.12951\n\nThe repository is a recent fork of the S4 repo (https://github.com/HazyResearch/state-spaces). It includes the Liquid-S4 KB and PB kernels.\n\n## Setup\n\n### Requirements\nThis repository requires Python 3.8+ and Pytorch 1.9+.  \nOther packages are listed in `requirements.txt`.\n\n`pip3 install -r requirement.txt`\n\nTo install the Custom Cauchy Kernel (more efficient) developed by [Gu et al. 2022](https://github.com/HazyResearch/state-spaces):\n```bash\ncd extensions/cauchy/\npython3 setup.py install\n```\n\nYou can also skip installing the custom cauchy kernel, and use the kernel provided by the [PyKeOps Library](https://www.kernel-operations.io/keops/python/installation.html). If you `pip3 install -r requirement.txt`, this package will be installed. \n\n## Datasets:\n\nsCIFAR is downloaded automatically when running a training job.  \nSpeech Commands dataset is downloaded automatically when running a training job.  \nAll Long Range Arena (LRA) (except IMDB and Cifar which are auto-downloaded) tasks could be downloaded directly from ths gziped file: [LRA Full Dataset](https://storage.googleapis.com/long-range-arena/lra_release.gz).  \nAfter downloading the LRA task, organize a `data/` folder with the following directory structure:\n\n```bash\n$data/\n  pathfinder/\n    pathfinder32/\n    pathfinder64/\n    pathfinder128/\n    pathfinder256/\n  aan/\n  listops/\n```\n\nIf the IMDB dataset did not get downloaded, you can run the following script to download it: [src/dataloaders/imdb_dataset.sh](https://github.com/raminmh/liquid-s4/blob/main/src/dataloaders/imdb_dataset.sh) This bash script places the imdb dataset into a proper \n\n## Train Liquid-S4 Models\n\n```bash\n# plain S4\npython3 -m train wandb=null experiment=lra/s4-lra-imdb\n\n# Liquid-S4 PB Kernel: (PB kernels are faster and perform better than KB)\npython3 -m train wandb=null experiment=lra/s4-lra-imdb model.layer.liquid_kernel=polyb\n# liquid-S4 KB Kernel:\npython3 -m train wandb=null experiment=lra/s4-lra-imdb model.layer.liquid_kernel=kb\n\n# Increase Liquid Order:\npython3 -m train wandb=null experiment=lra/s4-lra-imdb model.layer.liquid_kernel=polyb model.layer.liquid_degree=3\npython3 -m train wandb=null experiment=lra/s4-lra-imdb model.layer.liquid_kernel=kb model.layer.liquid_degree=3\n\n```\n\n\nThe default config files are all included in the `config/` folder. To run each experiment change the flag `experiment=` to any of the following YAML files:  \n\n```bash\nlra/s4-lra-listops # Long Range Arena: Listops\nlra/s4-lra-imdb # Long Range Arena: IMDB Character Level Sentiment Classification (text)\nlra/s4-lra-cifar # Long Range Arena: Sequential CIFAR (image)\nlra/s4-lra-aan # Long Range Arena: AAN (Retreival)\nlra/s4-lra-pathfinder # Long Range Arena: Pathfinder\nlra/s4-lra-pathx # Long Range Arena: Path-x\n\nsc/s4-sc  # Speech Commands Recognition Full Dataset\n\nbidmc/s4-bidmc # BIDMC Heart Rate (HR), Raspiratory Rate (RR), and Blood Oxygen Saturation (SpO2)\n\n\n#Example: \npython3 -m train wandb=null experiment=lra/s4-lra-listops model.layer.liquid_kernel=polyb model.layer.liquid_degree=2\n\n```\n\n### Optimizer Hyperparameters from [S4 Repo](https://github.com/HazyResearch/state-spaces)\n\nOne notable difference in this codebase is that some S4 parameters use different optimizer hyperparameters. In particular, the SSM kernel is particularly sensitive to the A, B, and dt parameters, so the optimizer settings for these parameters are usually fixed to learning rate 0.001 and weight decay 0.\n\nOur logic for setting these parameters can be found in the `OptimModule` class under `src/models/sequence/ss/kernel.py` and the corresponding optimizer hook in `SequenceLightningModule.configure_optimizers` under `train.py`.\n\n## Training from [S4 Repo](https://github.com/HazyResearch/state-spaces):\n\nThe core training infrastructure of this repository is based on [Pytorch-Lightning](https://pytorch-lightning.readthedocs.io/en/latest/) with a configuration scheme based on [Hydra](https://hydra.cc/docs/intro/).\nThe structure of this integration largely follows the Lightning+Hydra integration template described in https://github.com/ashleve/lightning-hydra-template.\n\nThe main experiment entrypoint is `train.py` and configs are found in `configs/`.\nIn brief, the main config is found at `configs/config.yaml`, which is combined with other sets of configs that can be passed on the command line, to define an overall YAML config.\nMost config groups define one single Python object (e.g. a PyTorch nn.Module).\nThe end-to-end training pipeline can broken down into the following rough groups, where group XX is found under `configs/XX/`:\n```\nmodel: the sequence-to-sequence model backbone (e.g. a src.models.sequence.SequenceModel)\ndataset: the raw dataset (data/target pairs) (e.g. a pytorch Dataset)\nloader: how the data is loaded (e.g. a pytorch DataLoader)\nencoder: defines a Module that interfaces between data and model backbone\ndecoder: defines a Module that interfaces between model backbone and targets\ntask: specifies loss and metrics\n```\n\n### Hydra from [S4 Repo](https://github.com/HazyResearch/state-spaces)\n\nIt is recommended to read the Hydra documentation to fully understand the configuration framework. For help launching specific experiments, please file an Issue.\n\n### Registries from [S4 Repo](https://github.com/HazyResearch/state-spaces)\n\nThis codebase uses a modification of the hydra `instantiate` utility that provides shorthand names of different classes, for convenience in configuration and logging.\nThe mapping from shorthand to full path can be found in `src/utils/registry.py`.\n\n### WandB from [S4 Repo](https://github.com/HazyResearch/state-spaces)\n\nLogging with [WandB](https://wandb.ai/site) is built into this repository.\nIn order to use this, simply set your `WANDB_API_KEY` environment variable, and change the `wandb.project` attribute of `configs/config.yaml` (or pass it on the command line `python -m train .... wandb.project=s4`).\n\nSet `wandb=null` to turn off WandB logging.\n\n\n## Citation\n\n```\n@article{hasani2022liquid,\n  title={Liquid Structural State-Space Models},\n  author={Hasani, Ramin and Lechner, Mathias and Wang, Tsun-Huang and Chahine, Makram and Amini, Alexander and Rus, Daniela},\n  journal={arXiv preprint arXiv:2209.12951},\n  year={2022}\n}\n\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Framinmh%2Fliquid-s4","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Framinmh%2Fliquid-s4","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Framinmh%2Fliquid-s4/lists"}