{"id":46238719,"url":"https://github.com/i404788/s5-pytorch","last_synced_at":"2026-03-03T19:34:38.173Z","repository":{"id":170022837,"uuid":"616717359","full_name":"i404788/s5-pytorch","owner":"i404788","description":"Pytorch implementation of Simplified Structured State-Spaces for Sequence Modeling (S5)","archived":false,"fork":false,"pushed_at":"2024-04-26T09:36:59.000Z","size":59,"stargazers_count":78,"open_issues_count":2,"forks_count":4,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-09-18T06:51:40.530Z","etag":null,"topics":["pytorch","s5","sequence-modeling","state-space"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/i404788.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-03-20T23:57:07.000Z","updated_at":"2025-09-12T20:52:35.000Z","dependencies_parsed_at":"2023-11-09T11:50:25.266Z","dependency_job_id":"08eb6a7f-fc56-481e-8a2e-262f7668e181","html_url":"https://github.com/i404788/s5-pytorch","commit_stats":null,"previous_names":["i404788/s5-pytorch"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/i404788/s5-pytorch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/i404788%2Fs5-pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/i404788%2Fs5-pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/i404788%2Fs5-pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/i404788%2Fs5-pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/i404788","download_url":"https://codeload.github.com/i404788/s5-pytorch/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/i404788%2Fs5-pytorch/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30056070,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-03T18:21:05.932Z","status":"ssl_error","status_checked_at":"2026-03-03T18:20:59.341Z","response_time":61,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["pytorch","s5","sequence-modeling","state-space"],"created_at":"2026-03-03T19:34:37.483Z","updated_at":"2026-03-03T19:34:38.162Z","avatar_url":"https://github.com/i404788.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# S5: Simplified State Space Layers for Sequence Modeling\nThis is a ported version derived from \u003chttps://github.com/lindermanlab/S5\u003e and \u003chttps://github.com/kavorite/S5\u003e.\nIt includes a bunch of functions ported from jax/lax/flax/whatever since they didn't exist yet. \n\n~~Jax is required because it relies on the pytree structure but it's not used for any computation.~~\nSince version 0.2.0 jax is not required, it's using the pytorch native `torch.utils._pytree` (this may be incompatible for pytorch future versions).\nPytorch 2 or later is required because it makes heavy use of `torch.vmap` and `torch.utils._pytree` to substitute it's jax counterpart.\nPython 3.10 or later is required due to usage of the `match` keyword\n\n\\--- \n\nUpdate:\n\nIn my experiments it follows the results found in the [Hyena Hierarchy](https://arxiv.org/abs/2302.10866) (\u0026 H3) paper that the state spaces alone lack the recall capabilities required for LLM but seem work well for regular sequence feature extraction and linear complexity.\n\nYou can use variable step-size as described in the paper using a 1D tensor for `step_scale` however this takes **a lot of memory** due to a lot of intermediate values needing to be held (which I believe is true for the official S5 repo, but not mentioned in the paper unless I missed it).\n\n## Install\n\n```sh\npip install s5-pytorch \n```\n\n## Example\n\n```py3\nfrom s5 import S5, S5Block\n\n# Raw S5 operator\nx = torch.rand([2, 256, 32])\nmodel = S5(32, 32)\nmodel(x) # [2, 256, 32]\n\n# S5-former block (S5+FFN-GLU w/ layernorm, dropout \u0026 residual)\nmodel = S5Block(32, 32, False)\nmodel(x) # [2, 256, 32]\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fi404788%2Fs5-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fi404788%2Fs5-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fi404788%2Fs5-pytorch/lists"}