awesome-ssm-ml
Reading list for research topics in state-space models
https://github.com/AvivBick/awesome-ssm-ml
Last synced: 4 days ago
JSON representation
-
Tutorials <a name="tutorials"></a>
- The Annotated S4
- Mamba: SSM, Theory, and Implementation in Keras and TensorFlow
- Mamba: SSM, Theory, and Implementation in Keras and TensorFlow
- Mamba: SSM, Theory, and Implementation in Keras and TensorFlow
- Mamba No. 5 (A Little Bit Of...)
- Mamba: SSM, Theory, and Implementation in Keras and TensorFlow
- Mamba: SSM, Theory, and Implementation in Keras and TensorFlow
- Mamba: SSM, Theory, and Implementation in Keras and TensorFlow
- S4 Series
- The Annotated S4D
- Do we need Attention? A Mamba Primer
- Mamba: The Easy Way
- A Visual Guide to Mamba and State Space Models
- State Space Models: A Modern Approach
- Efficiently Modeling Long Sequences with Structured State Spaces
- Mamba and S4 Explained: Architecture, Parallel Scan, Kernel Fusion, Recurrent, Convolution, Math
- MAMBA from Scratch
- Yannic Kilcher's Video
- Mamba: SSM, Theory, and Implementation in Keras and TensorFlow
- Mamba: SSM, Theory, and Implementation in Keras and TensorFlow
- Mamba: SSM, Theory, and Implementation in Keras and TensorFlow
- Mamba: SSM, Theory, and Implementation in Keras and TensorFlow
- Mamba: SSM, Theory, and Implementation in Keras and TensorFlow
- Mamba: SSM, Theory, and Implementation in Keras and TensorFlow
- Mamba: The Hard Way
- Mamba: SSM, Theory, and Implementation in Keras and TensorFlow
- Mamba: SSM, Theory, and Implementation in Keras and TensorFlow
- Mamba: SSM, Theory, and Implementation in Keras and TensorFlow
- Mamba: SSM, Theory, and Implementation in Keras and TensorFlow
-
Miscellaneous
- Legendre Memory Units: Continuous-Time Representation in Recurrent Neural Networks
- Variational learning for switching state-space models
- Liquid structural state-space models
- Resurrecting Recurrent Neural Networks for Long Sequences
- Learning Low Dimensional State Spaces with Overparameterized Recurrent Neural Nets
- Never Train from Scratch: Fair Comparison Of Long- Sequence Models Requires Data-Driven Pirors
-
Architecture
- S7: Selective and Simplified State Space Layers for Sequence Modeling
- Jamba-1.5: Hybrid Transformer-Mamba Models at Scale
- S5: Simplified State Space Layers for Sequence Modeling
- Pretraining Without Attention
- MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts - random/llm-random)
- LOCOST: State-Space Models for Long Document Abstractive Summarization - summarization)
- BlackMamba: Mixture of Experts for State-Space Models
- DenseMamba: State Space Models with Dense Hidden Connection for Efficient Large Language Models
- ZigMa: Zigzag Mamba Diffusion Model (ECCV 2024)
- Jamba: A Hybrid Transformer-Mamba Language Model
- Block-State Transformers
- Efficient Long Sequence Modeling via State Space Augmented Transformer
-
SSM Parameterization and Initialization
- From Generalization Analysis to Optimization Designs for State Space Models
- On the Parameterization and Initialization of Diagonal State Space Models
- Diagonal State Spaces are as Effective as Structured State Spaces
- How to Train your HIPPO: State Space Models with Generalized Orthogonal Basis Projections
- Robustifying State-space Models for Long Sequences via Approximate Diagonalization
- StableSSM: Alleviating the Curse of Memory in State-space Models through Stable Reparameterization
- Spectral State Space Models
- Efficiently Modeling Long Sequences with Structured State Spaces
-
Medical
- BioMamba: A Pre-trained Biomedical Language Representation Model Leveraging Mamba
- MambaMorph: a Mamba-based Backbone with Contrastive Feature Learning for Deformable MR-CT Registration - Stone/MambaMorph)
- Structured State Space Models for Multiple Instance Learning in Digital Pathology
- Modeling Multivariate Biosignals with Graph Neural Networks and Structured State Space
- Diffusion-based conditional ECG generation with structured state space models
- Improving the Diagnosis of Psychiatric Disorders with Self-Supervised Graph State Space Models
- fMRI-S4: learning short- and long-range dynamic fMRI dependencies using 1D Convolutions and State Space Models
- Vivim: a Video Vision Mamba for Medical Video Object Segmentation - yjyang/Vivim)
- SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image Segmentation - xing/SegMamba)
- U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentation - lab/U-Mamba)
- nnMamba: 3D Biomedical Image Segmentation, Classification and Landmark Detection with State Space Model
- VM-UNet: Vision Mamba UNet for Medical Image Segmentation
- MambaMIR: An Arbitrary-Masked Mamba for Joint Medical Image Reconstruction and Uncertainty Estimation
- ViM-UNet: Vision Mamba for Biomedical Segmentation
- I2I-Mamba: Multi-modal medical image synthesis via selective state space modeling - lab/I2I-Mamba)
- MambaRoll: Physics-Driven Autoregressive State Space Models for Medical Image Reconstruction - lab/MambaRoll)
-
Vision
- MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Models
- SUM: Saliency Unification through Mamba for Visual Attention Modeling
- State Space Models for Event Cameras - rpg/ssms_event_cameras) (CVPR 2024 Spotlight)
- DynamicVis: An Efficient and General Visual Foundation Model for Remote Sensing Image Understanding
- U-shaped Vision Mamba for Single Image Dehazing - idam/UVM-Net)
- S4ND: Modeling Images and Videos as Multidimensional Signals with State Spaces
- Long movie clip classification with state-space video models - mohaiminul/ViS4mer)
- Efficient Movie Scene Detection using State-Space Transformers
- Selective Structured State-Spaces for Long-Form Video Understanding
- 2-D SSM: A General Spatial Layer for Visual Transformers
- Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model
- VMamba: Visual State Space Model
- Res-VMamba: Fine-Grained Food Category Visual Classification Using Selective State Space Models with Deep Residual Learning
- Weak-Mamba-UNet: Visual Mamba Makes CNN and ViT Work Better for Scribble-based Medical Image Segmentation - UNet)
- LocalMamba: Visual State Space Model with Windowed Selective Scan
- Motion Mamba: Efficient and Long Sequence Motion Generation with Hierarchical and Bidirectional Selective SSM - zeyu-zhang.github.io/MotionMamba/?utm_source=catalyzex.com)
- A Survey on Visual Mamba
-
Systems Optimizations
-
Audio
- Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separation - j/Mamba-TasNet)
- Speech Slytherin: Examining the Performance and Efficiency of Mamba for Speech Separation, Recognition, and Synthesis - j/Mamba-ASR)
- A Neural State-Space Model Approach to Efficient Speech Separation
- It's Raw! Audio Generation with State-Space Models - spaces/s4)
- Augmenting conformers with structured state space models for online speech recognition
- Diagonal State Space Augmented Transformers for Speech Recognition
- Structured State Space Decoder for Speech Recognition and Synthesis
- Spiking Structured State Space Model for Monaural Speech Enhancement
- Multi-Head State Space Model for Speech Recognition
- SSAMBA: Self-Supervised Audio Representation Learning with Mamba State Space Model
- Audio Mamba: Bidirectional State Space Model for Audio Representation Learning - Mamba-AuM)
-
Surveys (Structured State Space Models) <a name="surveys"></a>
-
Books (Classical State Space Models) <a name="books"></a>
-
Foundation
- Structured state-space models are deep Wiener models
- State-space Models with Layer-wise Nonlinearity are Universal Approximators with Exponential Decaying Memory
- Repeat After Me: Transformers are Better than State Space Models at Copying
- Theoretical Foundations of Deep Selective State-Space Models
- The Hidden Attention of Mamba Models
- The Expressive Capacity of State Space Models: A Formal Language Perspective
- Simplifying and Understanding State Space Models with Diagonal Linear RNNs
-
Language
- Hungry Hungry Hippos: Towards Language Modeling with State Space Models
- MambaByte: Token-free Selective State Space Model - pytorch)
- Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks
- Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Models - mamba)
- Long range language modeling via gated state spaces - state-spaces-pytorch.git)
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces - spaces/mamba)
-
Time-Series
- Deep State Space Models for Time Series Forecasting
- FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecasting
- Effectively modeling time series with simple discrete state spaces
- Deep Latent State Space Models for Time-Series Generation
- Generative AI for End-to-End Limit Order Book Modelling
- On the Performance of Legendre State-Space Models in Short-Term Time Series Forecasting
- Neural Continuous-Discrete State Space Models for Irregularly-Sampled Time Series
- Diffusion-based Time Series Imputation and Forecasting with Structured State Space Models
-
Tabular
-
Reinforcement Learning
Programming Languages
Categories
Tutorials <a name="tutorials"></a>
29
Vision
17
Medical
16
Architecture
12
Audio
11
Time-Series
8
SSM Parameterization and Initialization
8
Foundation
7
Miscellaneous
6
Language
6
Reinforcement Learning
3
Surveys (Structured State Space Models) <a name="surveys"></a>
3
Books (Classical State Space Models) <a name="books"></a>
2
Systems Optimizations
2
Tabular
1
Sub Categories
Keywords