Projects in Awesome Lists tagged with efficient-attention
A curated list of projects in awesome lists tagged with efficient-attention .
https://github.com/thu-ml/sageattention
Quantized Attention achieves speedup of 2-3x and 3-5x compared to FlashAttention and xformers, without lossing end-to-end metrics across language, image, and video models.
attention cuda efficient-attention inference-acceleration llm llm-infra mlsys quantization triton video-generate video-generation vit
Last synced: 14 May 2025
https://github.com/lucidrains/ring-attention-pytorch
Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch
attention-mechanism distributed-attention efficient-attention long-context
Last synced: 15 May 2025
https://github.com/lucidrains/colt5-attention
Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch
artificial-intelligence attention-mechanisms deep-learning efficient-attention routing
Last synced: 09 Apr 2025
https://github.com/robflynnyh/hydra-linear-attention
Implementation of: Hydra Attention: Efficient Attention with Many Heads (https://arxiv.org/abs/2209.07484)
attention efficient-attention linear-attention machine-learning transformers
Last synced: 27 Apr 2025
https://github.com/gmlwns2000/sea-attention
Official Implementation of SEA: Sparse Linear Attention with Estimated Attention Mask (ICLR 2024)
attention efficient-attention linear-attention sea-attention
Last synced: 16 Jan 2026
https://github.com/pszemraj/samba-pytorch
Minimal implementation of Samba by Microsoft in PyTorch
efficient-attention language-model llm long-context-modeling mamba-state-space-models pytorch-implementation ssm
Last synced: 24 Mar 2025
