Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
Projects in Awesome Lists tagged with linear-attention
A curated list of projects in awesome lists tagged with linear-attention .
https://github.com/blinkdl/rwkv-lm
RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
attention-mechanism chatgpt deep-learning gpt gpt-2 gpt-3 language-model linear-attention lstm pytorch rnn rwkv transformer transformers
Last synced: 29 Sep 2024
https://github.com/BlinkDL/RWKV-LM
RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
attention-mechanism chatgpt deep-learning gpt gpt-2 gpt-3 language-model linear-attention lstm pytorch rnn rwkv transformer transformers
Last synced: 31 Jul 2024
https://github.com/lucidrains/taylor-series-linear-attention
Explorations into the recently proposed Taylor Series Linear Attention
artificial-intelligence attention-mechanisms deep-learning linear-attention
Last synced: 03 Oct 2024
https://github.com/lucidrains/agent-attention-pytorch
Implementation of Agent Attention in Pytorch
artificial-intelligence attention-mechanisms deep-learning linear-attention
Last synced: 03 Oct 2024
https://github.com/lucidrains/autoregressive-linear-attention-cuda
CUDA implementation of autoregressive linear attention, with all the latest research findings
artificial-intelligence attention-mechanisms cuda deep-learning linear-attention
Last synced: 03 Oct 2024
https://github.com/robflynnyh/hydra-linear-attention
Implementation of: Hydra Attention: Efficient Attention with Many Heads (https://arxiv.org/abs/2209.07484)
attention efficient-attention linear-attention machine-learning transformers
Last synced: 02 Aug 2024