Ecosyste.ms: Awesome

https://github.com/blinkdl/rwkv-lm

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

attention-mechanism chatgpt deep-learning gpt gpt-2 gpt-3 language-model linear-attention lstm pytorch rnn rwkv transformer transformers

Last synced: 02 Aug 2024

https://github.com/BlinkDL/RWKV-LM

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

attention-mechanism chatgpt deep-learning gpt gpt-2 gpt-3 language-model linear-attention lstm pytorch rnn rwkv transformer transformers

Last synced: 31 Jul 2024

https://github.com/blinkdl/chatrwkv

ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.

chatbot chatgpt language-model pytorch rnn rwkv

Last synced: 02 Aug 2024

https://github.com/BlinkDL/ChatRWKV

ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.

chatbot chatgpt language-model pytorch rnn rwkv

Last synced: 30 Jul 2024

https://github.com/BlinkDL/AI-Writer

AI 写小说，生成玄幻和言情网文等等。中文预训练生成模型。采用我的 RWKV 模型，类似 GPT-2 。AI写作。RWKV for Chinese novel generation.

Last synced: 02 Aug 2024

https://github.com/BlinkDL/Hua

Hua is an AI image editor with Stable Diffusion (and more).

Last synced: 01 Aug 2024

https://github.com/BlinkDL/nanoRWKV

RWKV in nanoGPT style

Last synced: 31 Jul 2024

https://github.com/BlinkDL/LinearAttentionArena

Here we will test various linear attention designs.

Last synced: 09 Aug 2024

https://github.com/BlinkDL/SmallInitEmb

LayerNorm(SmallInit(Embedding)) in a Transformer to improve convergence

Last synced: 09 Aug 2024