Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/Blealtan/RWKV-LM-LoRA
RWKV is a RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
https://github.com/Blealtan/RWKV-LM-LoRA
Last synced: about 2 months ago
JSON representation
RWKV is a RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
- Host: GitHub
- URL: https://github.com/Blealtan/RWKV-LM-LoRA
- Owner: Blealtan
- License: apache-2.0
- Fork: true (BlinkDL/RWKV-LM)
- Created: 2023-02-15T05:41:54.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2023-07-11T06:43:01.000Z (about 1 year ago)
- Last Synced: 2024-07-03T22:24:21.982Z (3 months ago)
- Language: Python
- Homepage:
- Size: 10.9 MB
- Stars: 406
- Watchers: 11
- Forks: 40
- Open Issues: 25