{"id":13641598,"url":"https://github.com/blinkdl/chatrwkv","last_synced_at":"2025-05-13T16:03:25.560Z","repository":{"id":65257520,"uuid":"588477692","full_name":"BlinkDL/ChatRWKV","owner":"BlinkDL","description":"ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.","archived":false,"fork":false,"pushed_at":"2025-05-02T05:44:41.000Z","size":31266,"stargazers_count":9480,"open_issues_count":57,"forks_count":704,"subscribers_count":93,"default_branch":"main","last_synced_at":"2025-05-06T16:06:48.361Z","etag":null,"topics":["chatbot","chatgpt","language-model","pytorch","rnn","rwkv"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BlinkDL.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"ko_fi":"rwkv_lm"}},"created_at":"2023-01-13T08:07:40.000Z","updated_at":"2025-05-06T04:11:27.000Z","dependencies_parsed_at":"2023-12-14T12:12:28.269Z","dependency_job_id":"a8f08867-d3f8-4d0c-9f6b-9fc9fb4fb976","html_url":"https://github.com/BlinkDL/ChatRWKV","commit_stats":{"total_commits":363,"total_committers":17,"mean_commits":"21.352941176470587","dds":0.5234159779614325,"last_synced_commit":"d67bbb34aeb01bff68a02aecb9721d992555dcea"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BlinkDL%2FChatRWKV","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BlinkDL%2FChatRWKV/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BlinkDL%2FChatRWKV/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BlinkDL%2FChatRWKV/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BlinkDL","download_url":"https://codeload.github.com/BlinkDL/ChatRWKV/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253979812,"owners_count":21994037,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatbot","chatgpt","language-model","pytorch","rnn","rwkv"],"created_at":"2024-08-02T01:01:22.117Z","updated_at":"2025-05-13T16:03:25.545Z","avatar_url":"https://github.com/BlinkDL.png","language":"Python","funding_links":["https://ko-fi.com/rwkv_lm"],"categories":["Chatbots"],"sub_categories":[],"readme":"# ChatRWKV (pronounced as \"RwaKuv\" (rʌkuv in IPA), from 4 major params: R W K V)\n\n## RWKV homepage: https://www.rwkv.com\n\nRWKV-7 code: https://github.com/BlinkDL/RWKV-LM/tree/main/RWKV-v7\n\nChatRWKV is like ChatGPT but powered by my RWKV (100% RNN) language model, which is the only RNN (as of now) that can match transformers in quality and scaling, while being faster and saves VRAM.\n\nOur latest version is **RWKV-7** https://arxiv.org/abs/2503.14456 (Preview models: https://huggingface.co/BlinkDL/temp )\n\nGradio Demo 1: https://huggingface.co/spaces/BlinkDL/RWKV-Gradio-1\n\nGradio Demo 2: https://huggingface.co/spaces/BlinkDL/RWKV-Gradio-2\n\n**RWKV-LM main repo**: https://github.com/BlinkDL/RWKV-LM (explanation, fine-tuning, training, etc.)\n\nChat Demo for developers: https://github.com/BlinkDL/ChatRWKV/blob/main/API_DEMO_CHAT.py\n\n## RWKV Discord: https://discord.gg/bDSBUMeFpc (7k+ members)\n\n**Twitter**: https://twitter.com/BlinkDL_AI\n\n**Homepage**: https://www.rwkv.com/\n\n**Raw cutting-edge RWKV weights:** https://huggingface.co/BlinkDL\n\n**HF-compatible RWKV weights:** https://huggingface.co/RWKV\n\nUse v2/convert_model.py to convert a model for a strategy, for faster loading \u0026 saves CPU RAM.\n\nNote RWKV_CUDA_ON will build a CUDA kernel (much faster \u0026 saves VRAM). Here is how to build it (\"pip install ninja\" first):\n```\n# How to build in Linux: set these and run v2/chat.py\nexport PATH=/usr/local/cuda/bin:$PATH\nexport LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH\n# How to build in win:\nInstall VS2022 build tools (https://aka.ms/vs/17/release/vs_BuildTools.exe select Desktop C++). Reinstall CUDA 11.7 (install VC++ extensions). Run v2/chat.py in \"x64 native tools command prompt\". \n```\n**RWKV pip package**: https://pypi.org/project/rwkv/ **(please always check for latest version and upgrade)**\n\nhttps://github.com/cgisky1980/ai00_rwkv_server Fastest GPU inference API with vulkan (good for nvidia/amd/intel)\n\nhttps://github.com/cryscan/web-rwkv backend for ai00_rwkv_server\n\nhttps://github.com/saharNooby/rwkv.cpp Fast CPU/cuBLAS/CLBlast inference: int4/int8/fp16/fp32\n\nhttps://github.com/JL-er/RWKV-PEFT lora/pissa/Qlora/Qpissa/state tuning\n\nhttps://github.com/RWKV/RWKV-infctx-trainer Infctx trainer\n\n**World demo script:** https://github.com/BlinkDL/ChatRWKV/blob/main/API_DEMO_WORLD.py\n\n**Raven Q\u0026A demo script:** https://github.com/BlinkDL/ChatRWKV/blob/main/v2/benchmark_more.py\n\n![ChatRWKV-strategy](ChatRWKV-strategy.png)\n\n**RWKV in 150 lines** (model, inference, text generation): https://github.com/BlinkDL/ChatRWKV/blob/main/RWKV_in_150_lines.py\n\n**🔥 RWKV v5 in 250 lines 🔥** (with tokenizer too): https://github.com/BlinkDL/ChatRWKV/blob/main/RWKV_v5_demo.py\n\n**🔥 Building your own RWKV inference engine 🔥**: begin with https://github.com/BlinkDL/ChatRWKV/blob/main/src/model_run.py which is easier to understand (used by https://github.com/BlinkDL/ChatRWKV/blob/main/chat.py).\n\n**RWKV preprint** https://arxiv.org/abs/2305.13048\n\n![RWKV-paper](RWKV-paper.png)\n\nRWKV v6 illustrated:\n\n![RWKV-v6](rwkv-x060.png)\n\n**Cool Community RWKV Projects**:\n\nhttps://github.com/saharNooby/rwkv.cpp fast i4 i8 fp16 fp32 CPU inference using [ggml](https://github.com/ggerganov/ggml)\n\nhttps://github.com/harrisonvanderbyl/rwkv-cpp-cuda fast windows/linux \u0026 cuda/rocm/vulkan GPU inference (no need for python \u0026 pytorch)\n\nhttps://github.com/Blealtan/RWKV-LM-LoRA LoRA fine-tuning\n\nhttps://github.com/josStorer/RWKV-Runner cool GUI\n\nMore RWKV projects: https://github.com/search?o=desc\u0026q=rwkv\u0026s=updated\u0026type=Repositories\n\nChatRWKV v2: with \"stream\" and \"split\" strategies, and INT8. 3G VRAM is enough to run RWKV 14B :) https://github.com/BlinkDL/ChatRWKV/tree/main/v2\n```python\nos.environ[\"RWKV_JIT_ON\"] = '1'\nos.environ[\"RWKV_CUDA_ON\"] = '0' # if '1' then use CUDA kernel for seq mode (much faster)\nfrom rwkv.model import RWKV                         # pip install rwkv\nmodel = RWKV(model='/fsx/BlinkDL/HF-MODEL/rwkv-4-pile-1b5/RWKV-4-Pile-1B5-20220903-8040', strategy='cuda fp16')\n\nout, state = model.forward([187, 510, 1563, 310, 247], None)   # use 20B_tokenizer.json\nprint(out.detach().cpu().numpy())                   # get logits\nout, state = model.forward([187, 510], None)\nout, state = model.forward([1563], state)           # RNN has state (use deepcopy if you want to clone it)\nout, state = model.forward([310, 247], state)\nprint(out.detach().cpu().numpy())                   # same result as above\n```\n![RWKV-eval](RWKV-eval.png)\n\nHere is https://huggingface.co/BlinkDL/rwkv-4-raven/blob/main/RWKV-4-Raven-14B-v7-Eng-20230404-ctx4096.pth in action:\n![ChatRWKV](ChatRWKV.png)\n\nWhen you build a RWKV chatbot, always check the text corresponding to the state, in order to prevent bugs.\n\n1. Never call raw forward() directly. Instead, put it in a function that will record the text corresponding to the state.\n\n**(For v4-raven models, use Bob/Alice. For v4/v5/v6-world models, use User/Assistant)**\n\n2. The best chat format (check whether your text is of this format):\n```Bob: xxxxxxxxxxxxxxxxxx\\n\\nAlice: xxxxxxxxxxxxx\\n\\nBob: xxxxxxxxxxxxxxxx\\n\\nAlice:```\n\n* There should not be any space after the final \"Alice:\". The generation result will have a space in the beginning, and you can simply strip it.\n* You can use \\n in xxxxx, but avoid \\n\\n. So simply do ```xxxxx = xxxxx.strip().replace('\\r\\n','\\n').replace('\\n\\n','\\n')```\n\nIf you are building your own RWKV inference engine, begin with https://github.com/BlinkDL/ChatRWKV/blob/main/src/model_run.py which is easier to understand (used by https://github.com/BlinkDL/ChatRWKV/blob/main/chat.py)\n\nThe lastest \"Raven\"-series Alpaca-style-tuned RWKV 14B \u0026 7B models are very good (almost ChatGPT-like, good at multiround chat too). Download: https://huggingface.co/BlinkDL/rwkv-4-raven\n\nPrevious old model results:\n![ChatRWKV](misc/sample-1.png)\n![ChatRWKV](misc/sample-2.png)\n![ChatRWKV](misc/sample-3.png)\n![ChatRWKV](misc/sample-4.png)\n![ChatRWKV](misc/sample-5.png)\n![ChatRWKV](misc/sample-6.png)\n![ChatRWKV](misc/sample-7.png)\n\n## 中文模型\n\nQQ群 553456870（加入时请简单自我介绍）。有研发能力的朋友加群 325154699。\n\n中文使用教程：https://zhuanlan.zhihu.com/p/618011122 https://zhuanlan.zhihu.com/p/616351661\n\n推荐UI：https://github.com/l15y/wenda\n\n## Star History\n\n[![Star History Chart](https://api.star-history.com/svg?repos=BlinkDL/ChatRWKV\u0026type=Date)](https://star-history.com/#BlinkDL/ChatRWKV\u0026Date)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fblinkdl%2Fchatrwkv","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fblinkdl%2Fchatrwkv","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fblinkdl%2Fchatrwkv/lists"}