{"id":13422959,"url":"https://github.com/project-baize/baize-chatbot","last_synced_at":"2025-04-13T22:29:01.295Z","repository":{"id":150601656,"uuid":"621934144","full_name":"project-baize/baize-chatbot","owner":"project-baize","description":"Let ChatGPT teach your own chatbot in hours with a single GPU!","archived":false,"fork":false,"pushed_at":"2024-03-17T18:06:03.000Z","size":71064,"stargazers_count":3166,"open_issues_count":37,"forks_count":290,"subscribers_count":49,"default_branch":"main","last_synced_at":"2025-04-06T18:14:12.859Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2304.01196","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/project-baize.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-31T17:56:03.000Z","updated_at":"2025-04-04T06:52:13.000Z","dependencies_parsed_at":"2024-12-13T07:12:14.863Z","dependency_job_id":null,"html_url":"https://github.com/project-baize/baize-chatbot","commit_stats":{"total_commits":69,"total_committers":6,"mean_commits":11.5,"dds":"0.33333333333333337","last_synced_commit":"4fae6c4e550f087958c1f60746f0e6290e6b9f02"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/project-baize%2Fbaize-chatbot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/project-baize%2Fbaize-chatbot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/project-baize%2Fbaize-chatbot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/project-baize%2Fbaize-chatbot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/project-baize","download_url":"https://codeload.github.com/project-baize/baize-chatbot/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248249453,"owners_count":21072369,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T23:01:00.679Z","updated_at":"2025-04-13T22:29:01.247Z","avatar_url":"https://github.com/project-baize.png","language":"Python","readme":"\u003cp align=\"center\"\u003e\n\u003cimg width=\"500px\" alt=\"Project Baize\" src=\"https://user-images.githubusercontent.com/22514219/229195563-0cddfa74-e52f-4413-b4b4-e4ba489c4b3d.png\"\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\u003ca href=\"https://arxiv.org/abs/2304.01196\"\u003e[📄 Paper]\u003c/a\u003e | \u003ca href=\"https://huggingface.co/spaces/project-baize/Baize-7B\"\u003e[🤗 Demo]\u003c/a\u003e | \u003ca href=\"https://github.com/project-baize/baize-chatbot#cli-and-api\"\u003e[🔧 CLI \u0026 API]\u003c/a\u003e \u003c/p\u003e\n\u003chr\u003e\n\n## News\n- **[May 23, 2023]** We are releasing Baize v2! Check out the [7B](https://huggingface.co/project-baize/baize-v2-7b) and [13B](https://huggingface.co/project-baize/baize-v2-13b) model. Code coming soon!\n- **[Apr. 27, 2023]** [Fastchat](https://github.com/lm-sys/FastChat) now supports Baize. Try the new [CLI and API](https://github.com/project-baize/baize-chatbot#cli-and-api)!\n- **[Apr. 21, 2023]** We now have a [script](https://github.com/project-baize/baize-chatbot#merge-lora-into-llama) to merge LoRA weights into standard HF model so you can use it everywhere HF is supported!\n\n## What's Baize?\nBaize is an open-source chat model trained with [LoRA](https://github.com/microsoft/LoRA). It uses 100k dialogs generated by letting ChatGPT chat with itself. We also use Alpaca's data to improve its performance. We have released 7B, 13B and 30B models. Please refer to the [paper](https://arxiv.org/pdf/2304.01196.pdf) for more details.\n\n## Why it's called Baize?\nBaize (pronounced as By-zor; Simplified Chinese 白泽, Traditional Chinese 白澤, Japanese 白沢, はくたく) is a mythical creature in Chinese folklore, who speaks human languages and knows everything. This is exactly what we expect from a chat model.\n\n## Overview\n⚠️ The code is released under [GPL-3.0](./LICENSE). All model weights and data are for **research use ONLY**. Commercial use is **strictly prohibited**. We accept **NO responsibility or liability** for any use of our data, code or weights.\n\nThis is the repo for the Baize project, which aims to build a chat model with LLaMA. This repository contains:\n\n- 54K/57K/47K [dialogs](data) from Quora, StackOverFlow and MedQuAD questions\n- The code for collecting self-chat data: [v1](collect.py), [v2](collect_v2.py)\n- The [code](finetune.py) for training Baize\n- The [code](demo/app.py) for chat model demo (forked from [ChuanhuChatGPT](https://github.com/GaiZhenbiao/ChuanhuChatGPT))\n\n### Model Release\n#### V1\n- [Baize-v1-7B (LoRA weights)](https://huggingface.co/project-baize/baize-lora-7B)\n- [Baize-v1-13B (LoRA weights)](https://huggingface.co/project-baize/baize-lora-13B)\n- [Baize-v1-30B (LoRA weights)](https://huggingface.co/project-baize/baize-lora-30B)\n- [Baize Healthcare-7B (LoRA weights)](https://huggingface.co/project-baize/baize-healthcare-lora-7b)\n\n#### V2\n- [Baize-v2-7B](https://huggingface.co/project-baize/baize-v2-7b)\n- [Baize-v2-13B](https://huggingface.co/project-baize/baize-v2-13b)\n\n### Community Models and Data\n- [Falcon-7B-Instruct](https://huggingface.co/tiiuae/falcon-7b-instruct) and [Falcon-40B-Instruct](https://huggingface.co/tiiuae/falcon-40b-instruct) are official Falcon models fine-tuned with Baize data. Falcon is the current state-of-the-art open-source model developed by [TII](https://www.tii.ae/).\n- [Fauno](https://github.com/RSTLess-research/Fauno-Italian-LLM/) is an Italian version of Baize.\n- [Dutch Data](https://github.com/project-baize/baize-chatbot/issues/34): Baize data translated into Dutch.\n\n## CLI and API\nNow you can use Baize with [Fastchat](https://github.com/lm-sys/FastChat) for the CLI and API provided by Fastchat!\n\nFirst, install the latest version of Fastchat:\n```bash\npip install git+https://github.com/huggingface/peft.git\npip install git+https://github.com/lm-sys/FastChat.git\n```\n\n(For v1 models only): Merge Baize's LoRA weights into LLaMA. Take 7B checkpoint as an example.\n```bash\n# Note you have to include \"baize\" in the target directory so Fastchat can recognize Baize.\npython3 -m fastchat.model.apply_lora --base huggyllama/llama-7b --target ./model_weights/baize-7b --lora project-baize/baize-lora-7B\n```\n\nNow, run the CLI in your terminal! More options and configs can be found [here](https://github.com/lm-sys/FastChat#inference-with-command-line-interface).\n```bash\n# Optional: Add `--style rich` for better style.\npython -m fastchat.serve.cli --model-path ./model_weights/baize-7b\n```\n\nYou can use Baize with OpenAI API or Hugging Face API following the instruction [here](https://github.com/lm-sys/FastChat#api).\n\n## Demo\n[![Open in Spaces](https://huggingface.co/datasets/huggingface/badges/raw/main/open-in-hf-spaces-md.svg)](https://huggingface.co/spaces/project-baize/Baize-7B) \n[![Duplicate this Space](https://huggingface.co/datasets/huggingface/badges/raw/main/duplicate-this-space-md.svg)](https://huggingface.co/spaces/project-baize/Baize-7B?duplicate=true)\n\n\u003cp align=\"center\"\u003e\n  \u003cimg alt=\"Demo\" src=\"https://user-images.githubusercontent.com/22514219/229863275-0e83c1cf-0661-4afa-9a47-1ce20fb521ae.gif\"\u003e\n\u003c/p\u003e\n\nYou can either host it on your local machine or access the [online demo](https://huggingface.co/spaces/project-baize/Baize-7B). The demo fetches the [LLaMA](https://huggingface.co/huggyllama/llama-7b) model and the [LoRA weights](https://huggingface.co/project-baize/baize-lora-7B) from the Hugging Face model hub, then runs a user-friendly Gradio interface for chatting.\n\n### How to Run Locally\n\nFirst, make sure your Python version is 3.8, and then install the required packages using the command below:\n\n```bash\ncd demo\npip install -r requirements.txt\n```\n\nYou can host the model on your local machine using the following command:\n\n```bash\n# We assume you have obtained access to use LLaMA. The following LLaMA weights are from a 3rd party.\nbase_model=huggyllama/llama-7b\nlora_model=project-baize/baize-lora-7B\npython app.py $base_model $lora_model\n```\nFor v2 models (already merged), simply run:\n```bash\n# We assume you have obtained access to use LLaMA.\nbase_model=project-baize/baize-v2-7b\npython app.py $base_model None\n```\n#### GPU VRAM Requirements\n|           | Inference (without int8) |\n|-----------|--------------------------|\n| Baize-7B  | 16GB                     |\n| Baize-13B | 28GB                     |\n| Baize-30B | 67GB                     |\n\nIf you have a GPU with smaller VRAM, you can do inference with `int8`, by passing the 8bit argument:\n\n```bash\npython app.py $base_model $lora_model 8bit\n```\n\n## How to Reproduce\n\n### Setup\n\n1. Install dependencies\n\n```bash\npip install -r requirements.txt\n```\n\n2. If `bitsandbytes` doesn't work, [install it from source](https://github.com/TimDettmers/bitsandbytes/blob/main/compile_from_source.md). Windows users can follow [these instructions](https://github.com/tloen/alpaca-lora/issues/17).\n\n### Data Collecting\n\nYou can use our [released data](data) or collect the data from ChatGPT using the following command:\n\n```bash\nnum_process=10 # The number of processes to collect data\nmax_total_tokens=500000 # Set maximum numbers of tokens to collect data\napi_key=xxxxxxxxxxxxxxxxx # Set your openai api key\nfor ((i=0; i\u003c$num_process; i++))\ndo\n    python collect.py $api_key $max_total_tokens $i $num_process stackoverflow \u0026\n    python collect.py $api_key $max_total_tokens $i $num_process quora \u0026\n    python collect.py $api_key $max_total_tokens $i $num_process medical \u0026\ndone\n```\n\nAfter collecting data, you use the following command to preprocess data:\n\n```bash\npython preprocess.py stackoverflow\npython preprocess.py quora\npython preprocess.py medical\n```\n\n### Use your own data\n\nIf there's a specific dataset you want to use as seeds for ChatGPT self-chatting, you can simply modify `collect.py` to load your own data. \n\n### Training\n\nThe fine-tuning code is designed to run on an A100-80G GPU. The `finetune.py` script accepts three parameters: foundation model size (i.e., 7B, 13B, or 30B), batch size, learning rate and datasets. Note the total batch size is fixed to 64 (can be modified [here](https://github.com/project-baize/baize/blob/cbcf39902fcdfab8d935b7ea771a4e7d452a1be0/finetune.py#L24)) and the batch size here is the per device batch size before gradient accumulation. Set it to a smaller value if you are training on a GPU with smaller VRAM.\n\n```bash\n# For the 7B model (takes about 9 hours)\npython finetune.py 7b 32 0.0002 alpaca,stackoverflow,quora\n\n# For the 13B model (takes about 16 hours)\npython finetune.py 13b 16 0.0001 alpaca,stackoverflow,quora\n\n# For the 30B model (takes about 36 hours)\npython finetune.py 30b 8 0.00005 alpaca,stackoverflow,quora\n```\n#### GPU VRAM Consumption\nWith the settings **above**:\n\n|           | Training (with int8) |\n|-----------|----------------------|\n| Baize-7B  | 26GB                 |\n| Baize-13B | 25GB                 |\n| Baize-30B | 42GB                 |\n\nGot a question? See [this issue](https://github.com/project-baize/baize-chatbot/issues/26).\n\n### Merge LoRA into LLaMA\nNow you can easily merge the trained LoRA weights into a LLaMA model so you can use it with everything that supports standard Hugging Face API!\n\nHere's an example for merging `baize-lora-7B` into LLaMA-7B.\n```bash\npython merge_lora.py \\\n--base huggyllama/llama-7b \\\n--target ~/model_weights/baize-7b \\\n--lora project-baize/baize-lora-7B\n```\n\n## Citation\n```bibtex\n@article{xu2023baize,\n  title={Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data},\n  author={Xu, Canwen and Guo, Daya and Duan, Nan and McAuley, Julian},\n  journal={arXiv preprint arXiv:2304.01196},\n  year={2023}\n}\n```\n\u003chr\u003e\n\n[![Share to Community](https://huggingface.co/datasets/huggingface/badges/raw/main/powered-by-huggingface-light.svg)](https://huggingface.co)\n","funding_links":[],"categories":["Statistics","Python","Open Source LLM","Chatbots","LLM-List","A01_文本生成_文本对话","Other","大语言模型LLMs","Repos","[TavernAI/TavernAI](https://github.com/TavernAI/TavernAI)"],"sub_categories":["Open-LLM","大语言对话模型及数据","Other sdk/libraries","数据"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fproject-baize%2Fbaize-chatbot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fproject-baize%2Fbaize-chatbot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fproject-baize%2Fbaize-chatbot/lists"}