{"id":13441824,"url":"https://github.com/InternLM/xtuner","last_synced_at":"2025-03-20T13:30:59.471Z","repository":{"id":191515488,"uuid":"664913876","full_name":"InternLM/xtuner","owner":"InternLM","description":"An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)","archived":false,"fork":false,"pushed_at":"2025-03-17T19:51:01.000Z","size":2392,"stargazers_count":4387,"open_issues_count":246,"forks_count":331,"subscribers_count":36,"default_branch":"main","last_synced_at":"2025-03-17T22:11:26.212Z","etag":null,"topics":["agent","baichuan","chatbot","chatglm2","chatglm3","conversational-ai","internlm","large-language-models","llama2","llama3","llava","llm","llm-training","mixtral","msagent","peft","phi3","qwen","supervised-finetuning"],"latest_commit_sha":null,"homepage":"https://xtuner.readthedocs.io/zh-cn/latest/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/InternLM.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-11T03:18:13.000Z","updated_at":"2025-03-17T13:47:59.000Z","dependencies_parsed_at":"2023-08-30T08:28:31.060Z","dependency_job_id":"25ed7b69-68cc-4580-8e7b-00871103677b","html_url":"https://github.com/InternLM/xtuner","commit_stats":{"total_commits":330,"total_committers":33,"mean_commits":10.0,"dds":0.5151515151515151,"last_synced_commit":"90192ffe42612b0f88409432e7b4860294432bcc"},"previous_names":["internlm/xtuner"],"tags_count":25,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InternLM%2Fxtuner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InternLM%2Fxtuner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InternLM%2Fxtuner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InternLM%2Fxtuner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/InternLM","download_url":"https://codeload.github.com/InternLM/xtuner/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244277137,"owners_count":20427309,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent","baichuan","chatbot","chatglm2","chatglm3","conversational-ai","internlm","large-language-models","llama2","llama3","llava","llm","llm-training","mixtral","msagent","peft","phi3","qwen","supervised-finetuning"],"created_at":"2024-07-31T03:01:38.486Z","updated_at":"2025-03-20T13:30:59.457Z","avatar_url":"https://github.com/InternLM.png","language":"Python","readme":"\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/InternLM/lmdeploy/assets/36994684/0cf8d00f-e86b-40ba-9b54-dc8f1bc6c8d8\" width=\"600\"/\u003e\n  \u003cbr /\u003e\u003cbr /\u003e\n\n[![GitHub Repo stars](https://img.shields.io/github/stars/InternLM/xtuner?style=social)](https://github.com/InternLM/xtuner/stargazers)\n[![license](https://img.shields.io/github/license/InternLM/xtuner.svg)](https://github.com/InternLM/xtuner/blob/main/LICENSE)\n[![PyPI](https://img.shields.io/pypi/v/xtuner)](https://pypi.org/project/xtuner/)\n[![Downloads](https://static.pepy.tech/badge/xtuner)](https://pypi.org/project/xtuner/)\n[![issue resolution](https://img.shields.io/github/issues-closed-raw/InternLM/xtuner)](https://github.com/InternLM/xtuner/issues)\n[![open issues](https://img.shields.io/github/issues-raw/InternLM/xtuner)](https://github.com/InternLM/xtuner/issues)\n\n👋 join us on [![Static Badge](https://img.shields.io/badge/-grey?style=social\u0026logo=wechat\u0026label=WeChat)](https://cdn.vansin.top/internlm/xtuner.jpg)\n[![Static Badge](https://img.shields.io/badge/-grey?style=social\u0026logo=twitter\u0026label=Twitter)](https://twitter.com/intern_lm)\n[![Static Badge](https://img.shields.io/badge/-grey?style=social\u0026logo=discord\u0026label=Discord)](https://discord.gg/xa29JuW87d)\n\n🔍 Explore our models on\n[![Static Badge](https://img.shields.io/badge/-gery?style=social\u0026label=🤗%20Huggingface)](https://huggingface.co/xtuner)\n[![Static Badge](https://img.shields.io/badge/-gery?style=social\u0026label=🤖%20ModelScope)](https://www.modelscope.cn/organization/xtuner)\n[![Static Badge](https://img.shields.io/badge/-gery?style=social\u0026label=🧰%20OpenXLab)](https://openxlab.org.cn/usercenter/xtuner)\n[![Static Badge](https://img.shields.io/badge/-gery?style=social\u0026label=🧠%20WiseModel)](https://www.wisemodel.cn/organization/xtuner)\n\nEnglish | [简体中文](README_zh-CN.md)\n\n\u003c/div\u003e\n\n## 🚀 Speed Benchmark\n\n- Llama2 7B Training Speed\n\n\u003cdiv align=center\u003e\n  \u003cimg src=\"https://github.com/InternLM/xtuner/assets/41630003/9c9dfdf4-1efb-4daf-84bf-7c379ae40b8b\" style=\"width:80%\"\u003e\n\u003c/div\u003e\n\n- Llama2 70B Training Speed\n\n\u003cdiv align=center\u003e\n  \u003cimg src=\"https://github.com/InternLM/xtuner/assets/41630003/5ba973b8-8885-4b72-b51b-c69fa1583bdd\" style=\"width:80%\"\u003e\n\u003c/div\u003e\n\n## 🎉 News\n- **\\[2025/02\\]** Support [OREAL](https://github.com/InternLM/OREAL), a new RL method for math reasoning!\n- **\\[2025/01\\]** Support [InternLM3 8B Instruct](https://huggingface.co/internlm/internlm3-8b-instruct)!\n- **\\[2024/07\\]** Support [MiniCPM](xtuner/configs/minicpm/) models!\n- **\\[2024/07\\]** Support [DPO](https://github.com/InternLM/xtuner/tree/main/xtuner/configs/dpo), [ORPO](https://github.com/InternLM/xtuner/tree/main/xtuner/configs/orpo) and [Reward Model](https://github.com/InternLM/xtuner/tree/main/xtuner/configs/reward_model) training with packed data and sequence parallel! See [documents](https://xtuner.readthedocs.io/en/latest/dpo/overview.html) for more details.\n- **\\[2024/07\\]** Support [InternLM 2.5](xtuner/configs/internlm/internlm2_5_chat_7b/) models!\n- **\\[2024/06\\]** Support [DeepSeek V2](xtuner/configs/deepseek/deepseek_v2_chat/) models! **2x faster!**\n- **\\[2024/04\\]** [LLaVA-Phi-3-mini](https://huggingface.co/xtuner/llava-phi-3-mini-hf) is released! Click [here](xtuner/configs/llava/phi3_mini_4k_instruct_clip_vit_large_p14_336) for details!\n- **\\[2024/04\\]** [LLaVA-Llama-3-8B](https://huggingface.co/xtuner/llava-llama-3-8b) and [LLaVA-Llama-3-8B-v1.1](https://huggingface.co/xtuner/llava-llama-3-8b-v1_1) are released! Click [here](xtuner/configs/llava/llama3_8b_instruct_clip_vit_large_p14_336) for details!\n- **\\[2024/04\\]** Support [Llama 3](xtuner/configs/llama) models!\n- **\\[2024/04\\]** Support Sequence Parallel for enabling highly efficient and scalable LLM training with extremely long sequence lengths! \\[[Usage](https://github.com/InternLM/xtuner/blob/docs/docs/zh_cn/acceleration/train_extreme_long_sequence.rst)\\] \\[[Speed Benchmark](https://github.com/InternLM/xtuner/blob/docs/docs/zh_cn/acceleration/benchmark.rst)\\]\n- **\\[2024/02\\]** Support [Gemma](xtuner/configs/gemma) models!\n- **\\[2024/02\\]** Support [Qwen1.5](xtuner/configs/qwen/qwen1_5) models!\n- **\\[2024/01\\]** Support [InternLM2](xtuner/configs/internlm) models! The latest VLM [LLaVA-Internlm2-7B](https://huggingface.co/xtuner/llava-internlm2-7b) / [20B](https://huggingface.co/xtuner/llava-internlm2-20b) models are released, with impressive performance!\n- **\\[2024/01\\]** Support [DeepSeek-MoE](https://huggingface.co/deepseek-ai/deepseek-moe-16b-chat) models! 20GB GPU memory is enough for QLoRA fine-tuning, and 4x80GB for full-parameter fine-tuning. Click [here](xtuner/configs/deepseek/) for details!\n- **\\[2023/12\\]** 🔥 Support multi-modal VLM pretraining and fine-tuning with [LLaVA-v1.5](https://github.com/haotian-liu/LLaVA) architecture! Click [here](xtuner/configs/llava/README.md) for details!\n- **\\[2023/12\\]** 🔥 Support [Mixtral 8x7B](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) models! Click [here](xtuner/configs/mixtral/README.md) for details!\n- **\\[2023/11\\]** Support [ChatGLM3-6B](xtuner/configs/chatglm) model!\n- **\\[2023/10\\]** Support [MSAgent-Bench](https://modelscope.cn/datasets/damo/MSAgent-Bench) dataset, and the fine-tuned LLMs can be applied by [Lagent](https://github.com/InternLM/lagent)!\n- **\\[2023/10\\]** Optimize the data processing to accommodate `system` context. More information can be found on [Docs](docs/en/user_guides/dataset_format.md)!\n- **\\[2023/09\\]** Support [InternLM-20B](xtuner/configs/internlm) models!\n- **\\[2023/09\\]** Support [Baichuan2](xtuner/configs/baichuan) models!\n- **\\[2023/08\\]** XTuner is released, with multiple fine-tuned adapters on [Hugging Face](https://huggingface.co/xtuner).\n\n## 📖 Introduction\n\nXTuner is an efficient, flexible and full-featured toolkit for fine-tuning large models.\n\n**Efficient**\n\n- Support LLM, VLM pre-training / fine-tuning on almost all GPUs. XTuner is capable of fine-tuning 7B LLM on a single 8GB GPU, as well as multi-node fine-tuning of models exceeding 70B.\n- Automatically dispatch high-performance operators such as FlashAttention and Triton kernels to increase training throughput.\n- Compatible with [DeepSpeed](https://github.com/microsoft/DeepSpeed) 🚀, easily utilizing a variety of ZeRO optimization techniques.\n\n**Flexible**\n\n- Support various LLMs ([InternLM](https://huggingface.co/internlm), [Mixtral-8x7B](https://huggingface.co/mistralai), [Llama 2](https://huggingface.co/meta-llama), [ChatGLM](https://huggingface.co/THUDM), [Qwen](https://huggingface.co/Qwen), [Baichuan](https://huggingface.co/baichuan-inc), ...).\n- Support VLM ([LLaVA](https://github.com/haotian-liu/LLaVA)). The performance of [LLaVA-InternLM2-20B](https://huggingface.co/xtuner/llava-internlm2-20b) is outstanding.\n- Well-designed data pipeline, accommodating datasets in any format, including but not limited to open-source and custom formats.\n- Support various training algorithms ([QLoRA](http://arxiv.org/abs/2305.14314), [LoRA](http://arxiv.org/abs/2106.09685), full-parameter fune-tune), allowing users to choose the most suitable solution for their requirements.\n\n**Full-featured**\n\n- Support continuous pre-training, instruction fine-tuning, and agent fine-tuning.\n- Support chatting with large models with pre-defined templates.\n- The output models can seamlessly integrate with deployment and server toolkit ([LMDeploy](https://github.com/InternLM/lmdeploy)), and large-scale evaluation toolkit ([OpenCompass](https://github.com/open-compass/opencompass), [VLMEvalKit](https://github.com/open-compass/VLMEvalKit)).\n\n## 🔥 Supports\n\n\u003ctable\u003e\n\u003ctbody\u003e\n\u003ctr align=\"center\" valign=\"middle\"\u003e\n\u003ctd\u003e\n  \u003cb\u003eModels\u003c/b\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cb\u003eSFT Datasets\u003c/b\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cb\u003eData Pipelines\u003c/b\u003e\n\u003c/td\u003e\n \u003ctd\u003e\n  \u003cb\u003eAlgorithms\u003c/b\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr valign=\"top\"\u003e\n\u003ctd align=\"left\" valign=\"top\"\u003e\n\u003cul\u003e\n  \u003cli\u003e\u003ca href=\"https://huggingface.co/internlm\"\u003eInternLM2 / 2.5\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://huggingface.co/meta-llama\"\u003eLlama 2 / 3\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://huggingface.co/collections/microsoft/phi-3-6626e15e9585a200d2d761e3\"\u003ePhi-3\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://huggingface.co/THUDM/chatglm2-6b\"\u003eChatGLM2\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://huggingface.co/THUDM/chatglm3-6b\"\u003eChatGLM3\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://huggingface.co/Qwen/Qwen-7B\"\u003eQwen\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://huggingface.co/baichuan-inc/Baichuan2-7B-Base\"\u003eBaichuan2\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1\"\u003eMixtral\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://huggingface.co/deepseek-ai/DeepSeek-V2-Chat\"\u003eDeepSeek V2\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://huggingface.co/google\"\u003eGemma\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://huggingface.co/openbmb\"\u003eMiniCPM\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e...\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cul\u003e\n  \u003cli\u003e\u003ca href=\"https://modelscope.cn/datasets/damo/MSAgent-Bench\"\u003eMSAgent-Bench\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://huggingface.co/datasets/fnlp/moss-003-sft-data\"\u003eMOSS-003-SFT\u003c/a\u003e 🔧\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://huggingface.co/datasets/tatsu-lab/alpaca\"\u003eAlpaca en\u003c/a\u003e / \u003ca href=\"https://huggingface.co/datasets/silk-road/alpaca-data-gpt4-chinese\"\u003ezh\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://huggingface.co/datasets/WizardLM/WizardLM_evol_instruct_V2_196k\"\u003eWizardLM\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://huggingface.co/datasets/timdettmers/openassistant-guanaco\"\u003eoasst1\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://huggingface.co/datasets/garage-bAInd/Open-Platypus\"\u003eOpen-Platypus\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://huggingface.co/datasets/HuggingFaceH4/CodeAlpaca_20K\"\u003eCode Alpaca\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://huggingface.co/datasets/burkelibbey/colors\"\u003eColorist\u003c/a\u003e 🎨\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://github.com/WangRongsheng/ChatGenTitle\"\u003eArxiv GenTitle\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://github.com/LiuHC0428/LAW-GPT\"\u003eChinese Law\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://huggingface.co/datasets/Open-Orca/OpenOrca\"\u003eOpenOrca\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://huggingface.co/datasets/shibing624/medical\"\u003eMedical Dialogue\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e...\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cul\u003e\n  \u003cli\u003e\u003ca href=\"docs/zh_cn/user_guides/incremental_pretraining.md\"\u003eIncremental Pre-training\u003c/a\u003e \u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"docs/zh_cn/user_guides/single_turn_conversation.md\"\u003eSingle-turn Conversation SFT\u003c/a\u003e \u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"docs/zh_cn/user_guides/multi_turn_conversation.md\"\u003eMulti-turn Conversation SFT\u003c/a\u003e \u003c/li\u003e\n\u003c/ul\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cul\u003e\n  \u003cli\u003e\u003ca href=\"http://arxiv.org/abs/2305.14314\"\u003eQLoRA\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"http://arxiv.org/abs/2106.09685\"\u003eLoRA\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003eFull parameter fine-tune\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://arxiv.org/abs/2305.18290\"\u003eDPO\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://arxiv.org/abs/2403.07691\"\u003eORPO\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003eReward Model\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n## 🛠️ Quick Start\n\n### Installation\n\n- It is recommended to build a Python-3.10 virtual environment using conda\n\n  ```bash\n  conda create --name xtuner-env python=3.10 -y\n  conda activate xtuner-env\n  ```\n\n- Install XTuner via pip\n\n  ```shell\n  pip install -U xtuner\n  ```\n\n  or with DeepSpeed integration\n\n  ```shell\n  pip install -U 'xtuner[deepspeed]'\n  ```\n\n- Install XTuner from source\n\n  ```shell\n  git clone https://github.com/InternLM/xtuner.git\n  cd xtuner\n  pip install -e '.[all]'\n  ```\n\n### Fine-tune\n\nXTuner supports the efficient fine-tune (*e.g.*, QLoRA) for LLMs. Dataset prepare guides can be found on [dataset_prepare.md](./docs/en/user_guides/dataset_prepare.md).\n\n- **Step 0**, prepare the config. XTuner provides many ready-to-use configs and we can view all configs by\n\n  ```shell\n  xtuner list-cfg\n  ```\n\n  Or, if the provided configs cannot meet the requirements, please copy the provided config to the specified directory and make specific modifications by\n\n  ```shell\n  xtuner copy-cfg ${CONFIG_NAME} ${SAVE_PATH}\n  vi ${SAVE_PATH}/${CONFIG_NAME}_copy.py\n  ```\n\n- **Step 1**, start fine-tuning.\n\n  ```shell\n  xtuner train ${CONFIG_NAME_OR_PATH}\n  ```\n\n  For example, we can start the QLoRA fine-tuning of InternLM2.5-Chat-7B with oasst1 dataset by\n\n  ```shell\n  # On a single GPU\n  xtuner train internlm2_5_chat_7b_qlora_oasst1_e3 --deepspeed deepspeed_zero2\n  # On multiple GPUs\n  (DIST) NPROC_PER_NODE=${GPU_NUM} xtuner train internlm2_5_chat_7b_qlora_oasst1_e3 --deepspeed deepspeed_zero2\n  (SLURM) srun ${SRUN_ARGS} xtuner train internlm2_5_chat_7b_qlora_oasst1_e3 --launcher slurm --deepspeed deepspeed_zero2\n  ```\n\n  - `--deepspeed` means using [DeepSpeed](https://github.com/microsoft/DeepSpeed) 🚀 to optimize the training. XTuner comes with several integrated strategies including ZeRO-1, ZeRO-2, and ZeRO-3. If you wish to disable this feature, simply remove this argument.\n\n  - For more examples, please see [finetune.md](./docs/en/user_guides/finetune.md).\n\n- **Step 2**, convert the saved PTH model (if using DeepSpeed, it will be a directory) to Hugging Face model, by\n\n  ```shell\n  xtuner convert pth_to_hf ${CONFIG_NAME_OR_PATH} ${PTH} ${SAVE_PATH}\n  ```\n\n### Chat\n\nXTuner provides tools to chat with pretrained / fine-tuned LLMs.\n\n```shell\nxtuner chat ${NAME_OR_PATH_TO_LLM} --adapter {NAME_OR_PATH_TO_ADAPTER} [optional arguments]\n```\n\nFor example, we can start the chat with InternLM2.5-Chat-7B :\n\n```shell\nxtuner chat internlm/internlm2_5-chat-7b --prompt-template internlm2_chat\n```\n\nFor more examples, please see [chat.md](./docs/en/user_guides/chat.md).\n\n### Deployment\n\n- **Step 0**, merge the Hugging Face adapter to pretrained LLM, by\n\n  ```shell\n  xtuner convert merge \\\n      ${NAME_OR_PATH_TO_LLM} \\\n      ${NAME_OR_PATH_TO_ADAPTER} \\\n      ${SAVE_PATH} \\\n      --max-shard-size 2GB\n  ```\n\n- **Step 1**, deploy fine-tuned LLM with any other framework, such as [LMDeploy](https://github.com/InternLM/lmdeploy) 🚀.\n\n  ```shell\n  pip install lmdeploy\n  python -m lmdeploy.pytorch.chat ${NAME_OR_PATH_TO_LLM} \\\n      --max_new_tokens 256 \\\n      --temperture 0.8 \\\n      --top_p 0.95 \\\n      --seed 0\n  ```\n\n  🔥 Seeking efficient inference with less GPU memory? Try 4-bit quantization from [LMDeploy](https://github.com/InternLM/lmdeploy)! For more details, see [here](https://github.com/InternLM/lmdeploy/tree/main#quantization).\n\n### Evaluation\n\n- We recommend using [OpenCompass](https://github.com/InternLM/opencompass), a comprehensive and systematic LLM evaluation library, which currently supports 50+ datasets with about 300,000 questions.\n\n## 🤝 Contributing\n\nWe appreciate all contributions to XTuner. Please refer to [CONTRIBUTING.md](.github/CONTRIBUTING.md) for the contributing guideline.\n\n## 🎖️ Acknowledgement\n\n- [Llama 2](https://github.com/facebookresearch/llama)\n- [DeepSpeed](https://github.com/microsoft/DeepSpeed)\n- [QLoRA](https://github.com/artidoro/qlora)\n- [LMDeploy](https://github.com/InternLM/lmdeploy)\n- [LLaVA](https://github.com/haotian-liu/LLaVA)\n\n## 🖊️ Citation\n\n```bibtex\n@misc{2023xtuner,\n    title={XTuner: A Toolkit for Efficiently Fine-tuning LLM},\n    author={XTuner Contributors},\n    howpublished = {\\url{https://github.com/InternLM/xtuner}},\n    year={2023}\n}\n```\n\n## License\n\nThis project is released under the [Apache License 2.0](LICENSE). Please also adhere to the Licenses of models and datasets being used.\n","funding_links":[],"categories":["Python","A01_文本生成_文本对话","微调 Fine-Tuning","agent","Autonomous Research \u0026 Content Generation","LLM Training / Finetuning","reinforcement-learning","Repos","📋 Contents","4. Fine-Tuning","GitHub projects"],"sub_categories":["大语言对话模型及数据","Programming Languages","🛠️ 7. Training \u0026 Fine-tuning Ecosystem","Frameworks"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FInternLM%2Fxtuner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FInternLM%2Fxtuner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FInternLM%2Fxtuner/lists"}