{"id":26728324,"url":"https://github.com/maaaxinfinity/ktrun","last_synced_at":"2025-04-14T08:54:32.130Z","repository":{"id":282682769,"uuid":"949344908","full_name":"maaaxinfinity/ktrun","owner":"maaaxinfinity","description":"KTransformers 一键部署脚本","archived":false,"fork":false,"pushed_at":"2025-04-13T16:18:41.000Z","size":134,"stargazers_count":35,"open_issues_count":8,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-13T16:25:55.664Z","etag":null,"topics":["bash-script"],"latest_commit_sha":null,"homepage":"https://bashllm.com","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/maaaxinfinity.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-03-16T08:38:28.000Z","updated_at":"2025-04-13T06:18:30.000Z","dependencies_parsed_at":"2025-04-05T22:19:41.399Z","dependency_job_id":"45908179-ae91-4f3f-820d-747b294cbe6b","html_url":"https://github.com/maaaxinfinity/ktrun","commit_stats":null,"previous_names":["maaaxinfinity/ktrun"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maaaxinfinity%2Fktrun","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maaaxinfinity%2Fktrun/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maaaxinfinity%2Fktrun/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maaaxinfinity%2Fktrun/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/maaaxinfinity","download_url":"https://codeload.github.com/maaaxinfinity/ktrun/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248852109,"owners_count":21171839,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bash-script"],"created_at":"2025-03-27T22:23:04.843Z","updated_at":"2025-04-14T08:54:32.124Z","avatar_url":"https://github.com/maaaxinfinity.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# KTransformers 一键部署及启动脚本\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/CUDA-支持-brightgreen\" alt=\"CUDA支持\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/PyTorch-兼容-blue\" alt=\"PyTorch兼容\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/平台-Linux-orange\" alt=\"平台\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/语言-Bash-yellow\" alt=\"语言\"\u003e\n\u003c/div\u003e\n\n## 💡QQ交流群 \n\n- [点击加入群聊](https://qm.qq.com/q/zBzV5CkDSM)  \n- QQ群号 1028429001\n\n## 🤗 最佳实践\n\n  目前已经测试成功一键部署的环境：\n\n- EPYC 9334QS*2 + NVIDIA 4090\n- EPYC 9375F + NVIDIA 4070tis\n- EPYC 9965*2 + NVIDIA 4090\n- EPYC 7532 + NVIDIA 3070\n\n 一键部署并不是万能的，因为每个系统的环境都不一样，我们均在Cuda 12.4 下测试通过，其他版本可能存在兼容性问题，请根据实际情况进行测试。\n\n## 📋 简介\n\nKTransformers 是一个高性能的 Transformer 模型加速框架，专为混合推理优化设计。本项目提供了完整的自动化安装和启动流程，帮助您快速部署 KTransformers 环境并启动模型，包括所有必要的依赖项和配置。\n\n## ✨ 功能特点\n\n- **全自动安装**：一键完成从环境检测到依赖安装的全过程\n- **智能检测**：自动检测 CUDA、GPU 驱动和系统环境\n- **多站点支持**：智能选择最佳 GitHub 镜像站点，加速下载\n- **环境兼容性**：支持 sudo 和非 sudo 环境，保留用户环境变量\n- **一键启动**：简化的模型启动脚本，支持丰富的参数配置\n- **详细日志**：提供完整的安装和运行日志，便于故障排除\n- **调试模式**：提供详细的调试信息和系统环境报告\n- **自动修复**：遇到常见问题时尝试自动修复\n\n## 🖥️ 系统要求\n\n- **操作系统**：Ubuntu 或其他基于 Debian 的 Linux 发行版\n- **硬件**：\n  - NVIDIA GPU (推荐 16GB+ 显存)\n  - 384GB+ 系统内存\n- **软件**：\n  - NVIDIA 驱动 (与 CUDA 兼容的版本)\n  - Git\n  - Python 3.11+\n  - **Cuda 12.4**\n\n## 🚀 安装步骤\n\n### 1. 获取安装脚本\n\n#### 方式〇：直接git clone吧（推荐）\n \n ```bash\n git clone https://github.com/maaaxinfinity/ktrun.git (github)\n git clone https://gitcode.com/Limitee/ktrun.git (gitcode)\n cd ./ktrun\n sudo bash run.sh\n ```\n\n#### 方式一：从GitHub下载\n\n```bash\nwget https://raw.githubusercontent.com/maaaxinfinity/ktrun/refs/heads/main/run.sh\nchmod +x run.sh\n```\n\n#### 方式二：从国内镜像仓库下载\n\n```bash\nwget https://gitcode.com/Limitee/ktrun/raw/main/run.sh\nchmod +x run.sh\n```\n\n### 2. 运行安装脚本\n\n```bash\nsudo ./run.sh\n```\n\n#### 脚本参数选项\n\n```\n选项:\n  -d, --debug           启用调试模式，记录详细日志\n  -f, --fast            快速模式，使用默认配置无需用户确认\n  -g, --git-debug       启用git详细日志输出（需要与-d一起使用）\n  -h, --help            显示帮助信息\n```\n\n\u003e **注意**：脚本开始时会提供选项让您配置安装目录、Conda环境名称、是否启用NUMA环境变量、编译线程数等参数。\n\n### 3. 安装过程\n\n安装过程包括以下主要步骤：\n\n1. 检测系统环境和必要工具\n2. 安装缺失的依赖项\n3. 测试 GitHub 连通性并选择最佳镜像\n4. 克隆 KTransformers 代码库\n5. 安装/配置 Miniconda 环境\n6. 创建并激活 conda 环境\n7. 初始化和更新 git 子模块\n8. 安装 NUMA 支持\n9. 安装 GPU 版本的 PyTorch\n10. 安装 Flash Attention 和 FlashInfer 加速库\n11. 编译和安装 KTransformers\n12. 更新系统库\n13. 验证安装\n\n## 🚀 启动模型\n\n### 1. 获取启动脚本\n\n#### 方式〇：直接git clone吧\n\n```bash\ngit clone https://github.com/maaaxinfinity/ktrun.git (github)\ngit clone https://gitcode.com/Limitee/ktrun.git (gitcode)\n\nsudo bash run.sh\n```\n\n#### 方式一：从GitHub下载\n\n```bash\nwget https://raw.githubusercontent.com/maaaxinfinity/ktrun/refs/heads/main/run.sh\nchmod +x start.sh\n```\n\n#### 方式二：从国内镜像仓库下载（推荐）\n\n```bash\nwget https://gitcode.com/Limitee/ktrun/raw/main/start.sh\nchmod +x start.sh\n```\n\n### 2. 运行启动脚本\n\n```bash\n./start.sh\n```\n\n### 3. 启动脚本参数\n\n```\n选项:\n  -h, --help                显示帮助信息\n  -m, --model_path PATH     设置模型路径 (默认: deepseek-ai/DeepSeek-R1)\n  -g, --gguf_path PATH      设置GGUF路径 (默认: ~/model)\n  -c, --cpu_infer NUM       设置CPU推理数量 (默认: 380)\n  -t, --max_new_tokens NUM  设置最大新token数 (默认: 16384)\n  -l, --cache_lens NUM      设置缓存长度 (默认: 8192)\n  -o, --optimize_config_path PATH  设置优化配置文件 (默认: DeepSeek-V3-Chat.yaml)\n```\n\n### 4. 示例用法\n\n```bash\n# 使用默认参数启动\n./start.sh\n\n# 指定模型路径和其他参数\n./start.sh -m custom-model -g /path/to/model -c 16 -t 8192 -l 2048\n\n# 使用特定的优化配置文件\n./start.sh -o DeepSeek-V3-Chat.yaml\n```\n\n## 📝 使用方法\n\n### 安装完成后激活环境\n\n安装完成后，您需要激活创建的 conda 环境：\n\n```bash\nsource /path/to/activate_env.sh\n```\n\n\u003e 脚本会在安装结束时显示确切的激活命令路径。\n\n### 验证安装\n\n激活环境后，您可以验证安装：\n\n```bash\npython -c \"import ktransformers; print(ktransformers.__version__)\"\npython -c \"import torch; print('CUDA可用:', torch.cuda.is_available())\"\n```\n\n## ❓ 常见问题\n\n### Q: 如何选择安装目录？\n\n**A**: 脚本会提示您输入安装目录，默认为当前目录下的 `ktransformers` 文件夹。\n\n### Q: 如何选择 conda 环境名称？\n\n**A**: 脚本会提示您输入环境名称，默认为 `ktrans_main`。\n\n### Q: 如何使用国内镜像加速安装？\n\n**A**: 在安装配置中选择\"是否使用国内代理和镜像站点?\"选项为\"是\"。\n\n### Q: 安装过程中断了怎么办？\n\n**A**: 您可以重新运行脚本，它会检测已完成的步骤并继续安装。\n\n### Q: 如何查看安装日志？\n\n**A**: 安装日志保存在当前目录，格式为 `ktransformers_install_日期时间.log`。\n\n### Q: 如何更改模型的默认参数？\n\n**A**: 可以通过start.sh脚本的命令行参数进行设置，如`-m`指定模型路径，`-c`指定CPU推理数量等。\n\n## 🛠️ 故障排除\n\n### CUDA 相关问题\n\n如果遇到 CUDA 相关问题：\n\n```bash\n# 检查 NVIDIA 驱动\nnvidia-smi\n\n# 检查 CUDA 版本\nnvcc --version\n\n# 检查 PyTorch CUDA 支持\npython -c \"import torch; print(torch.cuda.is_available())\"\n```\n\n### 依赖项问题\n\n如果某些依赖项安装失败：\n\n```bash\n# 手动安装 flashinfer\npip install flashinfer-python -f https://flashinfer.ai/whl/cu124/torch2.6\n\n# 手动安装 PyTorch\npip install torch torchvision torchaudio -f https://download.pytorch.org/whl/cu124\n```\n\n### 环境激活问题\n\n如果环境激活失败：\n\n```bash\n# 重新初始化 conda\nconda init bash\nsource ~/.bashrc\n\n# 手动激活环境\nconda activate ktrans_main\n```\n\n---\n\n## Star History\n\n[![Star History Chart](https://api.star-history.com/svg?repos=maaaxinfinity/ktrun\u0026type=Date)](https://www.star-history.com/#maaaxinfinity/ktrun\u0026Date)\n\n\u003cdiv align=\"center\"\u003e\n  \u003cp\u003e© 2024-2025 Limitee. 保留所有权利。\u003c/p\u003e\n  \u003cp\u003e如有问题，请提交 \u003ca href=\"https://github.com/kvcache-ai/ktransformers/issues\"\u003eGitHub Issue\u003c/a\u003e\u003c/p\u003e\n  \u003cp\u003e国内镜像仓库: \u003ca href=\"https://gitcode.com/Limitee/ktrun\"\u003ehttps://gitcode.com/Limitee/ktrun\u003c/a\u003e\u003c/p\u003e\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaaaxinfinity%2Fktrun","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaaaxinfinity%2Fktrun","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaaaxinfinity%2Fktrun/lists"}