{"id":13754183,"url":"https://github.com/soulteary/docker-llama2-chat","last_synced_at":"2025-04-05T05:08:59.443Z","repository":{"id":182340780,"uuid":"668338912","full_name":"soulteary/docker-llama2-chat","owner":"soulteary","description":"Play LLaMA2 (official / 中文版 / INT4 / llama2.cpp) Together! ONLY 3 STEPS! ( non GPU / 5GB vRAM / 8~14GB vRAM)","archived":false,"fork":false,"pushed_at":"2023-08-14T09:21:22.000Z","size":8961,"stargazers_count":541,"open_issues_count":11,"forks_count":86,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-28T16:04:55.192Z","etag":null,"topics":["llama","llama2","llama2-docker","llama2-playground","llm"],"latest_commit_sha":null,"homepage":"https://www.zhihu.com/people/soulteary/posts","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/soulteary.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-19T15:10:44.000Z","updated_at":"2025-03-14T04:52:43.000Z","dependencies_parsed_at":null,"dependency_job_id":"591cd04c-03d6-451c-8efb-aaafdf0af887","html_url":"https://github.com/soulteary/docker-llama2-chat","commit_stats":{"total_commits":36,"total_committers":2,"mean_commits":18.0,"dds":0.02777777777777779,"last_synced_commit":"4bc43122cfe43b10c7f008309fcd1772922f3204"},"previous_names":["soulteary/docker-llama2-chat"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soulteary%2Fdocker-llama2-chat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soulteary%2Fdocker-llama2-chat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soulteary%2Fdocker-llama2-chat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soulteary%2Fdocker-llama2-chat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/soulteary","download_url":"https://codeload.github.com/soulteary/docker-llama2-chat/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247289429,"owners_count":20914464,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["llama","llama2","llama2-docker","llama2-playground","llm"],"created_at":"2024-08-03T09:01:47.921Z","updated_at":"2025-04-05T05:08:59.424Z","avatar_url":"https://github.com/soulteary.png","language":"Python","funding_links":[],"categories":["A01_文本生成_文本对话","GitHub projects"],"sub_categories":["大语言对话模型及数据"],"readme":"# Docker LLaMA2 Chat / 羊驼二代\n\n\u003cp style=\"text-align: center;\"\u003e\n  \u003ca href=\"README.md\"  target=\"_blank\"\u003e中文文档\u003c/a\u003e | \u003ca href=\"README_EN.md\"\u003eENGLISH\u003c/a\u003e\n\u003c/p\u003e\n\n[![](https://img.shields.io/badge/LLaMA2-Official_7B_/_13B-blue)](https://huggingface.co/meta-llama) [![](https://img.shields.io/badge/LLaMA2-Chinese_7B-blue)](https://huggingface.co/soulteary/Chinese-Llama-2-7b-4bit) [![](https://img.shields.io/badge/LLaMA2-Chinese_GGMLQ4-blue)](https://huggingface.co/soulteary/Chinese-Llama-2-7b-ggml-q4) [![](https://img.shields.io/badge/License-Apache_v2-blue)](https://github.com/soulteary/docker-llama2-chat/blob/main/LICENSE)\n\n\u003cimg src=\".github/llama2.jpg\" width=\"40%\"\u003e\n\n三步上手 LLaMA2，一起玩！相关博客教程已更新，**同样欢迎“一键三连”** 🌟🌟🌟。\n\n\u003e 使用 Docker 快速上手，本地部署 7B 或 13B 官方模型，或者 7B 中文模型。\n\n### 博客教程\n\n| 类型 | 显存需求 | 特点 | 教程地址 | 教程时间 |\n| --- | --- | --- | --- | --- |\n| 官方版（英文） | 8~14GB | 原汁原味 | [使用 Docker 快速上手官方版 LLaMA2 开源大模型](https://soulteary.com/2023/07/21/use-docker-to-quickly-get-started-with-the-official-version-of-llama2-open-source-large-model.html) | 2023.07.21 |\n| LinkSoul 中文版（双语）| 8~14GB | 支持中文 | [使用 Docker 快速上手中文版 LLaMA2 开源大模型](https://soulteary.com/2023/07/21/use-docker-to-quickly-get-started-with-the-chinese-version-of-llama2-open-source-large-model.html) | 2023.07.21 |\n| Transformers 量化（中文/官方） | 5GB | 加速推理、节约显存 | [使用 Transformers 量化 Meta AI LLaMA2 中文版大模型](https://soulteary.com/2023/07/22/quantizing-meta-ai-llama2-chinese-version-large-models-using-transformers.html) | 2023.07.22 |\n| GGML (Llama.cpp) 量化 （中文/官方）| 可以不需要显存 | CPU 推理 | [构建能够使用 CPU 运行的 MetaAI LLaMA2 中文大模型](https://soulteary.com/2023/07/23/build-llama2-chinese-large-model-that-can-run-on-cpu.html) | 2023.07.23 |\n\n\n你可以参考项目代码，举一反三，把模型跑起来，接入到你想玩的地方，包括并不局限于支持 LLaMA 1代的各种开源软件中。\n\n## 预览图\n\n![](.github/preview.png)\n\n![](.github/llama2-cn-4bit.jpg)\n\n![](.github/clip.gif)\n\n## 使用方法\n\n1. 一条命令，从项目中构建官方版（7B或13B）模型镜像，或中文版镜像（7B或INT4量化版）：\n\n```bash\n# 7B\nbash scripts/make-7b.sh\n\n# 或 13B\nbash scripts/make-13b.sh\n\n# 或 7B Chinese\nbash scripts/make-7b-cn.sh\n\n# 或 7B Chinese 4bit\nbash scripts/make-7b-cn-4bit.sh\n```\n\n2. 选择适合你的命令，从 HuggingFace 下载 LLaMA2 或中文模型：\n\n```bash\n# MetaAI LLaMA2 Models (10~14GB vRAM)\ngit clone https://huggingface.co/meta-llama/Llama-2-7b-chat-hf\ngit clone https://huggingface.co/meta-llama/Llama-2-13b-chat-hf\n\nmkdir meta-llama\nmv Llama-2-7b-chat-hf meta-llama/\nmv Llama-2-13b-chat-hf meta-llama/\n\n# 或 Chinese LLaMA2 (10~14GB vRAM)\ngit clone https://huggingface.co/LinkSoul/Chinese-Llama-2-7b\n\nmkdir LinkSoul\nmv Chinese-Llama-2-7b LinkSoul/\n\n# 或 Chinese LLaMA2 4BIT (5GB vRAM)\ngit clone https://huggingface.co/soulteary/Chinese-Llama-2-7b-4bit\n\nmkdir soulteary\nmv Chinese-Llama-2-7b-4bit soulteary/\n```\n\n将下载好的模型，保持在一个正确的目录结构中。\n\n```bash\ntree -L 2 meta-llama\nsoulteary\n└── ...\nLinkSoul\n└── ...\nmeta-llama\n├── Llama-2-13b-chat-hf\n│   ├── added_tokens.json\n│   ├── config.json\n│   ├── generation_config.json\n│   ├── LICENSE.txt\n│   ├── model-00001-of-00003.safetensors\n│   ├── model-00002-of-00003.safetensors\n│   ├── model-00003-of-00003.safetensors\n│   ├── model.safetensors.index.json\n│   ├── pytorch_model-00001-of-00003.bin\n│   ├── pytorch_model-00002-of-00003.bin\n│   ├── pytorch_model-00003-of-00003.bin\n│   ├── pytorch_model.bin.index.json\n│   ├── README.md\n│   ├── Responsible-Use-Guide.pdf\n│   ├── special_tokens_map.json\n│   ├── tokenizer_config.json\n│   ├── tokenizer.model\n│   └── USE_POLICY.md\n└── Llama-2-7b-chat-hf\n    ├── added_tokens.json\n    ├── config.json\n    ├── generation_config.json\n    ├── LICENSE.txt\n    ├── model-00001-of-00002.safetensors\n    ├── model-00002-of-00002.safetensors\n    ├── model.safetensors.index.json\n    ├── models--meta-llama--Llama-2-7b-chat-hf\n    ├── pytorch_model-00001-of-00003.bin\n    ├── pytorch_model-00002-of-00003.bin\n    ├── pytorch_model-00003-of-00003.bin\n    ├── pytorch_model.bin.index.json\n    ├── README.md\n    ├── special_tokens_map.json\n    ├── tokenizer_config.json\n    ├── tokenizer.json\n    ├── tokenizer.model\n    └── USE_POLICY.md\n```\n\n3. 选择使用下面的适合你的命令，一键运行 LLaMA2 模型应用：\n\n```bash\n# 7B\nbash scripts/run-7b.sh\n# 或 13B\nbash scripts/run-13b.sh\n# 或 Chinese 7B\nbash scripts/run-7b-cn.sh\n# 或 Chinese 7B 4BIT\nbash scripts/run-7b-cn-4bit.sh\n```\n\n模型运行之后，在浏览器中访问 `http://localhost7860` 或者 `http://你的IP地址:7860` 就可以开始玩了。\n\n## 相关项目\n\n- MetaAI LLaMA2: https://ai.meta.com/llama/ ❤️\n- Meta LLaMA2 7B Chat: https://huggingface.co/meta-llama/Llama-2-7b-chat\n- Meta LLaMA2 13B Chat: https://huggingface.co/meta-llama/Llama-2-13b-chat\n- Chinese LLaMA2 7B: https://huggingface.co/LinkSoul/Chinese-Llama-2-7b ❤️\n- Chinese LLaMA2 7B GGML q4: https://huggingface.co/soulteary/Chinese-Llama-2-7b-ggml-q4\n- LLaMA2 GGML Converter: https://hub.docker.com/r/soulteary/llama2\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsoulteary%2Fdocker-llama2-chat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsoulteary%2Fdocker-llama2-chat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsoulteary%2Fdocker-llama2-chat/lists"}