https://github.com/ezforever/llama.cpp-static
Static builds of llama.cpp (Currently only amd64 server builds are available)
https://github.com/ezforever/llama.cpp-static
llama llama-cpp llamacpp localai self-hosted
Last synced: 3 months ago
JSON representation
Static builds of llama.cpp (Currently only amd64 server builds are available)
- Host: GitHub
- URL: https://github.com/ezforever/llama.cpp-static
- Owner: EZForever
- License: mit
- Created: 2024-03-26T03:10:16.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2024-09-18T04:14:39.000Z (9 months ago)
- Last Synced: 2025-04-08T08:44:56.117Z (3 months ago)
- Topics: llama, llama-cpp, llamacpp, localai, self-hosted
- Language: Dockerfile
- Homepage:
- Size: 50.8 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# llama.cpp-static
Your daily, minimal build of llama.cpp. [Also available on Docker Hub.](https://hub.docker.com/r/ezforever/llama.cpp-static)
Source code:
Built from:
## Usage
Please refer to [llama.cpp docker guide](https://github.com/ggerganov/llama.cpp/blob/master/README.md#docker) and [server README](https://github.com/ggerganov/llama.cpp/tree/master/examples/server).
## Tag format
**tl;dr:** Use `server-ssl-avx2` if you don't know what you're doing.
Server images are tagged in the format of `server--`.
`` is one of the following:
- `nossl`: Minimal build with no SSL/TLS capability.
- `ssl`: Built with OpenSSL (`LLAMA_SERVER_SSL=ON`), thus supports `--ssl-key-file` and `--ssl-cert-file`.
`` is one of the following:
- `noavx`: All AVX-related optimizations are disabled. Do not use this build unless you are working around some known bug, or running LLMs on a 10-year-old potato.
- `avx`: (Only) AVX instruction set is enabled. Might be useful if you are using some old CPUs that don't support AVX2.
- `avx2`: AVX2 instruction set is enabled. This build should support most modern/recent CPUs with reasonable performance.
- `avx512`: AVX512 base instruction set is enabled. Currently only some high-end or server-grade CPUs support this instruction set, so check your hardware specs before using this build.
- `oneapi`: Experimental build with the Intel oneAPI compiler, inspired by ggerganov/llama.cpp#5067. Offers a ~30% speed increase (~20tok/s vs ~15tok/s) in prompt processing on my machine compared to `avx2` builds. Not updated daily.
RPC server images are tagged in the format of `rpc-server--`. Refer to [rpc README](https://github.com/ggerganov/llama.cpp/tree/master/examples/rpc) for detailed information.