https://github.com/ggml-org/ggml
Tensor library for machine learning
https://github.com/ggml-org/ggml
automatic-differentiation large-language-models machine-learning tensor-algebra
Last synced: 7 days ago
JSON representation
Tensor library for machine learning
- Host: GitHub
- URL: https://github.com/ggml-org/ggml
- Owner: ggml-org
- License: mit
- Created: 2022-09-18T17:07:19.000Z (over 3 years ago)
- Default Branch: master
- Last Pushed: 2026-05-22T14:54:07.000Z (10 days ago)
- Last Synced: 2026-05-22T19:54:39.785Z (10 days ago)
- Topics: automatic-differentiation, large-language-models, machine-learning, tensor-algebra
- Language: C++
- Homepage:
- Size: 28.7 MB
- Stars: 14,678
- Watchers: 148
- Forks: 1,627
- Open Issues: 321
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Authors: AUTHORS
Awesome Lists containing this project
- my-awesome - ggml-org/ggml - differentiation,large-language-models,machine-learning,tensor-algebra pushed_at:2026-05 star:14.6k fork:1.6k Tensor library for machine learning (C++)
- awesome-production-machine-learning - GGML - org/ggml.svg?style=social) - GGML is a high-performance, tensor library for machine learning that enables efficient inference on CPUs, particularly optimized for large language models. (Model Storage Optimisation)
- AiTreasureBox - ggml-org/ggml - 11-15_13558_0](https://img.shields.io/github/stars/ggml-org/ggml.svg)|Tensor library for machine learning| (Repos)
- my-awesome-github-stars - ggml-org/ggml - Tensor library for machine learning (C++)
- awesome-opensource-ai - GGML - Tensor library for machine learning. The foundational C/C++ library powering llama.cpp and many on-device inference engines. MIT licensed.  (1. Core Frameworks & Libraries)
- awesome-2026-ai-machine-learning-1000projects - ggml
- awesome-open-source-ai-tools - ggml-org/ggml - Tensor library for machine learning (Coding & Development)
README
# ggml
[Manifesto](https://github.com/ggerganov/llama.cpp/discussions/205)
Tensor library for machine learning
***Note that this project is under active development. \
Some of the development is currently happening in the [llama.cpp](https://github.com/ggerganov/llama.cpp) and [whisper.cpp](https://github.com/ggerganov/whisper.cpp) repos***
## Features
- Low-level cross-platform implementation
- Integer quantization support
- Broad hardware support
- Automatic differentiation
- ADAM and L-BFGS optimizers
- No third-party dependencies
- Zero memory allocations during runtime
## Build
```bash
git clone https://github.com/ggml-org/ggml
cd ggml
# install python dependencies in a virtual environment
python3.10 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
# build the examples
mkdir build && cd build
cmake ..
cmake --build . --config Release -j 8
```
## GPT inference (example)
```bash
# run the GPT-2 small 117M model
../examples/gpt-2/download-ggml-model.sh 117M
./bin/gpt-2-backend -m models/gpt-2-117M/ggml-model.bin -p "This is an example"
```
For more information, checkout the corresponding programs in the [examples](examples) folder.
## Resources
- [Introduction to ggml](https://huggingface.co/blog/introduction-to-ggml)
- [The GGUF file format](https://github.com/ggerganov/ggml/blob/master/docs/gguf.md)