https://github.com/abetlen/ggml-python
Python bindings for ggml
https://github.com/abetlen/ggml-python
Last synced: 2 days ago
JSON representation
Python bindings for ggml
- Host: GitHub
- URL: https://github.com/abetlen/ggml-python
- Owner: abetlen
- License: mit
- Created: 2023-05-14T15:46:05.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-09-02T22:43:58.000Z (6 months ago)
- Last Synced: 2025-02-12T11:06:53.928Z (9 days ago)
- Language: Python
- Homepage: https://ggml-python.readthedocs.io
- Size: 1.24 MB
- Stars: 136
- Watchers: 4
- Forks: 11
- Open Issues: 20
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
- awesome-llm-and-aigc - ggml-python - python?style=social"/> : Python bindings for ggml. [ggml-python.readthedocs.io](https://ggml-python.readthedocs.io/) (Summary)
- awesome-llm-and-aigc - ggml-python - python?style=social"/> : Python bindings for ggml. [ggml-python.readthedocs.io](https://ggml-python.readthedocs.io/) (Summary)
README
# Python bindings for [`ggml`](https://github.com/ggerganov/ggml)
[](https://ggml-python.readthedocs.io/en/latest/?badge=latest)
[](https://github.com/abetlen/ggml-python/actions/workflows/test.yaml)
[](https://pypi.org/project/ggml-python/)
[](https://pypi.org/project/ggml-python/)
[](https://pypi.org/project/ggml-python/)
[](https://pypi.org/project/ggml-python/)
Python bindings for the [`ggml`](https://github.com/ggerganov/ggml) tensor library for machine learning.
> ⚠️ Neither this project nor `ggml` currently guarantee backwards-compatibility, if you are using this library in other applications I strongly recommend pinning to specific releases in your `requirements.txt` file.
# Documentation
- [Getting Started](https://ggml-python.readthedocs.io/en/latest/)
- [API Reference](https://ggml-python.readthedocs.io/en/latest/api-reference/)
- [Examples](https://github.com/abetlen/ggml-python/tree/main/examples)
# Installation
Requirements
- Python 3.8+
- C compiler (gcc, clang, msvc, etc)
You can install `ggml-python` using `pip`:
```bash
pip install ggml-python
```
This will compile ggml using cmake which requires a c compiler installed on your system.
To build ggml with specific features (ie. OpenBLAS, GPU Support, etc) you can pass specific cmake options through the `cmake.args` pip install configuration setting. For example to install ggml-python with cuBLAS support you can run:
```bash
pip install --upgrade pip
pip install ggml-python --config-settings=cmake.args='-DGGML_CUDA=ON'
```
## Options
| Option | Description | Default |
| --- | --- | --- |
| `GGML_CUDA` | Enable cuBLAS support | `OFF` |
| `GGML_CLBLAST` | Enable CLBlast support | `OFF` |
| `GGML_OPENBLAS` | Enable OpenBLAS support | `OFF` |
| `GGML_METAL` | Enable Metal support | `OFF` |
| `GGML_RPC` | Enable RPC support | `OFF` |
# Usage
```python
import ggml
import ctypes
# Allocate a new context with 16 MB of memory
params = ggml.ggml_init_params(mem_size=16 * 1024 * 1024, mem_buffer=None)
ctx = ggml.ggml_init(params)
# Instantiate tensors
x = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)
a = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)
b = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)
# Use ggml operations to build a computational graph
x2 = ggml.ggml_mul(ctx, x, x)
f = ggml.ggml_add(ctx, ggml.ggml_mul(ctx, a, x2), b)
gf = ggml.ggml_new_graph(ctx)
ggml.ggml_build_forward_expand(gf, f)
# Set the input values
ggml.ggml_set_f32(x, 2.0)
ggml.ggml_set_f32(a, 3.0)
ggml.ggml_set_f32(b, 4.0)
# Compute the graph
ggml.ggml_graph_compute_with_ctx(ctx, gf, 1)
# Get the output value
output = ggml.ggml_get_f32_1d(f, 0)
assert output == 16.0
# Free the context
ggml.ggml_free(ctx)
```
# Troubleshooting
If you are having trouble installing `ggml-python` or activating specific features please try to install it with the `--verbose` and `--no-cache-dir` flags to get more information about any issues:
```bash
pip install ggml-python --verbose --no-cache-dir --force-reinstall --upgrade
```
# License
This project is licensed under the terms of the MIT license.