https://github.com/KerfuffleV2/rusty-ggml
GGML bindings that aim to be idiomatic Rust rather than directly corresponding to the C/C++ interface
https://github.com/KerfuffleV2/rusty-ggml
Last synced: 4 months ago
JSON representation
GGML bindings that aim to be idiomatic Rust rather than directly corresponding to the C/C++ interface
- Host: GitHub
- URL: https://github.com/KerfuffleV2/rusty-ggml
- Owner: KerfuffleV2
- License: mit
- Created: 2023-04-22T17:32:15.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2023-09-25T07:30:17.000Z (over 1 year ago)
- Last Synced: 2024-09-25T16:09:35.443Z (7 months ago)
- Language: Rust
- Size: 114 KB
- Stars: 19
- Watchers: 1
- Forks: 3
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-rust-list - KerfuffleV2/rusty-ggml - ggml?style=social"/> : GGML bindings that aim to be idiomatic Rust rather than directly corresponding to the C/C++ interface. (Machine Learning)
- awesome-rust-list - KerfuffleV2/rusty-ggml - ggml?style=social"/> : GGML bindings that aim to be idiomatic Rust rather than directly corresponding to the C/C++ interface. (Machine Learning)
README
# rusty-ggml
GGML bindings that aim to be idiomatic Rust rather than directly corresponding to the C/C++ interface.
## GG-what?
See:
1. https://github.com/ggerganov/ggml/
2. https://github.com/KerfuffleV2/ggml-sys-bleedingedge ( https://crates.io/crates/ggml-sys-bleedingedge )
## WIP
Not suitable for general use. Consider this to be pre-alpha code.
**`v0.0.8` Warning**: Keeping this in sync with recent GGML changes has lagged. It compiles and seems to work but there might be weird stuff I haven't caught.
**Note**: There are special considerations when using GPU features like `cublas`, `hipblas`. See the `ggml-sys-bleedingedge` repo or crate documentation for more information
Example usage: https://github.com/KerfuffleV2/smolrsrwkv/blob/189915ec68b28d057b440f75803d3d056e150733/smolrwkv/src/ggml/graph.rs
## Related
For your token sampling needs see https://github.com/KerfuffleV2/llm-samplers ( https://crates.io/crates/llm-samplers )