https://github.com/dimforge/slai

Slang cross-platform GPU inference
https://github.com/dimforge/slai

Last synced: 6 months ago
JSON representation

Slang cross-platform GPU inference

• Host: GitHub
• URL: https://github.com/dimforge/slai
• Owner: dimforge
• License: apache-2.0
• Created: 2025-06-14T13:12:51.000Z (9 months ago)
• Default Branch: main
• Last Pushed: 2025-09-11T15:14:28.000Z (6 months ago)
• Last Synced: 2025-09-11T17:24:09.183Z (6 months ago)
• Language: Rust
• Size: 613 KB
• Stars: 1
• Watchers: 0
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# slai − GPU local inference every platform



  





    

        

    



-----

**slai** is a set of [Rust](https://www.rust-lang.org/) libraries exposing [Slang](https://shader-slang.org/) shaders

and kernels for local Large Language Models (LLMs) inference on the GPU. It is cross-platform and runs on the web.

**slai** can be used as a rust library to assemble your own transformer from the provided operators (and write your

owns on top of it).

Aside from the library, two binary crates are provided:

- **slai-bench** is a basic benchmarking utility for measuring calculation times for matrix multiplication with various

  quantization formats.

- **slai-chat** is a basic chat GUI application for loading GGUF files and chat with the model. It can be run natively

  or on the browser. Check out its [README](./crates/slai-chat/README.md) for details on how to run it. You can run

  it from your browser with the [online demo](https://slmath.rs/demos/slai/index.html).

⚠️ **slai** is still under heavy development and might be lacking some important features. Contributions  are welcome!

----