https://github.com/santhsecurity/spill

Intelligent GPU memory tiering for local LLM inference — your model spills past VRAM, spill makes it fast
https://github.com/santhsecurity/spill

Last synced: 2 months ago
JSON representation

Intelligent GPU memory tiering for local LLM inference — your model spills past VRAM, spill makes it fast

• Host: GitHub
• URL: https://github.com/santhsecurity/spill
• Owner: santhsecurity
• Created: 2026-04-12T12:39:55.000Z (4 months ago)
• Default Branch: main
• Last Pushed: 2026-05-09T07:52:39.000Z (3 months ago)
• Last Synced: 2026-05-09T09:36:36.361Z (3 months ago)
• Language: Rust
• Homepage: https://santh.dev
• Size: 21.5 KB
• Stars: 0
• Watchers: 0
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • Security: SECURITY.md

Awesome Lists containing this project

README

          
# helix

Intelligent GPU memory tiering plugin for llama.cpp — learns access patterns, prefetches data, makes local LLM inference faster

Intelligent GPU memory tiering primitives for llama.cpp backends.

The crate models memory placement, access tracking, prediction, prefetch,

and profile persistence without requiring GPU hardware in tests.

## License

MIT OR Apache-2.0