https://github.com/johnclaw/gemma-2-2b-it.cs
gemma-2-2b-it int8 cpu inference in one file of pure C#
https://github.com/johnclaw/gemma-2-2b-it.cs
cpu-inference csharp gemma gemma2 gemma2-2b-it inference inference-engine int8 int8-inference int8-quantization llm llm-inference llm-serving llms quantization
Last synced: 3 months ago
JSON representation
gemma-2-2b-it int8 cpu inference in one file of pure C#
- Host: GitHub
- URL: https://github.com/johnclaw/gemma-2-2b-it.cs
- Owner: JohnClaw
- Created: 2025-06-14T20:49:49.000Z (4 months ago)
- Default Branch: main
- Last Pushed: 2025-06-14T22:43:41.000Z (4 months ago)
- Last Synced: 2025-06-14T23:27:06.756Z (4 months ago)
- Topics: cpu-inference, csharp, gemma, gemma2, gemma2-2b-it, inference, inference-engine, int8, int8-inference, int8-quantization, llm, llm-inference, llm-serving, llms, quantization
- Language: C#
- Homepage:
- Size: 16.6 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# gemma-2-2b-it.cs
gemma-2-2b-it int8 cpu inference in one file of pure C#
I used Gemini 2.5 Pro Preview to port this Rust repo: https://github.com/samuel-vitorino/lm.rs/tree/24c45b01419e1d7989dfc93ac6209b13e6edbbfa to C#. So all credits go to creator of original repo and to Gemini 2.5 Pro Preview. You should use python scripts from original repo to create llm and tokenizer files.Then move them to the folder where you placed app's release files.
You can build project by double clicking on build.bat
To run project you should double click on launch.bat. By default it supposed that you called llm file "model.bin". Otherwise you will need to edit launch.but accordingly.