https://github.com/andyroiiid/unityllmdemo
A demo for running Llama 3.2 1B in Unity.
https://github.com/andyroiiid/unityllmdemo
llama3 llm unity
Last synced: about 2 months ago
JSON representation
A demo for running Llama 3.2 1B in Unity.
- Host: GitHub
- URL: https://github.com/andyroiiid/unityllmdemo
- Owner: andyroiiid
- License: gpl-3.0
- Created: 2024-12-02T02:07:39.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2024-12-02T02:40:30.000Z (over 1 year ago)
- Last Synced: 2025-03-24T16:11:48.962Z (over 1 year ago)
- Topics: llama3, llm, unity
- Language: C#
- Homepage:
- Size: 4.94 MB
- Stars: 3
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Unity LLM Demo
The project is in 6000.0.29f1 but should work under any recent Unity version.
## How to clone
```shell
git clone --recursive https://github.com/andyroiiid/UnityLLMDemo.git
```
Or if you've already cloned without `--recursive`:
```shell
git submodule update --init
```
## Dependencies & Licenses
- [ONNX Runtime](https://github.com/microsoft/onnxruntime) - [MIT License](https://github.com/microsoft/onnxruntime/blob/main/LICENSE)
- Assets/Microsoft.ML.OnnxRuntimeGenAI/onnxruntime.dll
- Assets/Microsoft.ML.OnnxRuntimeGenAI/onnxruntime_providers_shared.dll
- [ONNX Runtime generate() API](https://github.com/microsoft/onnxruntime-genai) - [MIT License](https://github.com/microsoft/onnxruntime-genai/blob/main/LICENSE)
- Assets/Microsoft.ML.OnnxRuntimeGenAI/*.cs
- Assets/Microsoft.ML.OnnxRuntimeGenAI/onnxruntime-genai.dll
- [Llama 3.2 ONNX models](https://huggingface.co/onnx-community/Llama-3.2-1B-Instruct-ONNX) - [LLAMA 3.2 Community License](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE)
- Assets/StreamingAssets/Llama-3.2-1B-Instruct-ONNX/*
## Code explanation
### Assets/Scripts/MyModel.cs
The model interface and chat history implementation.
You can remove `_history`-related code if you just need an one-off completion.
Check https://github.com/microsoft/onnxruntime-genai/tree/main/examples for more examples.
### Assets/Scripts/MyChat.cs
A (very) basic UI controller.
### Assets/Scripts/StringBuilderExtensions.cs
Just one `StringBuilder` helper function.
## License
**Here's a reminder that this repository is [GPL-licensed](LICENSE)**.
The code is super simple, and you can write your own in less than an hour - just don't copy paste directly from this
repo.