https://github.com/tsjdev-apps/phi-3-mini-onnx-console
Simple .NET console application to run the ONNX version of Phi-3.
https://github.com/tsjdev-apps/phi-3-mini-onnx-console
Last synced: 5 months ago
JSON representation
Simple .NET console application to run the ONNX version of Phi-3.
- Host: GitHub
- URL: https://github.com/tsjdev-apps/phi-3-mini-onnx-console
- Owner: tsjdev-apps
- License: mit
- Created: 2024-06-23T16:53:48.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-06-29T17:27:08.000Z (almost 2 years ago)
- Last Synced: 2024-06-29T18:37:01.870Z (almost 2 years ago)
- Language: C#
- Size: 787 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Using Phi-3 Mini with ONNX as local Small Language Model
This repository demonstrates how to seamlessly integrate the Phi-3 Mini Model using ONNX in a .NET Console application. By following this guide, you will learn how to set up and use a local Small Language Model to simulate chat interactions.
## Getting Started
Make sure that Git LFS (Large File Storage) is installed by running the following command:
```bash
git lfs install
```
Now, we can clone the model from [Hugging Face](https://huggingface.co/microsoft/Phi-3-mini-128k-instruct).
```bash
git clone https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx
```
In the downloaded folder, you will find a directory named `cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4`. This directory contains all the necessary files. Be sure to note the path to this folder.
Now, let's run the application.
## Screenshots
First, you need to specify the path to the model folder.
Now, you can start chatting with your local Small Language Model.
## Blog Post
If you are more interested into details, please see the following post on [medium.com](https://www.medium.com):
- [Using Phi-3 Mini with ONNX as local Small Language Model](https://medium.com/medialesson/using-phi-3-mini-with-onnx-as-local-small-language-model-2466f559af73)