https://github.com/chrishoy/aichat-example
AI chat client using Olamma LLM
https://github.com/chrishoy/aichat-example
Last synced: over 1 year ago
JSON representation
AI chat client using Olamma LLM
- Host: GitHub
- URL: https://github.com/chrishoy/aichat-example
- Owner: chrishoy
- License: mit
- Created: 2025-03-21T12:14:41.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2025-03-21T13:12:48.000Z (over 1 year ago)
- Last Synced: 2025-03-21T13:30:34.785Z (over 1 year ago)
- Language: C#
- Size: 54.7 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
Awesome Lists containing this project
README
# Ollama Chat Client Example
## Introduction
Simple .NET 9 chat client/api example that demonstrates how to use the `Ollama` Chat API.
### Getting Started
- `docker-compose up -d` to pull and start `Ollama` and `Redis` docker containers.
- `docker exec -it ollama ollama pull llama3` to pull the `llama3` model into the `Ollama` container (this will take a while).
- Use HTTP GET/POST examples in `Examples.http`, or use Postman.
- Switching between emulator and Ollama AI Chat Client is done in `appSettings.development.json`
```
"ChatSettings": {
"UseEmulator": true,
"EmulatorResponseTime": 10000 // milliseconds
}
```
### Things to Beware
- I am using `Redis` to store chat history, and this has been 'abused' to provide minimal orchestration of requests.
- The `ChatService` should really be implemented as a scalable background service. I am using `DotNet Channel` for background process management.
- Throttling/request scaling has not been implemented as I am using a non-distributed Chat API (`Ollama` in this case), so if you throw multiple requests at `Ollama` while it is still trying to figure things out, you will run into issues.
- `Redis` has been configured (in code) to invalidate conversations after 15 mins.
- Plenty more that can be done to make this production standard code.
- `ChatClientEmulator` is a simple IChatClient replacement purely there for development
### Other Useful Information
- `docker exec -it ollamma bash` to enter the container.`