https://github.com/simonbogaerts/bogsi.ceridwen
A small experiment with docker, dev containers and a LLM. An experiment, but quiet useful. Ceridwen is a personal assistant that can be run on the GPU.
https://github.com/simonbogaerts/bogsi.ceridwen
devcontainers docker docker-compose llms ollama
Last synced: 5 months ago
JSON representation
A small experiment with docker, dev containers and a LLM. An experiment, but quiet useful. Ceridwen is a personal assistant that can be run on the GPU.
- Host: GitHub
- URL: https://github.com/simonbogaerts/bogsi.ceridwen
- Owner: simonbogaerts
- Created: 2024-08-22T13:04:51.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-08-23T11:27:12.000Z (almost 2 years ago)
- Last Synced: 2025-04-08T16:54:02.170Z (about 1 year ago)
- Topics: devcontainers, docker, docker-compose, llms, ollama
- Language: C#
- Homepage:
- Size: 17.6 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Ceridwen by BOGsi
Ceridwen, a figure from Welsh mythology, said to be the keeper of the cauldron of knowledge, mother of transformation and the white lady of inspiration and death.
# What is Ceridwen?
An simple application that will run a Large Language Model (LLM) on your local machine, utilizing your GPU (preferably).
The application itself is a basic dotnet console application that will pass on your questions to an Ollama model by Meta.
This was inteded as an experiment while learning docker/docker-compose/devcontainers. Not sure if further updates will be made to the repo.
# Prerequisites
* Docker
* WSL2+
If usisng the GPU:
* Nvidia GPU
* up-to-date GPU drivers
# How to run Ceridwen
Simply use the following command in the terminal (when in the main project folder):
```
docker compose up -d
```
Afterwards attach to the running console application and get on typing.
```
docker attach
```
To stop Ceridwen ctrl + C out of the attached window and use the following docker command.
```
docker compose down
```
## Notes
* Cerdiwen can be run on the CPU instead of the GPU by commenting out the deploy section in the docker-compose.yml, however this will vastly decrease the time required for it to handle the prompts and make it far less useable as a tool.
* By default llama3 model is used, adjust the environment variable in the docker compose if you want to use a larger/newer model.
# Resources
Resources used to create this tiny project include:
* **Ollama**. https://hub.docker.com/r/ollama/ollama
* **OllamaSharp**. https://github.com/awaescher/OllamaSharp