https://github.com/lgdd/chatollama
A Docker Compose to run a local ChatGPT-like application using Ollama, Ollama Web UI & Mistral-7B-v0.1.
https://github.com/lgdd/chatollama
chatgpt-clone mistral mistral-7b ollama ollama-webui
Last synced: about 1 month ago
JSON representation
A Docker Compose to run a local ChatGPT-like application using Ollama, Ollama Web UI & Mistral-7B-v0.1.
- Host: GitHub
- URL: https://github.com/lgdd/chatollama
- Owner: lgdd
- Created: 2024-02-14T09:53:21.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-02-14T10:31:00.000Z (over 1 year ago)
- Last Synced: 2024-02-15T10:47:46.348Z (over 1 year ago)
- Topics: chatgpt-clone, mistral, mistral-7b, ollama, ollama-webui
- Homepage:
- Size: 2.93 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# ChatOllama
A Docker Compose to run a local ChatGPT-like application using [Ollama](https://github.com/ollama/ollama), [Open WebUI](https://github.com/open-webui/open-webui) with 2 models by default to try out: [Mistral NeMo](https://ollama.com/library/mistral-nemo) & [DeepSeek-R1-Distill-Llama-8B](https://ollama.com/library/deepseek-r1).
## Usage
Simply run:
```shell
docker compose up
```
The `ollama-pull--*` services will trigger an API call to Ollama to pull the associated models and shutdown when it's done. You should see the progress in the logs of that service which should end with:
```
{"status":"verifying sha256 digest"}
{"status":"writing manifest"}
{"status":"removing any unused layers"}
{"status":"success"}
```
> To verify the list of downloaded models, you can call Ollama on `http://localhost:11434/api/tags`.
The models are stored in a [volume](docker-compose.yml#L7-L8) to avoid downloading them at each restart of Ollama.
Once the model downloaded, you can go to http://localhost. By default, the port number mapped to the host is `80`, but you can change it by editing the [`docker-compose.yml`](docker-compose.yml#L27) file). Next, sign up to create an account (everything is local) and log in. On the top of the page, look for the `Select a model` dropdown menu and select `mistral:latest`. After selecting it, click on the `Set as default` link to avoid having to select it again each time you create a new discussion.
## Requirements
By default, Ollama is set to use 1 NVIDIA GPU:
```yaml
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [ gpu ]
```
If you want to run on CPU, you can comment the lines showed above in the [`docker-compose.yml`](docker-compose.yml#L15-L21) file and then run `docker compose up`.
If you want to run on an NVIDIA GPU, make sure that your Docker daemon configuration file contains the following:
```json
"runtimes": {
"nvidia": {
"path": "nvidia-container-runtime",
"runtimeArgs": []
}
}
```
> You can also add `"default-runtime": "nvidia"`.
Also, you should have installed the [NVIDIA CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit). To verify that Docker can access your GPU, you can run:
```shell
docker run --runtime nvidia --rm nvidia/cuda:12.3.1-base-ubuntu20.04 nvidia-smi
```
> [!NOTE]
> Make sure that the version of the `nvidia/cuda` image is aligned with the CUDA version installed using the toolkit.