https://github.com/parakeet-nest/tiny-genai-stack
https://github.com/parakeet-nest/tiny-genai-stack
Last synced: 18 days ago
JSON representation
- Host: GitHub
- URL: https://github.com/parakeet-nest/tiny-genai-stack
- Owner: parakeet-nest
- License: mit
- Created: 2024-05-05T06:47:54.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2024-09-13T04:10:32.000Z (almost 2 years ago)
- Last Synced: 2024-09-13T18:40:30.237Z (almost 2 years ago)
- Language: HTML
- Size: 200 KB
- Stars: 3
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Tiny GenAI Stack
This 🐳 Compose GenAI Stack is an example of the usage of the [Parakeet](https://github.com/parakeet-nest/parakeet) 🦜🪺 GoLang library, made to simplify the development of small generative AI applications.
## Start the stack
If you want to use the stack with your local install of Ollama:
```bash
HTTP_PORT=9999 LLM=tinydolphin OLLAMA_BASE_URL=http://host.docker.internal:11434 docker compose --profile webapp up
```
> Make sure that Ollama is started
If you want to use the stack with Ollama running in a container:
```bash
HTTP_PORT=8888 LLM=gemma:2b OLLAMA_BASE_URL=http://ollama:11434 docker compose --profile container up
```
> ✋ Compose will start the pull of the LLM, so be patient (my advice would be to use "small LLM" as tinyllama, tinydolphin or gemma:2b)
## Remarks
If you have a GPU on your workstation, you can turn on GPU access with Docker Compose: https://docs.docker.com/compose/gpu-support. With Mac M1, M2 and M3 use the local install of Ollama.
## Install dependencies (front)
**JavaScript**:
```bash
cd public/js
wget https://cdn.jsdelivr.net/npm/markdown-it@14.1.0/dist/markdown-it.min.js
wget https://cdn.jsdelivr.net/npm/beercss@3.7.8/dist/cdn/beer.min.js
wget https://cdn.jsdelivr.net/npm/material-dynamic-colors@1.1.2/dist/cdn/material-dynamic-colors.min.js
wget https://unpkg.com/htmx.org@2.0.2/dist/htmx.min.js
```
**Css**:
```bash
cd public/css
wget https://cdn.jsdelivr.net/npm/beercss@3.7.8/dist/cdn/beer.min.css
```