https://github.com/pradumnasaraf/saraf-ai
Saraf AI is a fully local assistant built with Next.js and Docker. It connects seamlessly to LLMs via Docker Model Runner using Docker Compose.
https://github.com/pradumnasaraf/saraf-ai
ai ai-models docker docker-compose docker-model-runner javascript llm nextjs openai typescript
Last synced: 10 months ago
JSON representation
Saraf AI is a fully local assistant built with Next.js and Docker. It connects seamlessly to LLMs via Docker Model Runner using Docker Compose.
- Host: GitHub
- URL: https://github.com/pradumnasaraf/saraf-ai
- Owner: Pradumnasaraf
- License: apache-2.0
- Created: 2025-07-26T14:24:13.000Z (11 months ago)
- Default Branch: main
- Last Pushed: 2025-08-14T07:29:13.000Z (11 months ago)
- Last Synced: 2025-08-29T09:23:14.277Z (10 months ago)
- Topics: ai, ai-models, docker, docker-compose, docker-model-runner, javascript, llm, nextjs, openai, typescript
- Language: TypeScript
- Homepage:
- Size: 203 KB
- Stars: 7
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- Funding: .github/FUNDING.yml
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Codeowners: CODEOWNERS
- Security: SECURITY.md
Awesome Lists containing this project
README
## Saraf AI
**Saraf AI** is a chat assistant built with **Next.js**, using the OpenAI API framework to interface with the [Docker Model Runner](https://docs.docker.com/ai/model-runner). It runs entirely locally, ensuring your data remains private. Communication between the application and the LLM model is handled using the latest Docker Compose support for [running LLM models](https://docs.docker.com/ai/compose/models-and-compose/), enabling seamless and secure integration.
## Demo
https://github.com/user-attachments/assets/32fb49de-336b-406a-89ad-c1e9fc80fa60
## Prerequisites
- Docker and Docker Compose (version **2.38.0** or newer)
## Getting Started
1. Clone this repository.
2. Run `docker compose up`. This launches both the LLM model and the Next.js frontend. (Yes, it's that simple :))
3. Visit `http://localhost:3000` in your browser.
## Docker Compose Configuration
```yaml
services:
saraf-ai:
build:
context: .
dockerfile: Dockerfile
ports:
- 3000:3000
# Models to run
models:
- llm
models:
# Model Name
llm:
# Model Image
model: ai/smollm2
```
This is a standard Docker Compose setup with an additional `models` block that lets you define and run multiple AI models within the same file. You can swap out `ai/smollm2` for any other model available in the [DockerHub AI Catalog](https://hub.docker.com/catalogs/gen-ai).
> The `saraf-ai` service (Next.js app) requires `LLM_URL` and `LLM_MODEL` environment variables to function. These are automatically generated by Docker and injected into the Next.js environment based on the model name, in this case `llm`. The app then uses them to connect to the local LLM instance.
## License
This project is licensed under the Apache-2.0 license - see the [LICENSE](LICENSE) file for details.
## Security
If you discover a security vulnerability within this project, please check the [SECURITY](SECURITY.md) for more information.