https://github.com/engineering87/sibillaos
Linux distro with a built-in LLM. Detects your hardware, picks vLLM or Ollama, downloads a model that fits, and serves an OpenAI-compatible API from first boot.
https://github.com/engineering87/sibillaos
debian-packages distro inference iso-builder linux linux-distribution llm local-ai ollama self-hosted ubuntu vllm
Last synced: 9 days ago
JSON representation
Linux distro with a built-in LLM. Detects your hardware, picks vLLM or Ollama, downloads a model that fits, and serves an OpenAI-compatible API from first boot.
- Host: GitHub
- URL: https://github.com/engineering87/sibillaos
- Owner: engineering87
- License: apache-2.0
- Created: 2026-07-03T08:40:23.000Z (10 days ago)
- Default Branch: main
- Last Pushed: 2026-07-03T10:24:35.000Z (10 days ago)
- Last Synced: 2026-07-03T11:25:50.197Z (10 days ago)
- Topics: debian-packages, distro, inference, iso-builder, linux, linux-distribution, llm, local-ai, ollama, self-hosted, ubuntu, vllm
- Language: Shell
- Homepage:
- Size: 819 KB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
**Install Linux, get a working LLM API. Nothing else to set up.**
[](LICENSE)
[](https://ubuntu.com)
[](https://github.com/engineering87/sibillaos/actions/workflows/build-iso.yml)
[](docs/architecture.md)
Running a local LLM still means picking an engine, matching a model to your VRAM, choosing a quantization and wiring up a service. SibillaOS does all of that in the installer. Boot the ISO, answer the usual questions, and the first thing your machine does is serve an OpenAI-compatible API.
```console
$ curl https://myserver:8080/v1/chat/completions \
-H "Authorization: Bearer $(cat /etc/llmd/apikey)" \
-d '{"model": "default", "messages": [{"role": "user", "content": "hello"}]}'
```
## How it works
The installer detects your hardware and makes the decisions a human would otherwise have to research.
| | |
|---|---|
| **Engine selection** | vLLM in an OCI container on datacenter GPUs (24 GB VRAM and up), Ollama everywhere else, CPU-only machines included. |
| **Model sizing** | [llmfit](https://github.com/AlexsJones/llmfit) recommends only models that actually fit your VRAM and RAM, with the best quantization and a speed estimate. |
| **Curated catalog** | Permissively licensed (Apache-2.0/MIT), non-gated Hugging Face repos only. Verified ids, signed list. |
| **Resilient download** | The model is pulled from Hugging Face during install and resumed at first boot if the connection drops. |
| **Single endpoint** | One OpenAI-compatible API on port 8080, TLS, bearer token. Engines stay on loopback. |
Two variants: headless server and desktop (GNOME, optional Open WebUI).
## Getting started
The easiest path is a prebuilt ISO from [Releases](https://github.com/engineering87/sibillaos/releases), verified against its `SHA256SUMS`.
To build from source you only need `xorriso` and `curl` (no root). The build downloads the official Ubuntu 24.04 live-server ISO, verifies its checksum and repacks it with the SibillaOS autoinstall, packages and branding:
```bash
sudo apt-get install xorriso
./packages/build-debs.sh # build the llmd-* debs
./iso/build.sh # out/sibillaos--amd64.iso
```
Every push to `main` also builds the ISO in CI, boots it in QEMU and runs the automated install end to end; the image is attached to each run as an artifact.
## Repository layout
```
iso/ ISO repack (official Ubuntu live-server + payload) and autoinstall
packages/ Debian packages: hardware detection, engines, gateway, first boot
catalog/ curated model list (signed JSON)
branding/ logo, banner and wallpaper
docs/ architecture document
```
## Status
Proof of concept, moving toward MVP. Before any real deployment be aware that the autoinstall user password is a placeholder and must be replaced, and that engine versions are not yet pinned in the ISO build. The full design, decision log and roadmap live in [docs/architecture.md](docs/architecture.md).
## License
Apache-2.0, see [LICENSE](LICENSE). Bundled components keep their own licenses: vLLM (Apache-2.0), Ollama (MIT), llmfit (MIT). NVIDIA drivers are not redistributed by this repository; the ISO installs them from the Ubuntu `restricted` component.