{"id":51902391,"url":"https://github.com/linuxserver/pelorus","last_synced_at":"2026-07-26T17:30:55.998Z","repository":{"id":372050785,"uuid":"1292907863","full_name":"linuxserver/pelorus","owner":"linuxserver","description":"Agentic control of Linux desktops over at-spi and text based primatives with visual analysis fallback. Control a linux desktop and run CI testing with simple prompts. ","archived":false,"fork":false,"pushed_at":"2026-07-18T14:58:50.000Z","size":50,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-07-18T16:23:18.314Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/linuxserver.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":"linuxserver","open_collective":"linuxserver"}},"created_at":"2026-07-08T00:34:26.000Z","updated_at":"2026-07-18T14:58:53.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/linuxserver/pelorus","commit_stats":null,"previous_names":["linuxserver/pelorus"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/linuxserver/pelorus","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linuxserver%2Fpelorus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linuxserver%2Fpelorus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linuxserver%2Fpelorus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linuxserver%2Fpelorus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/linuxserver","download_url":"https://codeload.github.com/linuxserver/pelorus/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linuxserver%2Fpelorus/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35922387,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-07-20T02:08:10.276Z","status":"online","status_checked_at":"2026-07-26T02:00:06.503Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-07-26T17:30:55.924Z","updated_at":"2026-07-26T17:30:55.982Z","avatar_url":"https://github.com/linuxserver.png","language":"Python","funding_links":["https://github.com/sponsors/linuxserver","https://opencollective.com/linuxserver"],"categories":[],"sub_categories":[],"readme":"# Pelorus\n\nAI navigator for Selkies-powered Linux desktops. Pelorus runs a FastAPI server that gives an LLM agent (Ollama, OpenAI-compatible, or Gemini) control over mouse, keyboard, screenshot, and window management via the Pixelflux computer-use backend, a Linux accessibility tree (AT-SPI), and optional KWin D-Bus integration.\n\n## How it works\n\nPelorus sits between an LLM and your desktop as an agent loop:\n\n1. **State capture** - reads the desktop accessibility tree (AT-SPI) to list icons, taskbar buttons, open windows, and the start menu. On KDE it also queries KWin D-Bus for accurate window geometry.\n2. **Model inference** - sends the captured state + system prompt + your task to an LLM. The model responds with text and/or tool calls (`mouse_move`, `click`, `key`, `type`, `scroll`, `screenshot`, `explore_window`, `desktop_state`, etc.).\n3. **Tool execution** - tool calls are dispatched to the Pixelflux computer-use backend for input actions, or handled locally for introspection (desktop state, window tree exploration, window close).\n4. **Repeat** - results feed back to the LLM until the task completes or `max_steps` is reached.\n\nAT-SPI works on any Linux desktop (GNOME, KDE, etc.). KWin D-Bus integration is specific to KDE and enhances window geometry accuracy, Pelorus works without it (with reduced coordinate precision for window interiors).\n\n### Vision vs. text-based control\n\nUnlike systems that pipe full screenshots into a vision model, Pelorus primarily works by reading the desktop accessibility tree (AT-SPI), giving the LLM structured, precise coordinates for every clickable element. This makes it far more reliable than vision-based approaches, which struggle with click targeting even on frontier models.\n\nVision mode is available (set `vision: true` on a server) and used as a fallback for applications that lack AT-SPI support, such as games, Electron apps without accessibility bridges, or custom OpenGL canvases. When vision is enabled, the agent can request per-application screenshots to interpret non-text interfaces. This is inherently less reliable for precise coordinate targeting, and text-based control via AT-SPI will always be preferred when available. The practical benefit is that even a quantized local model can control a desktop session with high accuracy, since it relies on structured data rather than pixel interpretation.\n\n## Quick Start\n\n```bash\npelorus\n# listens on 0.0.0.0:5100  (or $PELORUS_PORT)\n```\n\nOn first run a config file is created at `/config/agent/config.toml` seeded from environment variables.\n\nOpen `http://localhost:5100` in a browser for the chat UI, or use the REST API (see [API.md](API.md)).\n\n## Configuration\n\nServers are stored in `/config/agent/config.toml`:\n\n```toml\n[[servers]]\nid = \"svr_abc123def456\"\nname = \"Default Server\"\nprovider = \"ollama\"\nendpoint = \"http://localhost:11434\"\nmodel = \"gemma4:12b\"\napi_key = \"\"\nvision = false\ndefault = true\n```\n\nEnvironment variables in TOML values are resolved via `${VAR_NAME}` syntax.\n\n| Env var | Default | Purpose |\n|---|---|---|\n| `PELORUS_PORT` | `5100` | HTTP server port |\n| `PIXELFLUX_CU` | `5000` | Computer-use backend port |\n| `PELORUS_PROVIDER` | `ollama` | Provider for config seed |\n| `PELORUS_ENDPOINT` | `http://localhost:11434` | Endpoint for config seed |\n| `PELORUS_MODEL` | `gemma4:12b` | Model for config seed |\n| `PELORUS_API_KEY` | `\"\"` | API key for config seed |\n| `PELORUS_ENVIRONMENT` | `KDE Plasma Desktop (Linux)` | Desktop environment label shown in `desktop_state` header; override for other DEs (e.g. `GNOME (Linux)`, `Sway (Linux)`) |\n\n## Web Interface\n\nThe SPA frontend (vanilla JS, served at `/`) provides a full chat experience with session tabs, streaming output, server management, and settings, all communicating via WebSocket. Server selection and client settings are persisted to `localStorage`.\n\n## API Reference\n\nSee [API.md](API.md) for the complete REST API reference. The primary endpoint is `POST /api/run`, which accepts a task prompt and optional LLM configuration (provider, endpoint, model, api_key), either referencing a pre-configured server from `config.toml` or specifying credentials inline.\n\nInteractive OpenAPI documentation is available at `http://localhost:5100/docs` when the server is running.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flinuxserver%2Fpelorus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flinuxserver%2Fpelorus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flinuxserver%2Fpelorus/lists"}