https://github.com/bang404/openagent
Desktop AI agent client — multi-agent delegation, long-term memory, MCP, and Skills. Built with Tauri + SvelteKit + Rust.
https://github.com/bang404/openagent
ai-agent anthropic claude desktop-app langfuse llm mcp openai-compatible rust sqlite svelte sveltekit tauri typescript
Last synced: 2 days ago
JSON representation
Desktop AI agent client — multi-agent delegation, long-term memory, MCP, and Skills. Built with Tauri + SvelteKit + Rust.
- Host: GitHub
- URL: https://github.com/bang404/openagent
- Owner: BANG404
- License: gpl-3.0
- Created: 2026-05-17T16:17:03.000Z (2 months ago)
- Default Branch: master
- Last Pushed: 2026-07-23T15:13:15.000Z (4 days ago)
- Last Synced: 2026-07-23T17:07:31.442Z (4 days ago)
- Topics: ai-agent, anthropic, claude, desktop-app, langfuse, llm, mcp, openai-compatible, rust, sqlite, svelte, sveltekit, tauri, typescript
- Language: Rust
- Homepage: https://bang404.github.io/openagent/
- Size: 88.6 MB
- Stars: 5
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
- Notice: NOTICE
- Agents: AGENTS.md
Awesome Lists containing this project
README
---
## Table of contents
- [Table of contents](#table-of-contents)
- [What's new in 0.24](#whats-new-in-024)
- [Highlights](#highlights)
- [Agent runtime](#agent-runtime)
- [Interactive output](#interactive-output)
- [Tools and integrations](#tools-and-integrations)
- [Desktop experience](#desktop-experience)
- [Quick Start](#quick-start)
- [Install a release build](#install-a-release-build)
- [Prerequisites](#prerequisites)
- [Clone \& install](#clone--install)
- [Run in dev mode](#run-in-dev-mode)
- [Build a distributable](#build-a-distributable)
- [Configure your first provider](#configure-your-first-provider)
- [Enable web search (optional)](#enable-web-search-optional)
- [Choose a tool approval mode](#choose-a-tool-approval-mode)
- [Example: write a Skill](#example-write-a-skill)
- [Reusable roles and progressive Skill discovery](#reusable-roles-and-progressive-skill-discovery)
- [Reusable delegated roles](#reusable-delegated-roles)
- [Progressive Skill discovery](#progressive-skill-discovery)
- [Interactive prompts with `ask_user`](#interactive-prompts-with-ask_user)
- [AGUI — Inline Interactive Components](#agui--inline-interactive-components)
- [HTML Preview via Tool Call (`render_html`)](#html-preview-via-tool-call-render_html)
- [Memory file format](#memory-file-format)
- [Agent memory controls](#agent-memory-controls)
- [Architecture at a glance](#architecture-at-a-glance)
- [Project structure](#project-structure)
- [Repository activity](#repository-activity)
- [Star history](#star-history)
- [Contributors](#contributors)
- [Roadmap](#roadmap)
- [Contributing](#contributing)
- [Observability (optional)](#observability-optional)
- [Tool-call structured output integration test](#tool-call-structured-output-integration-test)
- [Further reading](#further-reading)
- [License](#license)
---
## What's new in 0.24
- **One window per workspace** — Open multiple projects at once while each window keeps its own active workspace; selecting an already-open workspace focuses its existing window.
- **Reusable delegated roles** — Save specialized agent workflows globally or per project, search them by responsibility, and dispatch them into live child conversations.
- **Smarter Skill discovery** — Group Skills with `metadata.category`; uncategorized Skills can be classified by a Flash task so large catalogs use progressive disclosure instead of filling every prompt.
- **Durable multimodal branches** — Image, PDF, and text attachments have rich previews, survive reloads and checkpoints, and remain available when editing a message to create a new branch.
- **Faster long conversations** — Conversation search and pagination, virtualized completed history, stable streaming rows, queued follow-up messages, and cancellable compaction keep large histories responsive.
- **Richer tool output** — Validated Mermaid rendering, image/video AGUI media, source-line previews, and managed background terminal sessions make long-running and visual work easier to follow.
For the complete release history and fixes, see [`CHANGELOG.md`](CHANGELOG.md).
---
## Highlights
### Agent runtime
- **Multi-Agent & Flash Agents Architecture** — A primary streaming **Chat Agent** for main conversations, and a suite of dedicated async **Flash Agents** (including **Memory Agent** for long-term memory synthesis, **Title Agent** for dynamic conversation renaming, and **Hook Agent** for background scheduled tasks).
- **Sub-Agent Delegation** — The Chat Agent can call `spawn_agent` to delegate tasks to nested sub-agents; progress streams in real-time into a sub-conversation shown nested under the parent in the sidebar.
- **Reusable Agent Roles** — Create global or project-scoped role workflows, discover them with hybrid search, and dispatch them as specialized child agents. Roles can be created automatically on first use or managed from the **Roles** panel.
- **Goal & Graph Loops (Autonomous Execution)** — Type `/goal` in the chat to run a self-correcting loop directly aiming at the objective. Type `/graph` to first plan a structured DAG task graph (`create_goal_graph_config`) and execute nodes asynchronously with parallel processing, feeding live graph status updates to the frontend.
- **Hybrid Long-Term Memory** — SQLite + FTS5 + 384-dim embeddings (fastembed `AllMiniLML6V2`) blended with time decay for cross-session recall. Before retrieval, an optional Flash task rewrites the latest message into a focused semantic query, so stored memories are matched to intent rather than just wording.
- **Interactive User Prompts (`ask_user`)** — The agent can pause mid-task and surface a structured form to the user — `text`, `select`, `checkbox_group`, `confirm`, `date`, and more. The agent blocks until the user responds, then continues with the collected values. No more one-shot guessing on ambiguous instructions.
### Interactive output
- **AGUI — Inline Interactive Components** — The agent can embed file and URL capsules, ECharts visualisations, source-line previews, and image/video media directly in its prose — all rendered live by the streamdown engine.
- **HTML Preview via Tool Call (`render_html`)** — The agent executes a dedicated `render_html` tool to display sandboxed native HTML previews (mockup pages, layouts, visual components) directly in the conversation stream, with fixed-height or expanded display modes and PNG export.
- **Validated Mermaid Rendering** — A dedicated render tool validates Mermaid source before presenting the diagram, while keeping the source available and supporting fullscreen inspection.
### Tools and integrations
- **MCP-Native** — Connect external MCP servers over HTTP or stdio; tools are injected into the agent at call time.
- **First-class Dev Tools** — Built-in file, search, fetch, and terminal tools. Managed terminal sessions support interactive or long-running background processes; `fetch` retrieves pages locally with Spider, extracts readable text, and supports pagination.
- **Tool Approval & Workspace Sandbox** — Choose per-call review, model-assisted approval, no approval, or a workspace-oriented sandbox for file and terminal tools.
- **Skills System** — Drop a `SKILL.md` into `~/.agents/skills/` or `/.agents/skills/`. Category-based progressive discovery keeps large global and project catalogs compact, with optional Flash classification for uncategorized Skills.
- **Checkpoints & File-Change Rollback** — Every turn is a checkpoint with reverse diffs; undo a single file or rewind the whole agent.
- **Pluggable LLMs & Multimodal** — Support multi-model selection and durable image, PDF, and text attachments with drag/paste, rich previews, checkpoint restoration, and branch editing across Anthropic, OpenAI, and compatible providers.
### Desktop experience
- **Context Compaction & Tree Conversations** — Automatically or manually compact long conversations into tree structures to save tokens, preserving history lineage via search-based message recall.
- **Responsive Conversation History** — Search and paginate the sidebar, queue follow-up messages during a run, and navigate virtualized transcripts without loading the entire history into the DOM.
- **Scheduled Chat Hooks** — Define recurring or one-off tasks triggered in the background. Hooks are fully persistent, auto-restored on startup, and supported by system tray notifications.
- **Project Drafts & Global/Local Scopes** — Keep drafts, memory, and skills scoped globally (in `~/.config/openagent`) or locally to your active workspace (in `.agents/`).
- **DESIGN.md & MDX Editor** — Dedicated edit panel for `DESIGN.md` in your workspace, plus a rich markdown editor (MdxMarkdownEditor) integrated into memory and skill management.
- **Multi-Workspace Desktop Integration** — Open each workspace in a dedicated window, focus existing workspace windows instead of duplicating them, launch on startup, minimize to the system tray, and reveal workspace locations in the native file manager.
- **Observability** — Optional Langfuse tracing via OpenTelemetry (`gen_ai.*` attributes).
- **Polished UI** — Apple-style design language, streaming markdown, Mermaid & ECharts rendering, light/dark themes, i18n (zh / en).
---
## Quick Start
### Install a release build
Download the latest installer or app bundle from [GitHub Releases](https://github.com/BANG404/openagent/releases). OpenAgent publishes separate **beta** and **stable** update channels; you can also check for updates manually from Settings.
To build from source instead, continue below.
### Prerequisites
| Tool | Version | Notes |
| ------ | ------- | ---------------------------------------------- |
| Bun | latest | Package manager — used instead of npm / yarn |
| Rust | 1.70+ | Required for the Tauri backend |
| Node | 18+ | Used by the SvelteKit toolchain |
> On Windows the Tauri prerequisites also include WebView2 and the MSVC build tools. See the [official Tauri prerequisites](https://tauri.app/start/prerequisites/) for platform-specific setup.
### Clone & install
```bash
git clone https://github.com/BANG404/openagent.git
cd openagent
bun install
```
### Run in dev mode
```bash
# Full Tauri desktop app (frontend + Rust backend)
bun tauri dev
# OR — frontend only (port 14221, no Rust)
bun run dev
```
### Build a distributable
```bash
bun tauri build
```
The built installers / app bundle land in `src-tauri/target/release/bundle/`.
---
## Configure your first provider
On first launch OpenAgent creates `~/.config/openagent/config.toml`. Open **Settings → Providers** and add a provider, or edit the file directly. Until an available model is configured, the composer keeps sending disabled and provides a **Configure models** shortcut to Settings.
```toml
[[providers]]
id = "anthropic-main"
name = "Anthropic"
provider = "anthropic"
api_key = "sk-ant-..."
base_url = "https://api.anthropic.com"
enabled = true
[defaults]
chat_model = { provider_id = "anthropic-main", model = "claude-sonnet-4-6" }
flash_model = { provider_id = "anthropic-main", model = "claude-haiku-4-5" }
```
OpenAI-compatible endpoints (DeepSeek, OpenRouter, local Ollama, etc.) work the same — just point `base_url` at the right host and set `provider = "openai"`. You may enter a host, a `/v1` API root, or a full `/chat/completions` URL; OpenAgent normalizes it to the API root.
### Enable web search (optional)
Open **Settings → Web Search** and configure the provider you want to use. The `web_search` tool is available only after the selected provider has the required value: a Brave or Tavily API key, or a SearXNG base URL. The independent `fetch` tool always retrieves a page directly with Spider, returns readable text, and paginates long results according to the configured fetch page size. Omit `page` for the first page and use subsequent 1-based page numbers for the rest. Saving Settings refreshes the tool list immediately.
### Choose a tool approval mode
Open **Settings → General → Approval Mode** to control how agent tool calls run. When a call needs your approval, OpenAgent pauses the conversation, shows the exact tool name and arguments, and resumes after you approve or deny it.
| Mode | Behavior |
| --- | --- |
| **Manual** | Ask you to approve every tool call. |
| **Automatic** | A Flash task assesses the impact; important or uncertain calls still come to you for review. |
| **Off** | Run all tool calls without the approval flow. |
| **Sandbox** (default) | Apply the workspace policy only to file-management and terminal tools: workspace-local work proceeds, while attempts to escape the workspace are denied. Other built-in and MCP tools keep their normal behavior. |
---
## Example: write a Skill
Create `~/.agents/skills/python-review/SKILL.md`:
```markdown
---
name: python-review
description: Review Python diffs for type-hint coverage, error handling, and PEP 8 compliance.
---
When asked to review Python code:
1. Check that public functions have type hints.
2. Flag bare `except:` clauses and silent failures.
3. Suggest more idiomatic stdlib alternatives where appropriate.
```
That's it — OpenAgent picks it up on the next message and lists it in the agent's system prompt. The agent reads the full body on demand via `read_file`.
OpenAgent also installs the bundled `find-skills` Skill into this global directory when it is missing.
---
## Reusable roles and progressive Skill discovery
### Reusable delegated roles
Open **Roles** in the sidebar to create and manage specialized workflows such as a code reviewer, release manager, or research assistant. A role contains a stable name plus the responsibilities, boundaries, workflow, and delivery standards appended to the delegated agent's system prompt.
- **Global roles** are reusable in every workspace.
- **Project roles** are visible only in the current workspace.
- The main agent can create a role on first dispatch, find saved roles by name or responsibility, and reuse them in child conversations. The Roles panel shows usage count and last-used time.
### Progressive Skill discovery
For a large Skill catalog, add a category under the frontmatter `metadata` map:
```yaml
---
name: python-review
description: Review Python diffs for correctness and maintainability.
metadata:
category: code-quality
---
```
OpenAgent initially exposes compact category summaries and loads the matching Skill descriptions on demand. Skills without a category can be grouped by the optional **Settings → Flash Tasks → Skill Category Task**; when it is disabled, uncategorized Skills continue to be listed with their full name, description, and path.
---
## Interactive prompts with `ask_user`
When the agent needs a decision before continuing — ambiguous instruction, technology choice, destructive operation, missing parameter — it calls `ask_user` to surface a structured form in the chat panel. The agent blocks until you respond; everything in the form is typed, so you can select from dropdowns, tick checkboxes, confirm a boolean, or pick a date without typing a sentence.
Supported field types:
| Type | Use case |
| --- | --- |
| `text` | Short free-form input |
| `textarea` | Multi-line text |
| `select` | Single choice from a list |
| `checkbox` | Single on/off toggle |
| `checkbox_group` | Multiple choices from a list |
| `date` | Date picker |
| `confirm` | Yes / No decision |
The agent is guided to ask once, ask clearly, and prefer structured fields over open text boxes.
---
## AGUI — Inline Interactive Components
Beyond markdown, the agent can embed interactive components directly in its responses. The frontend's streamdown renderer picks them up and renders them as rich, clickable elements — no copy-pasting paths or URLs needed.
Syntax: `ComponentName(prop: value, prop2: "string")`
| Component | Example | Renders as |
| --- | --- | --- |
| `File` | `File(path: "src/tools.rs", lines: "120-140")` | Clickable chip that opens the file at the given lines |
| `Url` | `Url(href: "https://docs.rs/rig", title: "rig docs")` | Capsule that opens the link in the browser |
| `Chart` | `Chart(type: "bar", labels: ["A","B"], data: [10,20])` | ECharts bar / line / pie chart |
| `Image` | `Image(src: "assets/result.png", caption: "Result")` | Workspace-local or HTTP(S) image with an optional caption |
| `Video` | `Video(src: "assets/demo.mp4", controls: true)` | Workspace-local or HTTP(S) video with playback controls |
Multi-series charts use `series: [{name, data}, ...]`.
---
## HTML Preview via Tool Call (`render_html`)
Instead of writing inline component tags, the agent can execute a dedicated `render_html` tool to display sandboxed HTML previews (such as mockup pages, layouts, or visual components) directly in the conversation stream. The rendered frame uses the HTML as provided without injecting app theme styles, supports a fixed-height window or expanded content height, and includes best-effort PNG download.
---
## Memory file format
Memory files have **two zones**. The Memory Agent only writes below the marker comment:
```markdown
## [User] Personal habits
## [Agent] Recent context summary
```
- Global memory → `~/.config/openagent/memory.md` (every conversation)
- Local memory → `/.agents/memory.md` (workspace-scoped)
## Agent memory controls
Open **Settings → Flash Tasks → Memory Task** to configure the long-term-memory workflow. Both options are enabled by default:
- **Agent memory retrieval** uses the Flash model to turn the current message into a focused query, retrieves relevant structured memory, and adds the results to the chat agent's system prompt. Turn it off when you want retrieval to use the message directly.
- **Personalized new-conversation greeting** runs only when the Memory Agent adds or removes memory. It generates and persists one short, natural greeting; memory may shape its tone or lightly suggest one relevant topic, but is never displayed as a user profile.
Disabling the Memory Agent stops its post-conversation extraction task; the two switches above control their respective follow-up behaviors independently.
---
## Architecture at a glance
```
┌──────────────────────────────┐ Tauri IPC ┌──────────────────────────────┐
│ SvelteKit Webview (src/) │ ◄──────────────────► │ Rust Core (src-tauri/) │
│ ── routes/+page.svelte │ invoke / event │ ── commands/ │
│ ── lib/components/... │ chat-chunk │ ── tools.rs · terminal.rs │
│ ── lib/chatStream.ts │ chat-tool-call │ ── conversation_memory.rs │
│ ── lib/streamdown/... │ chat-done │ ── skills.rs · mcp.rs │
└──────────────────────────────┘ │ ── checkpoint.rs · state.rs │
│ ── openagent-runtime crate │
└──────────────────────────────┘
│
┌────────────────────────────┼────────────────────────────┐
▼ ▼ ▼
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ SQLite + FTS5 │ │ LLM Providers │ │ MCP Servers │
│ + fastembed │ │ Claude · OpenAI │ │ HTTP · stdio │
└──────────────────┘ └──────────────────┘ └──────────────────┘
```
See [`AGENTS.md`](AGENTS.md) for the current contributor and architecture guide.
---
## Project structure
```
.
├── src/ # SvelteKit frontend (Svelte 5 · TypeScript)
│ ├── routes/ # Page components
│ └── lib/ # Components, stores, streamdown, types
├── src-tauri/ # Tauri / Rust backend
│ ├── crates/
│ │ └── openagent-runtime/ # Tauri-independent input routing contract
│ ├── src/
│ │ ├── lib.rs # App setup, plugin registration
│ │ ├── commands/ # Typed IPC commands grouped by domain
│ │ ├── config.rs # Config schema parsing and management
│ │ ├── state.rs # Global application state definition
│ │ ├── context.rs # System prompt assembly
│ │ ├── context_compaction.rs # Context compaction logic for tree-based history
│ │ ├── conversation_memory.rs # SQLite + fastembed hybrid retrieval
│ │ ├── checkpoint.rs # Per-turn checkpoints + reverse diffs
│ │ ├── skills.rs # SKILL.md discovery
│ │ ├── mcp.rs # MCP transports + tool injection
│ │ ├── tools.rs # Built-in dev tools (read/write file, web search, fetch, etc.)
│ │ ├── terminal.rs # Shell command execution and session management
│ │ ├── sub_agent.rs # Reusable role and child-agent execution
│ │ └── tracing_setup.rs # OpenTelemetry and Langfuse observability initialization
│ └── icons/ # App icons (all platforms)
└── docs/ # Design specs and architecture notes
```
---
## Repository activity
### Star history
---
## Contributors
Thanks goes to these wonderful people:
---
## Roadmap
- [ ] First-class screenshots & demo GIFs in this README
- [x] Pre-built installers via GitHub Releases (beta / stable channels)
- [ ] In-app skill marketplace
- [x] Dedicated windows for multiple workspaces
See [open issues](https://github.com/BANG404/openagent/issues) for the full backlog and feel free to file new ones.
---
## Contributing
Contributions are very welcome — feature ideas, bug fixes, and docs improvements alike.
1. Fork the repo and create a branch from `master`.
2. Follow the [Conventional Commits](https://www.conventionalcommits.org/) style — see existing log for the scopes we use (`feat(toast):`, `fix(mermaid):`, `refactor(ui):`, etc.).
3. Run `bun run check`, `bun run lint:actions`, and `cargo check --manifest-path src-tauri/Cargo.toml` before opening a PR.
4. Open a PR — describe **why**, not just **what**.
Project conventions live in [`AGENTS.md`](AGENTS.md); the UI/UX spec lives in [`docs/design.md`](docs/design.md).
---
## Observability (optional)
Drop a `.env` in the project root to enable Langfuse tracing:
```env
LANGFUSE_PUBLIC_KEY=pk-...
LANGFUSE_SECRET_KEY=sk-...
LANGFUSE_HOST=https://cloud.langfuse.com
```
When keys are present, Chat & Memory agent calls are instrumented with `gen_ai.*` OpenTelemetry attributes and exported via batch processor.
### Tool-call structured output integration test
The Rust test suite includes an opt-in integration test that verifies the configured model can produce structured results for title, memory, hook, and context-compaction tasks by calling the task-specific capture tool. Put the test provider settings in the repository root `.env`:
```env
OPENAGENT_STRUCTURED_OUTPUT_TESTS=1
OPENAGENT_STRUCTURED_OUTPUT_MODEL=your-model
OPENAGENT_STRUCTURED_OUTPUT_API_KEY=your-api-key
OPENAGENT_STRUCTURED_OUTPUT_BASE_URL=https://your-provider.example/v1
# Optional Langfuse tracing
LANGFUSE_PUBLIC_KEY=pk-...
LANGFUSE_SECRET_KEY=sk-...
LANGFUSE_HOST=https://cloud.langfuse.com
```
Then run:
```bash
cd src-tauri
cargo test env_configured_provider_supports_all_structured_output_tasks -- --nocapture
```
See [`.env.example`](.env.example) for a copyable template. The test defaults to the OpenAI-compatible provider, so `OPENAGENT_STRUCTURED_OUTPUT_MODEL`, `OPENAGENT_STRUCTURED_OUTPUT_API_KEY`, and `OPENAGENT_STRUCTURED_OUTPUT_BASE_URL` are enough for third-party OpenAI-compatible chat endpoints.
For Anthropic, set `OPENAGENT_STRUCTURED_OUTPUT_PROVIDER=anthropic` and provide `ANTHROPIC_API_KEY`.
---
## Further reading
- [`AGENTS.md`](AGENTS.md) — Contributor and architecture reference
- [`CHANGELOG.md`](CHANGELOG.md) — Full release history
- [`docs/release.md`](docs/release.md) — Versioning, beta/stable channels, and publishing workflow
- [`docs/design.md`](docs/design.md) — Apple-style design spec
- [Tauri docs](https://tauri.app/) · [SvelteKit docs](https://kit.svelte.dev/) · [rig (Rust LLM)](https://github.com/0xPlaygrounds/rig)
---
## License
OpenAgent is dual-licensed:
- **Open-source option:** [GNU GPL v3.0 or later](LICENSE) (`GPL-3.0-or-later`).
If you distribute OpenAgent or a derivative work under this option, the GPL
requires the corresponding source and GPL freedoms to be provided under its
terms; it does not allow a derivative work to be distributed as proprietary.
- **Commercial option:** a separate [commercial license](COMMERCIAL_LICENSE.md)
is available for organizations that need to distribute a proprietary
derivative work or otherwise need rights outside the GPL.
Commercial licensing is provided only through a separate written agreement.
The OpenAgent name and branding remain subject to
[TRADEMARKS.md](TRADEMARKS.md).
