{"id":47595514,"url":"https://github.com/mikkoparkkola/axterminator","last_synced_at":"2026-05-25T23:05:46.819Z","repository":{"id":345411664,"uuid":"1137031572","full_name":"MikkoParkkola/axterminator","owner":"MikkoParkkola","description":"MCP server for macOS desktop automation. Background GUI testing, 30 tools, audio capture, camera, virtual desktops. Built in Rust.","archived":false,"fork":false,"pushed_at":"2026-03-27T01:11:28.000Z","size":2462,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-27T02:35:36.352Z","etag":null,"topics":["accessibility","ai-agents","automation","background-testing","desktop-automation","gui-testing","macos","mcp","mcp-server","pyo3","python","rust","self-healing","testing-framework"],"latest_commit_sha":null,"homepage":"https://mikkoparkkola.github.io/axterminator/","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MikkoParkkola.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE-APACHE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-01-18T19:41:27.000Z","updated_at":"2026-03-24T00:37:12.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/MikkoParkkola/axterminator","commit_stats":null,"previous_names":["mikkoparkkola/axterminator"],"tags_count":13,"template":false,"template_full_name":null,"purl":"pkg:github/MikkoParkkola/axterminator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MikkoParkkola%2Faxterminator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MikkoParkkola%2Faxterminator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MikkoParkkola%2Faxterminator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MikkoParkkola%2Faxterminator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MikkoParkkola","download_url":"https://codeload.github.com/MikkoParkkola/axterminator/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MikkoParkkola%2Faxterminator/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31290740,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T13:12:26.723Z","status":"ssl_error","status_checked_at":"2026-04-01T13:12:25.102Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["accessibility","ai-agents","automation","background-testing","desktop-automation","gui-testing","macos","mcp","mcp-server","pyo3","python","rust","self-healing","testing-framework"],"created_at":"2026-04-01T18:01:15.462Z","updated_at":"2026-05-25T23:05:46.813Z","avatar_url":"https://github.com/MikkoParkkola.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AXTerminator\n\n\u003cdiv align=\"center\"\u003e\n\n[![CI](https://img.shields.io/github/actions/workflow/status/MikkoParkkola/axterminator/ci.yml?style=for-the-badge\u0026label=tests)](https://github.com/MikkoParkkola/axterminator/actions)\n[![Crates.io](https://img.shields.io/crates/v/axterminator?style=for-the-badge)](https://crates.io/crates/axterminator)\n[![Downloads](https://img.shields.io/crates/d/axterminator?style=for-the-badge)](https://crates.io/crates/axterminator)\n[![macOS](https://img.shields.io/badge/macOS-12%2B-black?style=for-the-badge\u0026logo=apple)](https://github.com/MikkoParkkola/axterminator)\n[![Rust](https://img.shields.io/badge/rust-stable-orange?style=for-the-badge\u0026logo=rust)](https://www.rust-lang.org/)\n[![License](https://img.shields.io/badge/license-community%20%2B%20commercial-blue?style=for-the-badge)](LICENSE.md)\n[![Discussions](https://img.shields.io/github/discussions/MikkoParkkola/axterminator?style=for-the-badge\u0026color=blue)](https://github.com/MikkoParkkola/axterminator/discussions)\n[![Install in VS Code](https://img.shields.io/badge/VS_Code-Install_MCP-0078d4?style=for-the-badge\u0026logo=visualstudiocode)](https://insiders.vscode.dev/redirect/mcp/install?name=axterminator\u0026config=%7B%22command%22%3A%22axterminator%22%2C%22args%22%3A%5B%22mcp%22%2C%22serve%22%5D%7D)\n[![Install in Cursor](https://img.shields.io/badge/Cursor-Install_MCP-black?style=for-the-badge\u0026logo=cursor)](cursor://anysphere.cursor-deeplink/mcp/install?name=axterminator\u0026config=%7B%22command%22%3A%22axterminator%22%2C%22args%22%3A%5B%22mcp%22%2C%22serve%22%5D%7D)\n\n**MCP server that gives AI agents the ability to see and control macOS applications.**\n\n[Deploy](#deploy) · [MCP Tools](#mcp-tools) · [CLI](#cli) · [Query Syntax](#query-syntax) · [AX-first vs Vision-first](#ax-first-vs-vision-first) · [FAQ](#faq) · [Troubleshooting](#troubleshooting) · [Wiki](https://github.com/MikkoParkkola/axterminator/wiki) · [Known Limitations](#known-limitations)\n\n\u003c/div\u003e\n\n---\n\nUp to 34+ MCP tools (27 core + optional audio, camera, spaces). Background interaction via the macOS Accessibility API. 379us per element access. Audio capture, camera input, virtual desktop isolation. Your AI agent connects and your Mac becomes an extension of it.\n\n**Platform scope.** AX-first-with-vision-fallback is **macOS-only**. iOS/iPadOS support is tracked in [#43](https://github.com/MikkoParkkola/axterminator/issues/43) as a future capability and, when shipped, will be **screenshot + vision only** — the `idevice`/RPPairing surface does not expose AX trees for third-party apps, and UIAutomation would require an on-device test runner (breaks the \"mac-only agent, no on-device prereqs\" contract). Reported screenshot latency via CoreDeviceProxy: **~350ms USB, ~700ms WiFi** (credit [@m13v](https://github.com/MikkoParkkola/axterminator/issues/43#issuecomment-4274488358)). tvOS is **unverified** — the RSD surface is narrower than `idevice`'s tvOS target suggests; please open an issue with device evidence if you've tested. See [#42](https://github.com/MikkoParkkola/axterminator/issues/42) for the AX-first-vs-vision-first positioning rationale.\n\n## Deploy\n\n**Tell your AI assistant** (recommended):\n\n\u003e Read https://github.com/MikkoParkkola/axterminator and install axterminator as my macOS GUI automation MCP server\n\nYour agent will install the binary, wire itself up, and request accessibility permissions. Works in Claude Code, Cursor, Windsurf, and any AI with terminal access.\n\n**Or install manually:**\n\n```bash\nbrew install MikkoParkkola/tap/axterminator\n```\n\nGrant accessibility permissions: **System Settings \u003e Privacy \u0026 Security \u003e Accessibility** (add your terminal app).\n\n`axterminator mcp install` auto-detects your AI client. Specify one with `--client`:\n\n```bash\naxterminator mcp install                       # Claude Desktop (default)\naxterminator mcp install --client claude-code  # Claude Code\naxterminator mcp install --client cursor       # Cursor\naxterminator mcp install --client windsurf     # Windsurf\naxterminator mcp install --client codex        # OpenAI Codex CLI\naxterminator mcp install --client vscode       # VS Code Copilot\naxterminator mcp install --client zed          # Zed\naxterminator mcp install --dry-run             # Preview without writing\n```\n\nAlso supported: `gemini`, `amazon-q`, `lm-studio`.\n\n\u003cdetails\u003e\n\u003csummary\u003eMore install options\u003c/summary\u003e\n\n```bash\n# From crates.io\ncargo install axterminator --features cli\n\n# Build from source\ngit clone https://github.com/MikkoParkkola/axterminator\ncd axterminator\ncargo build --release --features cli\n\n# Manual JSON (Claude Desktop, Cursor, Windsurf, etc.)\n# { \"mcpServers\": { \"axterminator\": { \"command\": \"axterminator\", \"args\": [\"mcp\", \"serve\"] } } }\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eCodex config\u003c/summary\u003e\n\nFor Codex (`~/.codex/config.toml`):\n\n```toml\n[mcp_servers.axterminator]\ncommand = \"axterminator\"\nargs = [\"mcp\", \"serve\"]\n```\n\n\u003c/details\u003e\n\nDone. Your agent has 27 core tools (up to 34+ with all feature flags) to control any macOS app.\n\n## MCP Tools\n\n| Category | Tools | What the agent can do |\n|----------|-------|----------------------|\n| **GUI** | `ax_connect`, `ax_find`, `ax_click`, `ax_click_at`, `ax_type`, `ax_set_value`, `ax_get_value`, `ax_scroll`, `ax_drag`, `ax_key_press` | Connect to apps, find elements, interact |\n| **Observe** | `ax_is_accessible`, `ax_screenshot`, `ax_get_tree`, `ax_get_attributes`, `ax_list_windows`, `ax_list_apps`, `ax_wait_idle` | Check permissions, see UI state, screenshots |\n| **Verify** | `ax_assert`, `ax_find_visual`, `ax_visual_diff`, `ax_a11y_audit` | Assert element state, AI vision fallback, visual regression, WCAG audit |\n| **System** | `ax_clipboard`, `ax_run_script`, `ax_undo`, `ax_session_info`, `ax_analyze` | Clipboard, AppleScript/JXA, undo actions, session state, UI analysis |\n| **Audio** | `ax_listen`, `ax_speak`, `ax_audio_voices`, `ax_audio_devices` | Capture mic/system audio, text-to-speech, inspect installed macOS voices; optional Kokoro/Piper TTS via `enhanced-tts` |\n| **Camera** | `ax_camera_capture`, `ax_gesture_detect`, `ax_gesture_listen` | Camera frames, gesture recognition |\n| **Spaces** | `ax_list_spaces`, `ax_create_space`, `ax_move_to_space`, `ax_switch_space`, `ax_destroy_space` | Virtual desktop isolation |\n\n### Resources\n\nAgents can browse app state without tool calls:\n\n| Resource | What |\n|----------|------|\n| `axterminator://apps` | Running applications |\n| `axterminator://app/{name}/tree` | Live element hierarchy |\n| `axterminator://app/{name}/screenshot` | Current screenshot |\n| `axterminator://app/{name}/state` | Focused element, window title |\n| `axterminator://system/displays` | Monitor layout |\n\n### Security\n\nDestructive actions require confirmation via elicitation. HTTP transport requires bearer token auth. The AI has hands, not root.\n\n## CLI\n\n```bash\naxterminator apps                        # List accessible apps\naxterminator find \"Save\" --app Safari    # Find element\naxterminator click \"Save\" --app Safari   # Click it\naxterminator screenshot --app Safari     # Capture screenshot\naxterminator tree --app Finder           # Element hierarchy\naxterminator mcp serve --http 8080 --token secret  # HTTP transport\n```\n\n## Query Syntax\n\n```bash\n# Simple text — matches ANY of: title, description, value, label, identifier\naxterminator find \"Save\" --app Safari\n\n# By role\naxterminator find \"role:AXButton\" --app Safari\n\n# Combined role + attribute (AND)\naxterminator find \"role:AXButton title:Save\" --app Safari\n\n# By description (useful for apps like Calculator)\naxterminator find \"description:equals\" --app Calculator\n\n# By value\naxterminator find \"value:42\" --app Calculator\n\n# XPath-like\naxterminator find \"//AXButton[@AXTitle='Save']\" --app Safari\n```\n\n## How It Works\n\nAXTerminator uses an undocumented behavior of Apple's Accessibility API: `AXUIElementPerformAction()` works on unfocused windows. Your agent clicks buttons in one app while you work in another. Neither notices.\n\n379us per element access (Criterion, M1 MacBook Pro). Appium needs 500ms for the same thing.\n\n7-strategy self-healing locators survive UI changes: data_testid, aria_label, identifier, title, xpath, position, visual_vlm.\n\n## AX-first vs Vision-first\n\nOpenAI Codex computer use, Anthropic Claude Computer Use, Google Gemini Operator, and Perplexity Personal Computer all use the same paradigm: **screenshot → vision LLM → pixel coordinates → cursor click**. The vendor changes; the paradigm does not.\n\naxterminator uses the opposite default: **AX semantic tree → element reference → action**. Vision is a fallback for the rare surfaces the AX tree cannot reach (canvas apps, games, OpenGL/Metal renderers).\n\n| Dimension | Vision-first (Codex CU / Claude CU / Gemini / …) | AX-first (axterminator) |\n|-----------|---------------------------------------------------|-------------------------|\n| **Speed** | seconds per action (vision round-trip + LLM) | ~379 µs per action (measured, Criterion) |\n| **Cost** | vision tokens on every action | ~free (AX API call) |\n| **Reliability** | pixel-brittle — breaks on theme, font, layout changes | element-stable — semantic addressing survives UI changes |\n| **Background** | visible cursor movement | truly background — no visible cursor |\n| **Dense / labeled UIs** | struggles with small targets | reads labels directly from the AX tree |\n| **Canvas / games / OpenGL** | works (universal fallback) | needs vision fallback (`ax_find_visual`) |\n| **Cross-platform** | anywhere a screenshot works | macOS only (see [#43](https://github.com/MikkoParkkola/axterminator/issues/43) for iOS roadmap) |\n\n**The key insight:** axterminator is not a competitor to those agents — it is a layer *underneath* them. Any agent that can call an MCP tool (Claude Code, Codex CLI, Cursor, Windsurf, VS Code Copilot, Gemini CLI) can use axterminator as its hands. They provide reasoning; axterminator provides reliable, cheap, background-safe action.\n\n\u003e **Coverage gate.** Before investing heavily in vision-fallback features, run the AX coverage audit below for one week on your actual app surface. If \u003e95% of your actions resolve via AX, vision fallback is a nice-to-have. If \u003c80%, expand the vision fallback path first. See [benches/probes/README.md](benches/probes/README.md) for the audit harness.\n\n## FAQ\n\n**Why use this instead of Codex computer use / Claude Computer Use / Gemini Operator?**\n\nYou probably use *both*. Those agents call axterminator as an MCP tool. axterminator gives them AX-semantic actions that are ~1,000× faster and cost nothing per call. The vision model stays for tasks that genuinely need it (canvas apps, games), not for clicking a Save button in TextEdit.\n\n**Isn't vision-first simpler — no setup, works everywhere?**\n\nIt works everywhere vision works: foreground, focused, slow, expensive. axterminator works in the background, costs nothing, and is sub-millisecond. For automation of native macOS apps — the majority of business software — AX is strictly better. Vision is the escape hatch, not the default.\n\nThe binding architecture decision is [ADR-0001: AX-first With Vision Fallback](docs/architecture/decisions/ADR-0001-ax-first-with-vision-fallback.md).\n\n**What app surfaces does axterminator cover?**\n\nAny macOS app that exposes the Accessibility API: all native AppKit/SwiftUI apps, Electron apps, web apps in Chrome/Safari/Firefox, terminal apps. Canvas-only surfaces (Figma canvas, game renderers, video players) use `ax_find_visual` — the built-in vision fallback that tries AX first and falls back to VLM automatically.\n\n**Does it work with agents that are already doing computer use?**\n\nYes. Add `axterminator mcp install --client codex` (or `--client claude-code`, etc.) and the agent gains 34 semantic tools it can call instead of pixel-clicking. It doesn't replace the agent's vision; it gives the agent a faster, cheaper path for the 90%+ of actions that don't need vision.\n\n**Who is it for beyond developers?**\n\n- **Executive assistants:** file triage, calendar entry, copy-paste between apps — all in the background while you work\n- **Sales ops:** CRM entry, proposal generation across native apps, screen scraping structured data from dense UIs\n- **Event coordinators:** spreadsheet-to-calendar sync, venue research across multiple apps, confirmation email drafts — reliable because it reads labels semantically, not by pixel position\n\n## Known Limitations\n\n| Operation | Background? | Notes |\n|-----------|:-----------:|-------|\n| Click, press, read values, screenshots | Yes | Core operations work without focus |\n| Text input | Partial | Some apps need focused text field |\n| Drag, system dialogs | No | Require cursor control / always grab focus |\n| Gesture recognition | Yes | Verified: thumbs_up at 88.8% confidence |\n| Speech transcription | Yes | Verified: on-device, requires Dictation enabled |\n\n**Platform coverage** (see [#43](https://github.com/MikkoParkkola/axterminator/issues/43)):\n\n| Platform | AX tree | Screenshot | App launch | Status |\n|----------|:-------:|:----------:|:----------:|--------|\n| macOS 12+ | ✅ | ✅ | ✅ | Shipped; sub-ms AX path |\n| iOS / iPadOS | ❌ (third-party apps) | ✅ (~350ms USB / ~700ms WiFi) | ✅ | **Planned, screenshot + vision only** — no AX semantics |\n| tvOS | ❔ | ❔ | ❔ | **Unverified** — RSD surface narrower than `idevice` suggests |\n| visionOS / watchOS | ❌ | ❌ | ❌ | Out of scope |\n\n## Troubleshooting\n\n| Symptom | Cause | Fix |\n|---------|-------|-----|\n| `Accessibility: DISABLED` | No permission granted | System Settings \u003e Privacy \u0026 Security \u003e Accessibility |\n| `Element not found` for short labels | App uses `AXDescription` not `AXTitle` | Try `description:label` or inspect with `axterminator tree` |\n| `Application not found` | Wrong name or app not running | Use bundle ID: `--bundle-id com.apple.calculator` |\n\n**AI agents**: Fetch [llms.txt](https://raw.githubusercontent.com/MikkoParkkola/axterminator/main/llms.txt) for machine-readable installation instructions.\n\n## Feature Flags\n\nThe Homebrew formula includes all features. When building from source, select capabilities with feature flags:\n\n```bash\ncargo build --release --features \"cli,audio,enhanced-tts,camera,spaces\"\n```\n\n| Flag | What |\n|------|------|\n| `cli` | CLI + MCP server (default) |\n| `audio` | Microphone/system audio, speech |\n| `enhanced-tts` | Optional Kokoro/Piper TTS engine routing and `axterminator models tts` downloads |\n| `camera` | Camera capture, gesture detection |\n| `spaces` | Virtual desktop management |\n| `http-transport` | HTTP MCP transport with auth |\n\n## Community\n\n- [Wiki](https://github.com/MikkoParkkola/axterminator/wiki) -- Full documentation\n- [Discussions](https://github.com/MikkoParkkola/axterminator/discussions) -- Questions, ideas, show-and-tell\n- [Issues](https://github.com/MikkoParkkola/axterminator/issues) -- Bugs\n\n## Acknowledgements\n\nInspired by [Terminator](https://github.com/mediar-ai/terminator) by [mediar-ai](https://github.com/mediar-ai), which pioneered accessible desktop GUI automation on Windows.\n\n## For AI Agents\n\nMachine-readable installation guide: [`llms.txt`](https://raw.githubusercontent.com/MikkoParkkola/axterminator/main/llms.txt)\n\nYour agent can fetch this URL to get step-by-step installation, MCP config for every host, and troubleshooting.\n\n## Ecosystem\n\naxterminator is part of a suite of MCP tools:\n\n| Tool | Description |\n|------|-------------|\n| [mcp-gateway](https://github.com/MikkoParkkola/mcp-gateway) | Universal MCP gateway — compact 12-15 tool surface replaces 100+ registrations |\n| [trvl](https://github.com/MikkoParkkola/trvl) | AI travel agent — 36 MCP tools for flights, hotels, ground transport |\n| [nab](https://github.com/MikkoParkkola/nab) | Web content extraction — fetch any URL with cookies + anti-bot bypass |\n| **[axterminator](https://github.com/MikkoParkkola/axterminator)** | **macOS GUI automation — 34 MCP tools via Accessibility API** |\n\n## License\n\nAXTerminator is free for personal, research, educational, noncommercial\nopen-source, and free public-good projects with attribution.\n\nBusiness use requires a written commercial license. See [LICENSE.md](LICENSE.md)\nand [COMMERCIAL.md](COMMERCIAL.md).\n\nEarlier releases published under `MIT OR Apache-2.0` remain under those earlier\nlicense grants for those earlier versions only.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmikkoparkkola%2Faxterminator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmikkoparkkola%2Faxterminator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmikkoparkkola%2Faxterminator/lists"}