https://github.com/openclaw/peekaboo
Peekaboo is a macOS CLI & optional MCP server that enables AI agents to capture screenshots of applications, or the entire system, with optional visual question answering through local or remote AI models.
https://github.com/openclaw/peekaboo
ai macos mcp screenshots swift
Last synced: 21 days ago
JSON representation
Peekaboo is a macOS CLI & optional MCP server that enables AI agents to capture screenshots of applications, or the entire system, with optional visual question answering through local or remote AI models.
- Host: GitHub
- URL: https://github.com/openclaw/peekaboo
- Owner: openclaw
- License: mit
- Created: 2025-05-22T16:09:06.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2026-05-20T03:03:58.000Z (25 days ago)
- Last Synced: 2026-05-21T08:40:22.231Z (24 days ago)
- Topics: ai, macos, mcp, screenshots, swift
- Language: Swift
- Homepage: http://www.peekaboo.boo
- Size: 600 MB
- Stars: 4,405
- Watchers: 15
- Forks: 331
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
- Security: docs/security.md
- Agents: AGENTS.md
Awesome Lists containing this project
README
# Peekaboo 🫣 - Mac automation that sees the screen and does the clicks.
[](https://www.npmjs.com/package/@steipete/peekaboo)
[](https://opensource.org/licenses/MIT)
[-0078d7?logo=apple&logoColor=white&style=flat-square)](https://www.apple.com/macos/)
[](https://swift.org/)
[](https://nodejs.org/)
[](https://github.com/steipete/peekaboo/releases/latest)
[](https://github.com/steipete/homebrew-tap)
[](https://deepwiki.com/steipete/peekaboo)
Peekaboo brings high-fidelity screen capture, AI analysis, and complete GUI automation to macOS. Version 3 adds native agent flows and multi-screen automation across the CLI and MCP server.
## What you get
- Pixel-accurate captures (windows, screens, menu bar) with optional Retina 2x scaling.
- Natural-language agent that chains Peekaboo tools (see, click, type, scroll, hotkey, menu, window, app, dock, space).
- Action-first UI automation for routine clicks/scrolls, with synthetic input fallback for apps that need it.
- Direct accessibility tools for settable values and named actions (`set-value`, `perform-action`).
- Menu and menubar discovery with structured JSON; no clicks required.
- Multi-provider AI through Tachikoma, including hosted, local, and OpenAI-/Anthropic-compatible providers.
- MCP server for Codex, Claude Code, and Cursor plus a native CLI; the same tools in both.
- Configurable, testable workflows with reproducible sessions and strict typing.
- Requires macOS Screen Recording + Accessibility permissions (see [docs/permissions.md](docs/permissions.md)).
## Install
- macOS app + CLI (Homebrew):
```bash
brew install steipete/tap/peekaboo
```
- MCP server (Node 22+, no global install needed):
```bash
npx -y @steipete/peekaboo
```
## Quick start
```bash
# Capture full screen at Retina scale and save to Desktop
peekaboo image --mode screen --retina --path ~/Desktop/screen.png
# Click a button by label (captures, resolves, and clicks in one go)
peekaboo see --app Safari --json | jq -r '.data.snapshot_id' | read SNAPSHOT
peekaboo click --on "Reload this page" --snapshot "$SNAPSHOT"
# Directly set a text field value when the accessibility value is settable
peekaboo set-value --on T1 --value "hello" --snapshot "$SNAPSHOT"
# Invoke a named accessibility action on an element
peekaboo perform-action --on B1 --action AXPress --snapshot "$SNAPSHOT"
# Run a natural-language automation
peekaboo agent "Open Notes and create a TODO list with three items"
# Run as an MCP server (Codex, Claude Code, Cursor)
npx -y @steipete/peekaboo
# Minimal MCP client config snippet:
# {
# "mcpServers": {
# "peekaboo": {
# "command": "npx",
# "args": ["-y", "@steipete/peekaboo"],
# "env": {
# "PEEKABOO_AI_PROVIDERS": "openai/gpt-5.5,anthropic/claude-opus-4-7"
# }
# }
# }
# }
```
## Shell completions
Peekaboo can generate shell-native completions directly from the same Commander
metadata that powers CLI help and docs:
```bash
# Current shell (recommended)
eval "$(peekaboo completions $SHELL)"
# Explicit shells
eval "$(peekaboo completions zsh)"
eval "$(peekaboo completions bash)"
peekaboo completions fish | source
```
For persistent setup and troubleshooting, see
[docs/commands/completions.md](docs/commands/completions.md).
| Command | Key flags / subcommands | What it does |
| --- | --- | --- |
| [see](docs/commands/see.md) | `--app`, `--mode screen/window`, `--retina`, `--json` | Capture and annotate UI, return snapshot + element IDs |
| [click](docs/commands/click.md) | `--on `, `--snapshot`, `--wait-for`, `--coords` | Click by element ID, label, or coordinates |
| [type](docs/commands/type.md) | `--text`, `--clear`, `--profile`, `--delay` | Enter text with pacing options |
| [set-value](docs/commands/set-value.md) | `--on `, `--value`, `--snapshot` | Directly set a settable accessibility value |
| [perform-action](docs/commands/perform-action.md) | `--on `, `--action`, `--snapshot` | Invoke a named accessibility action |
| [press](docs/commands/press.md) | key names, `--count`, `--delay`, `--hold` | Special keys and sequences |
| [hotkey](docs/commands/hotkey.md) | combos like `cmd,shift,t` | Modifier combos (cmd/ctrl/alt/shift) |
| [scroll](docs/commands/scroll.md) | `--on `, `--direction up/down`, `--amount` | Scroll views or elements |
| [swipe](docs/commands/swipe.md) | `--from/--to`, `--duration`, `--steps` | Smooth gesture-style drags |
| [drag](docs/commands/drag.md) | `--from/--to`, modifiers, Dock/Trash targets | Drag-and-drop between elements/coords |
| [move](docs/commands/move.md) | `--to `, `--screen-index` | Position the cursor without clicking |
| [window](docs/commands/window.md) | `list`, `move`, `resize`, `focus`, `set-bounds` | Move/resize/focus windows and Spaces |
| [app](docs/commands/app.md) | `launch`, `quit`, `relaunch`, `switch`, `list` | Launch, quit, relaunch, switch apps |
| [space](docs/commands/space.md) | `list`, `switch`, `move-window` | List or switch macOS Spaces |
| [menu](docs/commands/menu.md) | `list`, `list-all`, `click`, `click-extra` | List/click app menus and extras |
| [menubar](docs/commands/menubar.md) | `list`, `click` | Target status-bar items by name/index |
| [dock](docs/commands/dock.md) | `launch`, `right-click`, `hide`, `show`, `list` | Interact with Dock items |
| [dialog](docs/commands/dialog.md) | `list`, `click`, `input`, `file`, `dismiss` | Drive system dialogs (open/save/etc.) |
| [image](docs/commands/image.md) | `--mode screen/window/menu`, `--retina`, `--analyze` | Screenshot screen/window/menu bar (+analyze) |
| [list](docs/commands/list.md) | `apps`, `windows`, `screens`, `menubar`, `permissions` | Enumerate apps, windows, screens, permissions |
| [tools](docs/commands/tools.md) | `--verbose`, `--json`, `--no-sort` | Inspect native Peekaboo tools |
| [completions](docs/commands/completions.md) | `[shell]` | Generate zsh/bash/fish completion scripts from Commander metadata |
| [config](docs/commands/config.md) | `init`, `show`, `add`, `login`, `models` | Manage credentials/providers/settings |
| [permissions](docs/commands/permissions.md) | `status`, `grant` | Check/grant required macOS permissions |
| [run](docs/commands/run.md) | `.peekaboo.json`, `--output`, `--no-fail-fast` | Execute `.peekaboo.json` automation scripts |
| [sleep](docs/commands/sleep.md) | `--duration` (ms) | Millisecond delays between steps |
| [clean](docs/commands/clean.md) | `--all-snapshots`, `--older-than`, `--snapshot` | Prune snapshots and caches |
| [agent](docs/commands/agent.md) | `--model`, `--dry-run`, `--resume`, `--max-steps`, audio | Natural-language multi-step automation |
| [mcp](docs/commands/mcp.md) | `serve` (default) | Run Peekaboo as an MCP server |
## Models and providers
Peekaboo's provider list changes with Tachikoma and the tested model catalog. See
[docs/providers.md](docs/providers.md) for the current provider reference, including OpenAI, Anthropic, xAI/Grok,
Google Gemini, MiniMax, Ollama, LM Studio, and compatible custom endpoints.
Set providers via `PEEKABOO_AI_PROVIDERS` or `peekaboo config add`.
## Learn more
- Command reference: [docs/commands/](docs/commands/)
- Platform support: [docs/platform-support.md](docs/platform-support.md)
- Architecture: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)
- Building from source: [docs/building.md](docs/building.md)
- Testing guide: [docs/testing/tools.md](docs/testing/tools.md)
- MCP setup: [docs/commands/mcp.md](docs/commands/mcp.md)
- Permissions: [docs/permissions.md](docs/permissions.md)
- Ollama/local models: [docs/ollama.md](docs/ollama.md)
- Agent chat loop: [docs/agent-chat.md](docs/agent-chat.md)
- Service API reference: [docs/service-api-reference.md](docs/service-api-reference.md)
## Community
- [PeekabooWin](https://github.com/FelixKruger/PeekabooWin) — Windows-first rewrite of the Peekaboo automation loop (JavaScript + PowerShell) by [@FelixKruger](https://github.com/FelixKruger)
- [PeekabooX](https://github.com/nordbyte/PeekabooX) — Linux-first rewrite of the Peekaboo automation loop (Rust + Python) by [@nordbyte](https://github.com/nordbyte)
## Development basics
- Requirements: see [docs/platform-support.md](docs/platform-support.md). Node 22+ is only needed for the npm MCP wrapper and pnpm helper scripts.
- Install deps: `pnpm install` then `pnpm run build:cli` or `pnpm run test:safe`.
- Lint/format: `pnpm run lint && pnpm run format`.
## License
MIT