{"id":49341096,"url":"https://github.com/syedazharmbnr1/computer-use-mcp","last_synced_at":"2026-04-27T04:02:51.850Z","repository":{"id":350831442,"uuid":"1208432638","full_name":"syedazharmbnr1/computer-use-mcp","owner":"syedazharmbnr1","description":"macOS Computer Use MCP Server - 33 tools for screen control via Model Context Protocol. Works with Claude Code, Cursor, LM Studio, Ollama, llama.cpp, MLX, and all MCP clients.","archived":false,"fork":false,"pushed_at":"2026-04-12T09:25:46.000Z","size":28,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-12T11:19:15.328Z","etag":null,"topics":["ai-tools","automation","claude-code","computer-use","cursor","llama-cpp","lm-studio","macos","mcp","mcp-server","mlx","model-context-protocol","ollama"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/syedazharmbnr1.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-12T09:17:39.000Z","updated_at":"2026-04-12T09:25:53.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/syedazharmbnr1/computer-use-mcp","commit_stats":null,"previous_names":["syedazharmbnr1/computer-use-mcp"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/syedazharmbnr1/computer-use-mcp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/syedazharmbnr1%2Fcomputer-use-mcp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/syedazharmbnr1%2Fcomputer-use-mcp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/syedazharmbnr1%2Fcomputer-use-mcp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/syedazharmbnr1%2Fcomputer-use-mcp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/syedazharmbnr1","download_url":"https://codeload.github.com/syedazharmbnr1/computer-use-mcp/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/syedazharmbnr1%2Fcomputer-use-mcp/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32321940,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-26T23:26:28.701Z","status":"online","status_checked_at":"2026-04-27T02:00:06.769Z","response_time":128,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-tools","automation","claude-code","computer-use","cursor","llama-cpp","lm-studio","macos","mcp","mcp-server","mlx","model-context-protocol","ollama"],"created_at":"2026-04-27T04:02:42.771Z","updated_at":"2026-04-27T04:02:51.843Z","avatar_url":"https://github.com/syedazharmbnr1.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Computer Use MCP Server\n\n![Python](https://img.shields.io/badge/python-3.10+-blue.svg)\n![MCP](https://img.shields.io/badge/MCP-1.26+-green.svg)\n![macOS](https://img.shields.io/badge/platform-macOS-lightgrey.svg)\n![License](https://img.shields.io/badge/license-MIT-orange.svg)\n![Tools](https://img.shields.io/badge/tools-33-purple.svg)\n\nA production-grade **macOS Computer Use MCP Server** that exposes 33 tools across 10 categories for full desktop automation via the [Model Context Protocol](https://modelcontextprotocol.io). Control mouse, keyboard, screenshots, clipboard, windows, and more from any MCP-compatible AI client.\n\nWorks with **Claude Code**, **Cursor**, **VS Code**, **Windsurf**, **LM Studio**, **Ollama**, **llama.cpp**, **MLX**, and any MCP-compatible tool.\n\n---\n\n## Features\n\n### 33 Tools Across 10 Categories\n\n| Category | Tools | Description |\n|----------|-------|-------------|\n| **Mouse** (12) | `mouse_click`, `left_click`, `right_click`, `middle_click`, `double_click`, `triple_click`, `left_mouse_down`, `left_mouse_up`, `mouse_move`, `mouse_drag`, `scroll`, `mouse_scroll` | Full mouse control with coordinate-based clicking, dragging with 20-step interpolation, directional scrolling |\n| **Keyboard** (5) | `key`, `hold_key`, `keyboard_type`, `keyboard_press`, `keyboard_hotkey` | Unified key combos (`cmd+c`), hold-for-duration, Unicode text typing, individual key press, modifier hotkeys |\n| **Screenshot** (1) | `take_screenshot` | Full-screen or region capture with Retina scaling, coordinate metadata, and configurable resolution |\n| **Display** (3) | `switch_display`, `zoom`, `list_displays` | Multi-monitor switching, high-res region zoom for reading small text, display enumeration |\n| **Clipboard** (2) | `read_clipboard`, `write_clipboard` | Read/write system clipboard via NSPasteboard |\n| **Window** (2) | `get_active_window`, `list_windows` | Frontmost window info, enumerate all visible windows with position/size |\n| **Screen** (2) | `get_screen_info`, `get_cursor_position` | Display dimensions, Retina scale, accessibility status, cursor coordinates |\n| **System** (3) | `open_application`, `wait`, `run_shell_command` | Launch apps by name, timed waits, shell command execution |\n| **Access** (2) | `request_access`, `list_granted_applications` | App permission tracking for session-based access control |\n| **Batch** (1) | `computer_batch` | Execute multiple actions in a single call - eliminates round-trip latency |\n\n---\n\n## Quick Start\n\n```bash\n# 1. Clone\ngit clone https://github.com/syedazharmbnr1/computer-use-mcp.git\ncd computer-use-mcp\n\n# 2. Setup\npython3 -m venv .venv\nsource .venv/bin/activate\npip install -r requirements.txt  # or: pip install mcp mss pillow pyobjc-framework-Quartz\n\n# 3. Test\npython3 __main__.py\n```\n\nThe server communicates over **stdio** (stdin/stdout) using the MCP JSON-RPC protocol.\n\n---\n\n## Installation\n\n### Prerequisites\n\n- **macOS** (uses Quartz framework for input simulation)\n- **Python 3.10+**\n- **Accessibility permissions** (System Settings \u003e Privacy \u0026 Security \u003e Accessibility)\n\n### Install Dependencies\n\n```bash\ngit clone https://github.com/syedazharmbnr1/computer-use-mcp.git\ncd computer-use-mcp\npython3 -m venv .venv\nsource .venv/bin/activate\npip install mcp\u003e=1.26.0 mss pillow pyobjc-framework-Quartz\n```\n\n### Verify Installation\n\n```bash\npython3 -c \"\nfrom server.computer_use_server import ComputerUseMCPServer\nserver = ComputerUseMCPServer()\ntools = server._collect_all_tools()\nprint(f'Server OK - {len(tools)} tools registered')\n\"\n```\n\nExpected output: `Server OK - 33 tools registered`\n\n### Grant Accessibility Permission\n\nThe server needs macOS accessibility access to simulate mouse/keyboard input:\n\n1. Open **System Settings** \u003e **Privacy \u0026 Security** \u003e **Accessibility**\n2. Add your terminal app (Terminal, iTerm2, VS Code, etc.)\n3. Toggle the permission **ON**\n\n\u003e Screenshot capture works without accessibility permission. Only mouse/keyboard tools require it.\n\n---\n\n## Configuration for AI Coding Tools\n\nThe server uses **stdio transport** - it reads from stdin and writes to stdout. Every MCP client connects the same way: spawn the Python process and pipe stdio.\n\n### Claude Code\n\nEdit `~/.claude/settings.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"computer-use\": {\n      \"command\": \"/path/to/computer-use-mcp/.venv/bin/python3\",\n      \"args\": [\"/path/to/computer-use-mcp/__main__.py\"],\n      \"cwd\": \"/path/to/computer-use-mcp\"\n    }\n  }\n}\n```\n\nThen run `/mcp` in Claude Code to connect.\n\n### Cursor\n\nCreate `.cursor/mcp.json` in your project root (or `~/.cursor/mcp.json` globally):\n\n```json\n{\n  \"mcpServers\": {\n    \"computer-use\": {\n      \"command\": \"/path/to/computer-use-mcp/.venv/bin/python3\",\n      \"args\": [\"/path/to/computer-use-mcp/__main__.py\"]\n    }\n  }\n}\n```\n\n### VS Code + GitHub Copilot\n\nCreate `.vscode/mcp.json` in your workspace:\n\n```json\n{\n  \"mcpServers\": {\n    \"computer-use\": {\n      \"command\": \"/path/to/computer-use-mcp/.venv/bin/python3\",\n      \"args\": [\"/path/to/computer-use-mcp/__main__.py\"]\n    }\n  }\n}\n```\n\n### Windsurf\n\nEdit `~/.codeium/windsurf/mcp_config.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"computer-use\": {\n      \"command\": \"/path/to/computer-use-mcp/.venv/bin/python3\",\n      \"args\": [\"/path/to/computer-use-mcp/__main__.py\"]\n    }\n  }\n}\n```\n\n### JetBrains IDEs\n\nAdd via Settings \u003e Tools \u003e MCP Servers, using the same command/args pattern.\n\n### Zed\n\nAdd to your Zed settings (`~/.config/zed/settings.json`):\n\n```json\n{\n  \"language_models\": {\n    \"mcp_servers\": {\n      \"computer-use\": {\n        \"command\": \"/path/to/computer-use-mcp/.venv/bin/python3\",\n        \"args\": [\"/path/to/computer-use-mcp/__main__.py\"]\n      }\n    }\n  }\n}\n```\n\n### Cline / Continue.dev\n\nBoth support the standard MCP JSON config format. Add to their respective config files using the same `command` + `args` pattern shown above.\n\n---\n\n## Recommended Models for Tool Calling (April 2026)\n\nPer [LM Arena](https://arena.ai/leaderboard/) rankings and real-world testing, these are the best models for MCP tool calling:\n\n### Top Open Source Models (LM Arena Elo)\n\n| Rank | Model | Provider | Parameters | Highlights |\n|------|-------|----------|------------|------------|\n| 1 | **GLM-5** | Zhipu AI | MoE | #1 open source (Elo 1451), 77.8% SWE-bench Verified |\n| 2 | **Kimi K2.5** | Moonshot AI | MoE | HumanEval 99.0, stable across 200-300 sequential tool calls |\n| 3 | **GLM-4.7** | Zhipu AI | MoE | HumanEval 94.2, AIME 2025 95.7, GPQA 85.7 |\n| 4 | **GLM-5.1** | Zhipu AI | 744B MoE / 40B active | MIT license, 200K context, 8+ hour continuous agentic sessions |\n| 5 | **Qwen 3.6 Plus** | Alibaba | Dense | 1M context, native function calling, always-on CoT reasoning |\n| 6 | **Gemma 4 31B** | Google | 31B Dense | #3 Arena text, Apache 2.0, native tool calling, 256K context |\n| 7 | **Llama 4 Scout** | Meta | 17B active / 16 experts | 10M context window, multimodal, beats Gemini 2.0 Flash-Lite |\n| 8 | **Llama 4 Maverick** | Meta | 17B active / 128 experts | Beats GPT-4o, best multimodal in class |\n| 9 | **Mistral Small 4** | Mistral AI | 119B MoE / 6B active | Unified instruct+reasoning+coding+vision, 256K context |\n| 10 | **Qwen 3.5** | Alibaba | Multiple sizes | Most stable tool calling, rarely hallucinates calls |\n\n### Best Models by Platform\n\n**Ollama** (run locally via `ollama pull \u003cmodel\u003e`):\n- `gemma4` (E2B / E4B / 26B MoE / 31B Dense) — native function calling, best sub-32B for agents\n- `qwen3.5` / `qwen3.6-plus` — most stable tool calling, rarely drops parameters\n- `llama4` (Scout / Maverick) — native multimodal + tools, 10M context\n- `kimi-k2.5` — 200+ sequential tool calls without drift\n- `glm-5.1` — long-horizon agentic coding (8+ hours continuous)\n- `mistral-small4` — unified model, 6B active, fast\n- `granite4` — enterprise-grade tool calling\n- `phi-4-mini` — compact with function calling support\n- `deepseek-r1` — strong reasoning + tool use\n\n**llama.cpp** (GGUF format):\n- `bartowski/Gemma-4-31B-IT-GGUF` — best open weight for agents\n- `bartowski/Qwen3.5-32B-Instruct-GGUF` — stable tool calling\n- `bartowski/Llama-4-Scout-17B-GGUF` — 10M context, multimodal\n- `bartowski/GLM-5.1-40B-GGUF` — top open source coding\n- Any model with Jinja chat template + function calling support\n\n**MLX** (Apple Silicon via `mlx-community`):\n- `mlx-community/Gemma-4-31B-IT-4bit` — best performance/quality on Apple Silicon\n- `mlx-community/Qwen3.5-32B-Instruct-4bit` — stable tool calls\n- `mlx-community/Llama-4-Scout-17B-4bit` — multimodal + tools\n- `mlx-community/Mistral-Small-4-6B-4bit` — fast, 6B active\n\n**LM Studio**: All of the above models are available through LM Studio's model browser with native MCP host support.\n\n---\n\n## Configuration for Local Model Frameworks\n\n### LM Studio\n\nLM Studio has **native MCP host support** since v0.3.17.\n\n1. Open LM Studio \u003e Settings \u003e MCP\n2. Add a new MCP server with:\n   - **Command**: `/path/to/computer-use-mcp/.venv/bin/python3`\n   - **Args**: `[\"/path/to/computer-use-mcp/__main__.py\"]`\n3. Select a model with tool calling support:\n   - **Top picks**: Gemma 4 31B, Qwen 3.5/3.6, Llama 4 Scout, GLM-5.1, Mistral Small 4, Kimi K2.5\n4. The tools will appear in the chat interface\n\n### llama.cpp (Native MCP - March 2026+)\n\nllama.cpp merged native MCP client support in March 2026 (PR #18655), adding a full agentic loop with MCP server management in the WebUI.\n\n**Start llama-server with MCP:**\n\n```bash\n# Start with a top function-calling model (pick one)\nllama-server --jinja -fa -hf bartowski/Gemma-4-31B-IT-GGUF:Q4_K_M --port 8080\nllama-server --jinja -fa -hf bartowski/Qwen3.5-32B-Instruct-GGUF:Q4_K_M --port 8080\nllama-server --jinja -fa -hf bartowski/Llama-4-Scout-17B-GGUF:Q4_K_M --port 8080\n```\n\nThen in the llama.cpp WebUI:\n1. Go to **MCP Server Settings**\n2. Add this server with command: `/path/to/.venv/bin/python3 /path/to/__main__.py`\n3. The 33 tools will be available in the agentic loop\n\n**Via llama-mcp-server bridge:**\n\n```bash\nnpm install -g llama-mcp-server\n```\n\nConfigure in `claude_desktop_config.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"computer-use\": {\n      \"command\": \"/path/to/computer-use-mcp/.venv/bin/python3\",\n      \"args\": [\"/path/to/computer-use-mcp/__main__.py\"]\n    }\n  }\n}\n```\n\n**Supported models for tool calling:** Gemma 4, Qwen 3.5/3.6, Llama 4 Scout/Maverick, GLM-5.1, Kimi K2.5, Mistral Small 4, Llama 3.3, DeepSeek R1, Granite 4, Phi-4-mini, Hermes 3, Functionary v3.\n\n### Ollama\n\nOllama does not have native MCP support yet, but several bridge solutions work:\n\n**Option A: MCP-Bridge (recommended)**\n\n[MCP-Bridge](https://github.com/SecretiveShell/MCP-Bridge) acts as middleware between Ollama's OpenAI-compatible API and MCP servers.\n\n```bash\ngit clone https://github.com/SecretiveShell/MCP-Bridge.git\ncd MCP-Bridge\n```\n\nConfigure `config.json`:\n\n```json\n{\n  \"inference_server\": {\n    \"base_url\": \"http://localhost:11434/v1\",\n    \"api_key\": \"ollama\"\n  },\n  \"mcp_servers\": {\n    \"computer-use\": {\n      \"command\": \"/path/to/computer-use-mcp/.venv/bin/python3\",\n      \"args\": [\"/path/to/computer-use-mcp/__main__.py\"]\n    }\n  }\n}\n```\n\n**Option B: ollama-mcp-bridge**\n\n```bash\ngit clone https://github.com/patruff/ollama-mcp-bridge.git\ncd ollama-mcp-bridge\nnpm install \u0026\u0026 npm run build\n```\n\nAdd the computer-use server to the bridge config.\n\n**Recommended Ollama models (April 2026):**\n- `gemma4:31b` — best sub-32B for agents, native function calling\n- `qwen3.5:32b` — most stable tool calling\n- `llama4:scout` — 10M context, multimodal + tools\n- `kimi-k2.5` — 200+ sequential tool calls without drift\n- `glm-5.1` — long-horizon agentic (8+ hours continuous)\n- `mistral-small4` — fast, 6B active params\n- `granite4` — enterprise tool calling\n\n### MLX / Apple Silicon\n\nFor Apple Silicon Macs, use **vLLM-MLX** for optimized local inference with MCP bridge:\n\n**Install vLLM-MLX:**\n\n```bash\npip install git+https://github.com/waybarrios/vllm-mlx.git\n```\n\n**Start the inference server:**\n\n```bash\n# Pick a model (top recommendations for tool calling)\nvllm-mlx serve mlx-community/Gemma-4-31B-IT-4bit --port 8000\nvllm-mlx serve mlx-community/Qwen3.5-32B-Instruct-4bit --port 8000\nvllm-mlx serve mlx-community/Llama-4-Scout-17B-4bit --port 8000\n```\n\n**Connect via MCP-Bridge:**\n\n```json\n{\n  \"inference_server\": {\n    \"base_url\": \"http://localhost:8000/v1\",\n    \"api_key\": \"not-needed\"\n  },\n  \"mcp_servers\": {\n    \"computer-use\": {\n      \"command\": \"/path/to/computer-use-mcp/.venv/bin/python3\",\n      \"args\": [\"/path/to/computer-use-mcp/__main__.py\"]\n    }\n  }\n}\n```\n\n**Performance:** M4 Max achieves ~402 tokens/sec on small models, ~1112 tokens/sec with continuous batching.\n\n**Alternative:** [oMLX](https://github.com/jundot/omlx) provides a macOS menu bar app with MCP tool integration.\n\n### vLLM\n\nvLLM has native MCP integration with GPU-optimized inference.\n\n```bash\npip install vllm\nvllm serve google/gemma-4-31b-it --port 8000       # or any tool-calling model\nvllm serve Qwen/Qwen3.5-32B-Instruct --port 8000   # stable tool calling\nvllm serve meta-llama/Llama-4-Scout-17B --port 8000 # multimodal + tools\n```\n\nConnect via MCP-Bridge using `http://localhost:8000/v1` as the base URL.\n\n### Generic OpenAI-Compatible API\n\nAny service exposing an OpenAI-compatible API (local or remote) can use this server through [MCP-Bridge](https://github.com/SecretiveShell/MCP-Bridge):\n\n1. Start your inference server (Ollama, llama.cpp, vLLM, MLX, TGI, etc.)\n2. Point MCP-Bridge at it with the `base_url`\n3. Add this server to MCP-Bridge's `mcp_servers` config\n4. MCP-Bridge intercepts API requests, enriches them with tool definitions, executes tool calls, and returns results\n\n---\n\n## Tool Reference\n\n### Batch Operations — `computer_batch`\n\nExecute multiple actions in a single call to eliminate round-trip latency:\n\n```json\n{\n  \"actions\": [\n    {\"action\": \"left_click\", \"coordinate\": [100, 200]},\n    {\"action\": \"type\", \"text\": \"Hello, world!\"},\n    {\"action\": \"key\", \"text\": \"Return\"},\n    {\"action\": \"wait\", \"duration\": 1},\n    {\"action\": \"screenshot\"}\n  ]\n}\n```\n\n**Supported actions:** `key`, `type`, `mouse_move`, `left_click`, `left_click_drag`, `right_click`, `middle_click`, `double_click`, `triple_click`, `scroll`, `hold_key`, `screenshot`, `cursor_position`, `left_mouse_down`, `left_mouse_up`, `wait`\n\n### Mouse Tools\n\n| Tool | Parameters | Description |\n|------|-----------|-------------|\n| `left_click` | `coordinate: [x, y]` | Left-click at coordinates |\n| `right_click` | `coordinate: [x, y]` | Right-click (context menu) |\n| `middle_click` | `coordinate: [x, y]` | Middle-click (scroll wheel) |\n| `double_click` | `coordinate: [x, y]` | Double-click (select word) |\n| `triple_click` | `coordinate: [x, y]` | Triple-click (select line) |\n| `mouse_click` | `x, y, button, click_count` | General click with full control |\n| `mouse_move` | `x, y` or `coordinate: [x, y]` | Move cursor without clicking |\n| `mouse_drag` | `start_coordinate, coordinate` | Drag with 20-step interpolation |\n| `left_mouse_down` | (none) | Press and hold left button |\n| `left_mouse_up` | (none) | Release left button |\n| `scroll` | `coordinate, scroll_direction, scroll_amount` | Directional scroll (up/down/left/right) |\n| `mouse_scroll` | `amount, x, y` | Scroll wheel (positive=up, negative=down) |\n\n### Keyboard Tools\n\n| Tool | Parameters | Description |\n|------|-----------|-------------|\n| `key` | `text: \"cmd+c\"`, `repeat` | Unified key press with modifiers joined by `+` |\n| `hold_key` | `text: \"shift\"`, `duration` | Hold key for N seconds then release |\n| `keyboard_type` | `text` | Type text character by character (Unicode) |\n| `keyboard_press` | `key` | Press a single named key |\n| `keyboard_hotkey` | `keys: [\"cmd\", \"c\"]` | Press key combination as array |\n\n**Supported keys:** `return`, `tab`, `space`, `delete`, `escape`, arrows (`left`, `right`, `up`, `down`), `home`, `end`, `pageup`, `pagedown`, `f1`-`f12`, `a`-`z`, `0`-`9`, symbols.\n\n**Modifiers:** `cmd`/`command`, `shift`, `alt`/`option`, `ctrl`/`control`, `fn`\n\n### Screenshot \u0026 Display Tools\n\n| Tool | Parameters | Description |\n|------|-----------|-------------|\n| `take_screenshot` | `region` (optional), `max_dimension` | Capture screen as base64 PNG with coordinate metadata |\n| `zoom` | `region: [x0, y0, x1, y1]` | High-res crop of last screenshot (for reading small text) |\n| `switch_display` | `display` | Switch active monitor for screenshots. Use `\"auto\"` for main. |\n| `list_displays` | (none) | Enumerate all connected displays |\n\n### Other Tools\n\n| Tool | Parameters | Description |\n|------|-----------|-------------|\n| `read_clipboard` | (none) | Read clipboard text |\n| `write_clipboard` | `text` | Write text to clipboard |\n| `get_active_window` | (none) | Frontmost window app, title, position, size |\n| `list_windows` | (none) | All visible windows |\n| `get_screen_info` | (none) | Screen dimensions, Retina scale, accessibility status |\n| `get_cursor_position` | (none) | Current cursor coordinates |\n| `open_application` | `name` or `app` | Launch macOS app by name |\n| `wait` | `duration` | Pause for N seconds (0-100) |\n| `run_shell_command` | `command`, `timeout` | Execute shell command |\n| `request_access` | `apps[], reason` | Register apps for session access control |\n| `list_granted_applications` | (none) | List currently granted apps |\n\n---\n\n## Architecture\n\n```\ncomputer-use-mcp/\n├── __main__.py                    # Entry point (python -m or direct)\n├── __init__.py                    # Package metadata\n├── pyproject.toml                 # Dependencies \u0026 build config\n├── .mcp.json                     # Universal MCP client config\n└── server/\n    ├── __init__.py                # Re-exports all tool modules\n    ├── computer_use_server.py     # MCP Server class, tool registry, stdio transport\n    └── tools/\n        ├── __init__.py            # Exports all tool getters/handlers\n        ├── access_tools.py        # request_access, list_granted_applications\n        ├── batch_tools.py         # computer_batch (action orchestrator)\n        ├── clipboard_tools.py     # read/write clipboard (NSPasteboard)\n        ├── display_tools.py       # switch_display, zoom, list_displays\n        ├── keyboard_tools.py      # key, hold_key, type, press, hotkey (Quartz)\n        ├── mouse_tools.py         # 12 mouse tools (Quartz CGEvent)\n        ├── screen_tools.py        # screen info, cursor position (Quartz)\n        ├── screenshot_tools.py    # screenshot capture (mss + PIL)\n        ├── system_tools.py        # open app, wait, shell command\n        └── window_tools.py        # active window, list windows (Quartz + AppKit)\n```\n\n### How It Works\n\n1. **Transport**: stdio (JSON-RPC 2.0 over stdin/stdout)\n2. **Tool Registry**: `ComputerUseMCPServer` collects tools from 10 category modules, maps tool names to handlers\n3. **Input Simulation**: macOS Quartz `CGEvent` API for mouse/keyboard events posted to `kCGHIDEventTap`\n4. **Screenshots**: `mss` library for fast capture, PIL for resizing, base64 encoding\n5. **Coordinate System**: All tools use **logical screen coordinates** (Retina-aware). The server handles physical-to-logical scaling automatically.\n\n### Coordinate Mapping\n\nScreenshots include metadata for mapping image pixels to screen coordinates:\n\n```\nclick_x = (pixel_x / image_width) * logical_screen_width\nclick_y = (pixel_y / image_height) * logical_screen_height\n```\n\nOn Retina displays, logical coordinates differ from physical pixels. The server handles this transparently.\n\n---\n\n## Troubleshooting\n\n### \"Accessibility permission not granted\"\n\nGo to **System Settings \u003e Privacy \u0026 Security \u003e Accessibility** and add your terminal/IDE app.\n\n### Server fails to start\n\nEnsure you're using the venv Python (not system Python):\n\n```bash\n/path/to/computer-use-mcp/.venv/bin/python3 __main__.py\n```\n\n### Mouse/keyboard tools return errors but screenshots work\n\nScreenshot capture doesn't need accessibility permission, but input simulation does. Grant accessibility access to the process running the server.\n\n### \"ModuleNotFoundError: No module named 'server'\"\n\nThe `__main__.py` adds its directory to `sys.path` automatically. If running as a module (`python -m computer_use`), set the `cwd` to the parent directory of `computer_use/`.\n\n### Multi-monitor: wrong screen captured\n\nUse `list_displays` to see all monitors, then `switch_display` to select the correct one. Use `switch_display(\"auto\")` to reset.\n\n---\n\n## Contributing\n\nContributions are welcome! This server is designed to be extensible:\n\n1. Add new tools by creating a file in `server/tools/`\n2. Define `get_*_tools()` and `handle_*_tool()` functions\n3. Register in `server/computer_use_server.py` tool_sources list\n4. Update `server/tools/__init__.py` exports\n\nPlease ensure new tools follow the existing patterns for error handling and JSON response format.\n\n---\n\n## License\n\nMIT License - see [LICENSE](LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsyedazharmbnr1%2Fcomputer-use-mcp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsyedazharmbnr1%2Fcomputer-use-mcp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsyedazharmbnr1%2Fcomputer-use-mcp/lists"}