{"id":51308008,"url":"https://github.com/sshh12/windows-computer-use-mcp","last_synced_at":"2026-07-01T01:30:43.396Z","repository":{"id":363223048,"uuid":"1262364028","full_name":"sshh12/windows-computer-use-mcp","owner":"sshh12","description":"Full computer-use for Windows over MCP: multi-monitor screen capture, scan-code keyboard/mouse input, video recording, and a play-test loop. Works in Claude Code (incl. headless -p), Claude Desktop, or any MCP client.","archived":false,"fork":false,"pushed_at":"2026-06-08T02:04:37.000Z","size":92,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-08T02:20:15.723Z","etag":null,"topics":["claude-code","computer-use","mcp","windows"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sshh12.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-07T22:54:14.000Z","updated_at":"2026-06-08T02:04:40.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/sshh12/windows-computer-use-mcp","commit_stats":null,"previous_names":["sshh12/windows-computer-use-mcp"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/sshh12/windows-computer-use-mcp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sshh12%2Fwindows-computer-use-mcp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sshh12%2Fwindows-computer-use-mcp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sshh12%2Fwindows-computer-use-mcp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sshh12%2Fwindows-computer-use-mcp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sshh12","download_url":"https://codeload.github.com/sshh12/windows-computer-use-mcp/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sshh12%2Fwindows-computer-use-mcp/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34989787,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-30T02:00:05.919Z","response_time":92,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["claude-code","computer-use","mcp","windows"],"created_at":"2026-07-01T01:30:42.790Z","updated_at":"2026-07-01T01:30:43.385Z","avatar_url":"https://github.com/sshh12.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# windows-computer-use-mcp\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/unreal-playtest.gif\" width=\"820\" alt=\"Claude play-testing an Unreal Engine project — entering Play-In-Editor and running up a ramp, driven through the MCP, with the full editor (toolbar, viewport, Outliner) visible the whole time\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\u003cem\u003eClaude play-testing a live \u003cstrong\u003eUnreal Engine\u003c/strong\u003e project — entering Play-In-Editor and running up a ramp, driven entirely through this MCP. The full editor stays on screen, so you can see it really is the desktop app.\u003c/em\u003e\u003c/p\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eHow that clip was made\u003c/strong\u003e — one \u003ccode\u003eplay\u003c/code\u003e call\u003c/summary\u003e\n\nA timed input script run at a cadence while the window is recorded (mouse-look is relative, keys are hardware scan codes, so the game responds). The raw input was exactly:\n\n```python\nplay(\n    target=\"window:MyProject - Unreal Editor\",\n    script=\"\"\"\n        lmb 557 460\n        wait 0.4\n        look 150 0\n        wait 0.15\n        look 150 0\n        wait 0.2\n        down w\n        wait 1.7\n        tap space\n        wait 0.7\n        up w\n        wait 0.4\n    \"\"\",\n)\n```\n\nThat is: click the viewport to capture the mouse (`lmb`), turn to face the ramp (`look`, relative pixels), run forward up it (`down w` … `up w`), and jump at the top (`tap space`).\n\n\u003c/details\u003e\n\nA [Model Context Protocol](https://modelcontextprotocol.io) server that gives a Claude agent\n**full control of the local Windows desktop** — native screen capture, low-level input\ninjection, video recording, and a play-test loop for driving games and apps.\n\nUnlike Anthropic's sandboxed computer-use tool, this runs **on the machine it controls**: it\nreads the actual current displays (no resolution requests), is **multi-monitor and per-monitor\nDPI aware**, and injects input via `SendInput` **scan codes** so it works in games that ignore\nsynthetic virtual-key events. Built for full Claude control — no security gating.\n\n## Why this exists\n\nAnthropic's official [computer use in the Claude Code CLI](https://code.claude.com/docs/en/computer-use)\nis a **macOS-only research preview** — Pro/Max only, interactive sessions only (not available with\nthe `-p` flag). The cross-platform alternative is the Claude Desktop app. There is **no official,\nnon-Desktop computer-use for Windows**: nothing you can drive headlessly from `claude -p`, from the\nAPI, or wire into an agent over MCP.\n\nThis server fills that gap. It's a standard MCP server, so it works on Windows in Claude Code\n(interactive **and** `-p`), in Claude Desktop, or from any MCP client / custom agent — with no plan\ngating — and it's tuned for what a Windows agent actually needs that the sandboxed cloud tool can't\ndo: real multi-monitor capture, per-window GPU capture, game-grade input, and play-testing.\n\n## Tools\n\n| Tool | What it does |\n|---|---|\n| **`screenshot`** | See the screen: whole desktop, a `display:N`, a window (even occluded/DirectX via PrintWindow), or a region. Downscaled inline image + a coordinate frame for clicks. |\n| **`act`** | Do input, batched: `left_click`, `type`, `key`, `scroll`, `drag`, `hold_key`, `paste`, `click_element` (UIA, no pixels), `mouse_move_relative` (game look), … Coordinates are in the last screenshot's image space; the server maps them to physical pixels. |\n| **`record`** | Record N seconds → a single timestamped **frame montage** (not N images) + an mp4. Judge motion/animation/stutter. |\n| **`play`** | Drive a **timed input script** at a cadence while recording (scan codes + relative mouse). `probe`/`until` read telemetry per sample and stop early — the closed loop for play-testing. |\n| **`window`** | Find / focus / close / **read** (`get_text` via UI Automation + OCR) / `click_element` controls with **no screenshot** — token-cheap. |\n| **`process`** | Launch (incl. `shell:true` for URLs / `ms-settings:` / Store apps), kill, wait, run a shell command (real stdout), and wait on readiness (`wait_for_window`, `wait_for_file`). |\n| **`system`** | Monitor layout + DPI/scale, cursor position, clipboard get/set, and viewport management. |\n\n### Coordinates \u0026 multi-monitor\n\nCoordinates are physical pixels in virtual-desktop space (primary monitor's top-left is `(0,0)`;\nmonitors to the left/above are negative). You click in the **image space** of the last\n`screenshot`; the server maps that back to physical pixels (handling downscale, per-monitor\noffset, and DPI). Every screenshot returns a `capture_id`; `act` errors loudly if you click\nagainst a stale frame instead of mis-clicking. Use `system displays` to see the layout, then\ntarget a specific monitor with `display:0` / `display:primary|left|right`.\n\n## Install\n\n### As a Claude Code / Desktop plugin (recommended)\n\n```\n/plugin marketplace add sshh12/claude-plugins\n/plugin install windows-computer-use@shrivu-plugins\n```\n\nThe plugin bootstraps a Python virtual environment and installs this package from GitHub on\nfirst run, then starts the MCP server automatically.\n\n### Standalone (project-local MCP)\n\nRequires Python 3.10+ and (for video) [ffmpeg](https://ffmpeg.org) on `PATH`.\n\n```bash\npip install git+https://github.com/sshh12/windows-computer-use-mcp\n```\n\nThen add to your MCP client config (e.g. a project `.mcp.json`):\n\n```json\n{\n  \"mcpServers\": {\n    \"windows-computer-use\": {\n      \"command\": \"python\",\n      \"args\": [\"-m\", \"windows_computer_use\"]\n    }\n  }\n}\n```\n\n## Development\n\n```bash\npython -m venv .venv\n.venv\\Scripts\\python.exe -m pip install -e .\n.venv\\Scripts\\python.exe tests\\smoke_engine.py    # capture/input/display engine\n.venv\\Scripts\\python.exe tests\\smoke_server.py     # assembled MCP tool surface\n```\n\n`MCP_OUTPUT_DIR` overrides where screenshots/video are written (default: a client root →\n`~/Pictures/windows-computer-use` → `%TEMP%`).\n\n### Debugging\n\nSet `WCU_DEBUG_HTML_DIR` to a directory and the server writes a per-session\n`session_\u003cstamp\u003e.html` that pretty-prints every tool call — arguments, result text, and the\nreturned screenshots inline — so you can replay exactly what the agent saw and did:\n\n\u003cimg src=\"https://github.com/user-attachments/assets/4395906b-cfe2-4e25-b560-261a9ebc782d\" width=\"820\" alt=\"Per-session debug HTML dump showing tool calls, arguments, results, and inline screenshots\"\u003e\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsshh12%2Fwindows-computer-use-mcp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsshh12%2Fwindows-computer-use-mcp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsshh12%2Fwindows-computer-use-mcp/lists"}