{"id":30827667,"url":"https://github.com/g4-api/uia-peek","last_synced_at":"2025-09-18T00:52:33.546Z","repository":{"id":313375664,"uuid":"1049915170","full_name":"g4-api/uia-peek","owner":"g4-api","description":"Windows UI Automation inspector \u0026 recorder. Peek element hierarchies, capture keyboard/mouse with UI context, and stream in real time via SignalR — or use via REST/CLI.","archived":false,"fork":false,"pushed_at":"2025-09-12T15:34:12.000Z","size":101,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-12T18:09:21.276Z","etag":null,"topics":["dotnet","rest-api","signalr","ui-inspector","uiautomation","user32","windows"],"latest_commit_sha":null,"homepage":"https://github.com/g4-api/uia-peek","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/g4-api.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-03T17:20:04.000Z","updated_at":"2025-09-12T15:34:01.000Z","dependencies_parsed_at":"2025-09-05T17:46:06.490Z","dependency_job_id":"f96a0d91-9c22-4d9b-9f26-fab30f45322a","html_url":"https://github.com/g4-api/uia-peek","commit_stats":null,"previous_names":["g4-api/uia-peek"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/g4-api/uia-peek","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/g4-api%2Fuia-peek","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/g4-api%2Fuia-peek/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/g4-api%2Fuia-peek/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/g4-api%2Fuia-peek/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/g4-api","download_url":"https://codeload.github.com/g4-api/uia-peek/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/g4-api%2Fuia-peek/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":275691192,"owners_count":25510500,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-17T02:00:09.119Z","response_time":84,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dotnet","rest-api","signalr","ui-inspector","uiautomation","user32","windows"],"created_at":"2025-09-06T13:35:21.836Z","updated_at":"2025-09-18T00:52:33.529Z","avatar_url":"https://github.com/g4-api.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# UiaPeek — UI Automation Peek \u0026 Record Tool\n\n[![Build \u0026 Release](https://github.com/g4-api/uia-peek/actions/workflows/release-pipline.yml/badge.svg)](https://github.com/g4-api/uia-peek/actions/workflows/release-pipline.yml)\n\n## Table of Contents\n\n* [What is UiaPeek?](#what-is-uiapeek)\n* [Quick Start](#quick-start)\n  * [Download and Run](#download-and-run)\n  * [Verify the Service](#verify-the-service)\n* [Comprehensive Usage](#comprehensive-usage)\n  * [Command-Line Usage](#command-line-usage)\n  * [cURL API Usage (PeekController)](#curl-api-usage-peekcontroller)\n  * [SignalR Hub Usage](#signalr-hub-usage)\n    * [Python Example](#python-example)\n    * [JavaScript Example](#javascript-example)\n    * [C# Example](#c-example)\n* [UiaPeek Path Finder — Quick Start](#uiapeek-path-finder--quick-start)\n  * [Requirements](#requirements)\n  * [Launch](#launch)\n  * [Start peeking](#start-peeking)\n  * [Adjust update speed](#adjust-update-speed)\n  * [Copy the locator](#copy-the-locator)\n  * [Stop peeking](#stop-peeking)\n  * [Typical workflow](#typical-workflow)\n  * [Tips](#tips)\n  * [Troubleshooting](#troubleshooting)\n  * [Keyboard \u0026 accessibility](#keyboard--accessibility)\n  * [Exit](#exit)\n* [License](#license)\n\n---\n\n## What is UiaPeek?\n\n**UiaPeek** is a Windows UI Automation **inspection and recording tool**. It provides the ability to:\n\n* **Peek**: Inspect UI elements at screen coordinates or the currently focused element and return their **ancestor chain** (like an XPath for Windows UI).\n* **Record**: Capture global keyboard and mouse events in real time, enriched with UI context, and broadcast them to connected clients.\n* **Expose**: Provide results via REST APIs, a SignalR hub stream, or a command-line interface.\n\nUse cases:\n\n* Debug and inspect Windows desktop applications to troubleshoot UI behavior and layout issues.\n* Create automated UI tests or implement advanced robotic process automation (RPA) workflows.\n* Recording user input with UI context for replay or analysis.\n* Real-time monitoring of UI and user actions.\n\n---\n\n## Quick Start\n\n\u003e **Requirements:** Windows OS, .NET 8 Runtime/SDK. Administrator privileges are recommended for global input hooks.\n\n### Download and Run\n\n1. Go to **Releases** and download the latest artifact for your platform:\n   👉 **Releases:** [https://github.com/g4-api/uia-peek/releases](https://github.com/g4-api/uia-peek/releases)\n2. Unzip the archive to a folder with write permissions (e.g., `C:\\Tools\\UiaPeek`).\n3. Run - as administrator - the executable (`UiaPeek.exe`) from that folder.\n\n### Verify the Service\n\nOnce running:\n\n* Open Swagger: `http://localhost:9955/swagger`\n* Health check:\n\n```bash\ncurl http://localhost:9955/api/v4/g4/ping\n# Output: Pong\n```\n\n\u003e **CORS tip:** If you call from a webview or browser, ensure your origin is allowed via `ORIGINS` env var or `AllowedOrigins` config. VS Code webviews start with `vscode-webview://` which is already handled.\n\n---\n\n## Comprehensive Usage\n\n### Command-Line Usage\n\n```bash\n# Peek at focused element\nuiapeek peek -f\n\n# Peek at specific coordinates\nuiapeek peek -x 100 -y 200\n```\n\n---\n\n### cURL API Usage (PeekController)\n\nThe **PeekController** provides REST endpoints to peek at UI elements.\n\n```bash\n# Peek at specific coordinates (x=250, y=300)\ncurl \"http://localhost:9955/api/v4/g4/peek?x=250\u0026y=300\"\n\n# Peek at the currently focused element\ncurl \"http://localhost:9955/api/v4/g4/peek?focused=true\"\n```\n\n**Sample JSON Response:**\n\n```json\n{\n    \"value\": {\n        \"chain\": {\n            \"locator\": \"/Desktop/Window[@Name='...']/Document[@Name='Text Area']\",\n            \"path\": [\n                {\n                    \"automationId\": \"Console Window\",\n                    \"className\": \"ConsoleWindowClass\",\n                    \"controlTypeId\": 50032,\n                    \"controlType\": \"Window\",\n                    \"isTopWindow\": true,\n                    \"isTriggerElement\": false,\n                    \"name\": \"...exe\",\n                    \"processId\": 31036\n                },\n                {\n                    \"automationId\": \"Text Area\",\n                    \"bounds\": {\n                        \"height\": 519,\n                        \"X\": 104,\n                        \"Y\": 104,\n                        \"width\": 993\n                    },\n                    \"controlTypeId\": 50030,\n                    \"controlType\": \"Document\",\n                    \"isTopWindow\": false,\n                    \"isTriggerElement\": true,\n                    \"machine\": {\n                        \"name\": \"DESKTOP-12345\",\n                        \"publicAddress\": \"172.23.32.1\"\n                    },\n                    \"name\": \"Text Area\",\n                    \"patterns\": [\n                        {\n                            \"id\": 10018,\n                            \"name\": \"LegacyIAccessible\"\n                        },\n                        {\n                            \"id\": 10014,\n                            \"name\": \"Text\"\n                        }\n                    ],\n                    \"processId\": 31036,\n                    \"runtimeId\": [\n                        42,\n                        590408,\n                        4,\n                        -1\n                    ]\n                }\n            ],\n            \"topWindow\": {\n                \"automationId\": \"Console Window\",\n                \"className\": \"ConsoleWindowClass\",\n                \"controlTypeId\": 50032,\n                \"controlType\": \"Window\",\n                \"isTopWindow\": true,\n                \"isTriggerElement\": false,\n                \"name\": \"...exe\",\n                \"processId\": 31036\n            },\n            \"trigger\": \"Focus\"\n        },\n        \"event\": \"Down\",\n        \"timestamp\": 1757618366611,\n        \"type\": \"Keyboard\",\n        \"value\": {\n            \"scanCode\": 30,\n            \"virtualKey\": 65,\n            \"key\": \"a\"\n        }\n    }\n}\n```\n\n\u003e **Note:** The `path` array is ordered **top-down**. The **last element** in the path is always the **target element** (the trigger). This element includes **more metadata** than its ancestors to reduce payload size and avoid bloating.\n\n---\n\n### SignalR Hub Usage\n\n**Hub URL:** `http://localhost:9955/hub/v4/g4/peek`\n\n**Client → Server Methods**\n\n* `SendHeartbeat()`\n* `SendPeekAt({ x, y })`\n* `SendPeekFocused()`\n* `StartRecordingSession()`\n* `StopRecordingSession(sessionId)`\n\n**Server → Client Events**\n\n* `ReceiveHeartbeat` — heartbeat confirmation\n* `ReceivePeek` — ancestor chain result\n* `ReceiveRecordingEvent` — keyboard/mouse events with UI context\n\n\u003e **Note:** Recording events flow only after `StartRecordingSession()`.\n\n---\n\n#### Python Example\n\nUsing the community client **signalrcore**.\n\n```bash\npip install signalrcore\n```\n\n```python\nfrom signalrcore.hub_connection_builder import HubConnectionBuilder\n\nhub = (\n    HubConnectionBuilder()\n    .with_url(\"http://localhost:9955/hub/v4/g4/peek\")\n    .build()\n)\n\n# ---- Wire server -\u003e client callbacks ----\n\ndef on_heartbeat(payload):\n    print(\"Heartbeat:\", payload)\n\ndef on_peek(payload):\n    print(\"Peek:\", payload)\n\ndef on_recording_event(payload):\n    ev = payload.get(\"value\", {})\n    print(f\"RecordingEvent: type={ev.get('type')} event={ev.get('event')} value={ev.get('value')}\")\n\nhub.on(\"ReceiveHeartbeat\", on_heartbeat)\nhub.on(\"ReceivePeek\", on_peek)\nhub.on(\"ReceiveRecordingEvent\", on_recording_event)\n\nhub.start()\n\n# ---- Call client -\u003e server hub methods ----\nhub.send(\"SendHeartbeat\", [])\nhub.send(\"SendPeekAt\", [{\"xPos\": 250, \"yPos\": 300}])\nhub.send(\"SendPeekFocused\", [])\nhub.send(\"StartRecordingSession\", [])\n\ninput(\"Press \u003cEnter\u003e to stop...\\n\")\n# If you captured a session id from RecordingSessionStarted, you can stop it:\n# hub.send(\"StopRecordingSession\", [session_id])\n\nhub.stop()\n```\n\n---\n\n#### JavaScript Example\n\nUsing **@microsoft/signalr** (browser or Node.js).\n\n```bash\nnpm i @microsoft/signalr\n```\n\n```javascript\nimport * as signalR from \"@microsoft/signalr\";\n\nconst connection = new signalR.HubConnectionBuilder()\n    .withUrl(\"http://localhost:9955/hub/v4/g4/peek\")\n    .withAutomaticReconnect()\n    .build();\n\n// ---- Wire server -\u003e client callbacks ----\nconnection.on(\"ReceiveHeartbeat\", (payload) =\u003e {\n    console.log(\"Heartbeat:\", payload);\n});\n\nconnection.on(\"ReceivePeek\", (payload) =\u003e {\n    console.log(\"Peek chain:\", payload?.value);\n});\n\nconnection.on(\"ReceiveRecordingEvent\", (payload) =\u003e {\n    const ev = payload?.value || {};\n    console.log(\"RecordingEvent:\", ev.type, ev.event, ev.value);\n});\n\nasync function main() {\n    await connection.start();\n    await connection.invoke(\"SendHeartbeat\");\n    await connection.invoke(\"SendPeekAt\", { xPos: 250, yPos: 300 });\n    await connection.invoke(\"SendPeekFocused\");\n    await connection.invoke(\"StartRecordingSession\");\n}\n\nmain().catch(console.error);\n```\n\n---\n\n#### C# Example\n\nUsing **Microsoft.AspNetCore.SignalR.Client**.\n\n```bash\n# .NET CLI\ndotnet add package Microsoft.AspNetCore.SignalR.Client\n```\n\n```csharp\nusing Microsoft.AspNetCore.SignalR.Client;\nusing System;\nusing System.Threading.Tasks;\n\nclass Program\n{\n    static async Task Main()\n    {\n        var connection = new HubConnectionBuilder()\n            .WithUrl(\"http://localhost:9955/hub/v4/g4/peek\")\n            .WithAutomaticReconnect()\n            .Build();\n\n        // ---- Wire server -\u003e client callbacks ----\n        connection.On\u003cobject\u003e(\"ReceiveHeartbeat\", payload =\u003e\n        {\n            Console.WriteLine($\"Heartbeat: {payload}\");\n        });\n\n        connection.On\u003cobject\u003e(\"ReceivePeek\", payload =\u003e\n        {\n            Console.WriteLine(\"Peek chain received\");\n            Console.WriteLine(payload);\n        });\n\n        connection.On\u003cobject\u003e(\"ReceiveRecordingEvent\", payload =\u003e\n        {\n            Console.WriteLine($\"RecordingEvent: {payload}\");\n        });\n\n        await connection.StartAsync();\n        await connection.InvokeAsync(\"SendHeartbeat\");\n        await connection.InvokeAsync(\"SendPeekAt\", new { xPos = 250, yPos = 300 });\n        await connection.InvokeAsync(\"SendPeekFocused\");\n        await connection.InvokeAsync(\"StartRecordingSession\");\n\n        Console.WriteLine(\"Press \u003cEnter\u003e to exit...\");\n        Console.ReadLine();\n    }\n}\n```\n---\n\n## UiaPeek Path Finder — Quick Start\n\nUiaPeek Path Finder shows the UI-Automation path (an XPath-like locator) for whatever is currently under your mouse pointer. Hover any app control; the locator appears in the text box for copy/paste into your automation or QA tools.\n\n### Requirements\n\n* Windows desktop.\n* If you need to peek elevated apps, run Path Finder **as Administrator**.\n\n### Launch\n\n* Open **UiaPeek Path Finder v1.0** by running `UiaPeek.PathFinder.exe`.\n* You’ll see a title, a locator text box, a **Start/Stop** button, and a **Faster/Slower** slider.\n\n  ```none\n  ╭── UiaPeek — Title ────────────────────────────────────╮\n  │                                                       │\n  │  Locator                                              │\n  │  ┌─────────────────────────────────────────────────┐  │\n  │  │                                                 │  │\n  │  └─────────────────────────────────────────────────┘  │\n  │                                                       │\n  │  [ ▶ Start / ⬛ Stop]                                │\n  │                                                       │\n  │  Faster  ◄───────┈┈┈┈┈●┈┈┈┈────────►  Slower          │\n  │                                                       │\n  ╰───────────────────────────────────────────────────────╯\n  ```\n\n### Start peeking\n\n* Click **▶ Start** (or press **Alt+S**).\n* Move your mouse over the target application.\n* The locator appears and updates in the text box.\n\n### Adjust update speed\n\n* Use the **Faster ←→ Slower** slider to change refresh rate.\n* Faster ≈ \\~500 ms; Slower ≈ \\~3000 ms.\n* Faster feels live; slower reduces CPU and makes copying easier.\n\n### Copy the locator\n\n* Select the locator in the text box and press **Ctrl+C**.\n* Paste into your test or automation scripts.\n* If the text changes while selecting, press **⬛ Stop** first, then copy.\n\n### Stop peeking\n\n* Click **⬛ Stop** (or **Alt+S**) to pause tracking.\n\n### Typical workflow\n\n* Open the target app.\n* Start Path Finder and hover precisely over the desired control (button, textbox, menu item).\n* Adjust the slider if updates are too fast or too slow.\n* Stop, copy the locator, and paste where needed.\n\n### Tips\n\n* Be precise: tiny mouse movements can change the target element.\n* Hover the actionable sub-area (icon/text) if a widget is composite.\n* Keep the Path Finder window out of the target area to avoid peeking it.\n* Run as Administrator when inspecting admin-level windows.\n\n### Troubleshooting\n\n* **No updates**: ensure you pressed **Start**; try a slower rate; run as Administrator for elevated apps.\n* **Hard to copy**: press **Stop**, then copy; or slide toward **Slower**.\n* **Unexpected element**: re-hover the exact clickable region; complex controls may have nested elements.\n\n### Keyboard \u0026 accessibility\n\n* **Alt+S** toggles Start/Stop.\n* **Ctrl+C** copies the locator from the text box.\n\n### Exit\n\n* Close the window (X or Alt+F4). Any active tracking stops automatically.\n\n---\n\n## License\n\nMIT License. See `LICENSE` for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fg4-api%2Fuia-peek","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fg4-api%2Fuia-peek","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fg4-api%2Fuia-peek/lists"}