{"id":28718729,"url":"https://github.com/ota-tech-ai/web-agent-protocol","last_synced_at":"2025-06-15T05:03:31.487Z","repository":{"id":293889030,"uuid":"985409238","full_name":"OTA-Tech-AI/web-agent-protocol","owner":"OTA-Tech-AI","description":"🌐Web Agent Protocol (WAP) - Record and replay user interactions in the browser with MCP support","archived":false,"fork":false,"pushed_at":"2025-06-03T16:01:57.000Z","size":5588,"stargazers_count":447,"open_issues_count":3,"forks_count":78,"subscribers_count":58,"default_branch":"main","last_synced_at":"2025-06-15T05:03:30.331Z","etag":null,"topics":["ai-agents","ai-tools","browser-automation","browser-use","llm","mcp","mcp-server","modelcontextprotocol","playwright","python","record-replay","wap","web-agent-protocol","web-agents"],"latest_commit_sha":null,"homepage":"https://otatech.ai","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OTA-Tech-AI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-17T17:47:15.000Z","updated_at":"2025-06-15T05:01:44.000Z","dependencies_parsed_at":"2025-05-24T03:35:58.697Z","dependency_job_id":null,"html_url":"https://github.com/OTA-Tech-AI/web-agent-protocol","commit_stats":null,"previous_names":["ota-tech-ai/webagentprotocol","ota-tech-ai/web-agent-protocol"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/OTA-Tech-AI/web-agent-protocol","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OTA-Tech-AI%2Fweb-agent-protocol","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OTA-Tech-AI%2Fweb-agent-protocol/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OTA-Tech-AI%2Fweb-agent-protocol/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OTA-Tech-AI%2Fweb-agent-protocol/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OTA-Tech-AI","download_url":"https://codeload.github.com/OTA-Tech-AI/web-agent-protocol/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OTA-Tech-AI%2Fweb-agent-protocol/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259924679,"owners_count":22932782,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-agents","ai-tools","browser-automation","browser-use","llm","mcp","mcp-server","modelcontextprotocol","playwright","python","record-replay","wap","web-agent-protocol","web-agents"],"created_at":"2025-06-15T05:03:30.676Z","updated_at":"2025-06-15T05:03:31.434Z","avatar_url":"https://github.com/OTA-Tech-AI.png","language":"Python","readme":"\u003c!-- markdownlint-disable first-line-h1 --\u003e\n\u003c!-- markdownlint-disable html --\u003e\n\u003c!-- markdownlint-disable no-duplicate-header --\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"chrome-extension/assets/beholder-tool-kit-long.png\" width=\"100%\" alt=\"OTA-tool-kits\" style=\"border-radius: 10px;\" /\u003e\n\u003c/div\u003e\n\u003cbr\u003e\n\u003cdiv align=\"center\" style=\"line-height: 1;\"\u003e\n  \u003ca href=\"https://www.otatech.ai/\"\u003e\u003cimg alt=\"Homepage\"\n\tsrc=\"https://img.shields.io/badge/Visit-otatech.ai-blue\"/\u003e\u003c/a\u003e\n  \u003ca href=\"https://huggingface.co/OTA-AI/OTA-v1\"\u003e\u003cimg alt=\"Hugging Face\"\n\tsrc=\"https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-OTA%20AI-ffc107?color=ffc107\u0026logoColor=white\"/\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/OTA-Tech-AI/webagentprotocol/blob/main/LICENSE\"\u003e\u003cimg alt=\"Code License\"\n\tsrc=\"https://img.shields.io/badge/Code_License-MIT-f5de53?\u0026color=f5deff\"/\u003e\u003c/a\u003e\n  \u003cbr\u003e\u003cbr\u003e\u003cbr\u003e\n\u003c/div\u003e\n\n# Web Agent Protocol\n\n## Overview\n\nThe Web Agent Protocol (WAP) is a standardized framework designed to enable seamless interaction between users, web agents, and browsers by recording and replaying browser actions. It separates the concerns of action recording and execution, allowing for efficient automation and reusability. The Python SDK for WAP implements the full specification, making it easy to:\n\n1. **Collect** user‑interaction data with the [OTA‑WAP Chrome extension](https://github.com/OTA-Tech-AI/webagentprotocol/tree/main/chrome-extension).\n2. **Convert** the raw event stream into either **_exact‑replay_** or **_smart‑replay_** action lists.\n3. **Convert** recorded actions into **_MCP_** servers for reuse by any agent or user\n4. **Replay** those lists using the **_WAP-Replay_** protocol to ensure accurate browser operations.\n\n### WAP FULL DEMO\n\n[![Watch the video](https://img.youtube.com/vi/joh9FXJfnwk/0.jpg)](https://www.youtube.com/watch?v=joh9FXJfnwk)\n\n### Without WAP\n![image](https://github.com/user-attachments/assets/843ea9da-45c0-48e9-8a25-44f5bfb31786)\n\n### WAP Record\n![image](https://github.com/user-attachments/assets/3d041f56-9e76-4b61-9b56-0686070723a3)\n\n### WAP Replay\n![image](https://github.com/user-attachments/assets/e13ca7c7-3cc1-4952-8a79-3bd1e9e98580)\n\n## Example using WAP\n![image](https://github.com/user-attachments/assets/ccb7387b-0677-498c-b4ad-a10590e37e27)\n\n## Setup\nInstall the dependencies with the following command:\n\nCreate a conda env\n\n```bash\nconda create -n WAP python=3.11\n```\n\nActivate the conda env\n\n```bash\nconda activate WAP\n```\n\nInstall the dependencies\n\n```bash\npip install -r requirements.txt\n```\n\nSetup your repo source path:\n```\nset PYTHONPATH=C:/path/to/webagentprotocol # for Windows\nexport PYTHONPATH=/path/to/webagentprotocol # for Linux\n```\n\nCreate **.env** file under the repo root directory with your own API keys:\n```\nOPENAI_API_KEY=sk-proj-...\nDEEPSEEK_API_KEY=sk-...\n```\n\n## Record\n\n### WAP record extension\nPlease refer to [OTA‑WAP Chrome Extension](https://github.com/OTA-Tech-AI/webagentprotocol/tree/main/chrome-extension) to setup action capturer in your Chrome browser.\n\n### Start data‑collection server\n\nRun the following command to start the server to collect data from the extension:\n```bash\npython action_collect_server.py\n```\n**Once the server is up, you can start to record from the page using WAP Chrome extension.**\n\nThe server listens on http://localhost:4934/action-data by default, please make sure the Host and Port in the extension settings match this server config.\nEach session will be saved to:\n\n```bash\ndata/YYYYMMDD/taskid/summary_event_\u003ctimestamp\u003e.json\n```\n\nAn example of the formatted data which you will received in the WAP backend server is like:\n\n```json\n{\n  \"taskId\": \"MkCAhQsHgXn7YgaK\",\n  \"type\": \"click\",\n  \"actionTimestamp\": 1746325231479,\n  \"eventTarget\": {\n    \"type\": \"click\",\n    \"target\": \"\u003ca ota-use-interactive-target=\\\"1\\\" data-ordinal=\\\"3\\\" href=\\\"https://www.allrecipes.com/recipe/68925/cheesy-baked-salmon/\\\" data-tax-levels=\\\"\\\" data-doc-id=\\\"6592066\\\" class=\\\"comp mntl-card-list-card--extendable mntl-universal-card mntl-document-card mntl-card card card--no-image\\\" id=\\\"mntl-card-list-card--extendable_3-0\\\"\u003e\\n\u003cdiv class=\\\"loc card__top\\\"\u003e\u003cdiv class=\\\"card__media mntl-image card__media universal-image__container\\\"\u003e...\",\n    \"targetId\": \"mntl-card-list-card--extendable_3-0\",\n    \"targetClass\": \"comp mntl-card-list-card--extendable mntl-universal-card mntl-document-card mntl-card card card--no-image\"\n  },\n  \"allEvents\": {},\n  \"pageHTMLContent\": \"\u003cheader data-tracking-container=\\\"true\\\" data-collapsible=\\\"true\\\" class=\\\"comp header mntl-header mntl-header--magazine mntl-header--open-search-bar mntl-header--myr\\\" id=\\\"header_1-0\\\"\u003e\u003ca data-tracking-container=\\\"true\\\" id=\\\"mntl-skip-to-content_1-0\\\" class=\\\"mntl-skip-to-content mntl-text-link\\\" rel=\\\"nocaes\\\" href=\\\"#main\\\"\u003e\u003c/a\u003e\u003cdiv class=\\\"mntl-header__menu-top\\\"\u003e...\"\n}\n```\n\n\n## Generate replay lists\n\n| Mode                                               | Command                                                                                                                                                                |\n| -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| **Exact replay** – exactly reproduce every action | `python wap_replay/generate_exact_replay_list.py --data_dir_path data/\u003cdate\u003e/\u003ctask_id\u003e --output_dir_path data_processed/exact_replay` |\n| **Smart replay** – condensed goal‑oriented steps   | `python wap_replay/generate_smart_replay_list.py --data_dir_path data/\u003cdate\u003e/\u003ctask_id\u003e --output_dir_path data_processed/smart_replay` |\n\nReplace **\u003ctask_id\u003e** with the folder produced by the extension\n(e.g. em3h6UBDZykz0gnH).\n\nOutput structure:\n```bash\ndata_processed/smart_replay/\n ├─ subgoals_\u003ctask_id\u003e/                     # intermediate prompts \u0026 replies\n └─ wap_smart_replay_list_\u003ctask_id\u003e.json   # final smart replay list for the agent\n\ndata_processed/exact_replay/\n └─ wap_smart_replay_list_\u003ctask_id\u003e.json   # final exact replay list for the agent\n```\n\n## Replay\n```bash\npython run_replay.py --model-provider openai --wap_replay_list data_processed/exact_replay/wap_exact_replay_list_\u003ctask_id\u003e.json --max-concurrent 1\n```\nFor **smart-replay**, replace the path with a smart‑replay JSON to test this mode.\n\n## Convert to MCP Server\n\n```bash\npython wap_replay\\generate_mcp_server.py --task_id \u003ctask_id\u003e\n```\n\nconverted MCP servers will be located under ``` mcp_servers ``` folder\n\n## Replay with MCP\n\nYou would need 2 terminals to replay with MCP. In the first termnial\n```bash\npython wap_service.py\n```\n\nIn the second termnial\n```bash\npython mcp_client.py\n```\n\nThen enter your prompt in the second terminal\n\n```bash\nexample: find a top rated keyboard on amazon.ca using smart replay\n```\n\n## Replay with our Desktop App\n\nWe provide out-of-box desktop app for running replay lists. It is easy to install and you don't need any extra steps for setup and deployments. Visit [WAP Replay Tool releases](https://github.com/OTA-Tech-AI/web-agent-protocol/releases) for more details.\n\n\u003cimg src=\"assets/wap_replay_tool_demo.gif\" alt=\"WAP Replay Tool Demo GIF\" width=\"500\"/\u003e\n\n\n## Troubleshooting\n\n**ModuleNotFoundError** – run commands from the project root or export PYTHONPATH=. (set PYTHONPATH=. for Windows).\n\n“no task‑start file” – ensure the extension recorded a full session;\nthe generators require exactly one task-start and one task-finish record.\n\n## Acknowledgement\n\nBrowser-Use: https://github.com/browser-use/browser-use\n\nMCP: https://github.com/modelcontextprotocol/python-sdk\n\nDOM Extension: https://github.com/kdzwinel/DOMListenerExtension\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fota-tech-ai%2Fweb-agent-protocol","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fota-tech-ai%2Fweb-agent-protocol","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fota-tech-ai%2Fweb-agent-protocol/lists"}