{"id":133072,"url":"https://github.com/12britz/awesome-computer-use","name":"awesome-computer-use","description":"139 open-source projects that let AI click, type, scroll, and generally make a mess of your desktop — so you don't have to.","projects_count":147,"last_synced_at":"2026-09-04T17:00:27.785Z","repository":{"id":346645374,"uuid":"1190929059","full_name":"12britz/awesome-computer-use","owner":"12britz","description":"139 open-source projects that let AI click, type, scroll, and generally make a mess of your desktop — so you don't have to.","archived":false,"fork":false,"pushed_at":"2026-03-24T19:38:58.000Z","size":18,"stargazers_count":7,"open_issues_count":2,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-27T13:05:44.944Z","etag":null,"topics":["awesome","awesome-computer","awesome-list"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/12britz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-24T18:55:22.000Z","updated_at":"2026-07-26T02:23:19.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/12britz/awesome-computer-use","commit_stats":null,"previous_names":["12britz/awesome-computer-use"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/12britz/awesome-computer-use","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/12britz%2Fawesome-computer-use","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/12britz%2Fawesome-computer-use/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/12britz%2Fawesome-computer-use/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/12britz%2Fawesome-computer-use/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/12britz","download_url":"https://codeload.github.com/12britz/awesome-computer-use/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/12britz%2Fawesome-computer-use/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":36698916,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-08-06T04:43:03.162Z","status":"online","status_checked_at":"2026-08-15T02:00:05.847Z","response_time":94,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"created_at":"2026-06-03T22:10:12.800Z","updated_at":"2026-09-04T17:00:27.785Z","primary_language":"Python","list_of_lists":false,"displayable":true,"categories":["Research Papers","Desktop Automation Libraries","Desktop Agents","Agent Memory","RPA Tools","OCR and Screen Reading","Browser Agents","Vision and Grounding Models","Sandbox and Containerization","Game-Playing Agents","Mobile Agents","Related Awesome Lists","Voice-Controlled Agents","Screen Capture and Recording","MCP Servers for Desktop","Agent Orchestration","Agent Testing and Evaluation","Training Data Tools","Browser Extension Agents","Desktop-Native AI Assistants","IDE-Based Agents","Screen Understanding"],"sub_categories":[],"readme":"# Awesome Computer Use [![Awesome](https://awesome.re/badge.svg)](https://awesome.re)\n\n\u003e Open-source projects that let AI click, type, scroll, and generally make a mess of your desktop — so you don't have to.\n\nYour AI can write poetry, debug code, and explain quantum physics. But ask it to open Spotify and it stares at you blankly. **Not anymore.**\n\nComputer-use agents (CUAs) are AI systems that see your screen, understand what's on it, and interact with it — just like a human would, minus the coffee breaks and existential dread.\n\n## Contents\n\n- [Desktop Agents](#desktop-agents)\n- [Browser Agents](#browser-agents)\n- [Mobile Agents](#mobile-agents)\n- [Vision and Grounding Models](#vision-and-grounding-models)\n- [Screen Capture and Recording](#screen-capture-and-recording)\n- [OCR and Screen Reading](#ocr-and-screen-reading)\n- [Desktop Automation Libraries](#desktop-automation-libraries)\n- [RPA Tools](#rpa-tools)\n- [MCP Servers for Desktop](#mcp-servers-for-desktop)\n- [Sandbox and Containerization](#sandbox-and-containerization)\n- [Voice-Controlled Agents](#voice-controlled-agents)\n- [Agent Memory](#agent-memory)\n- [Agent Orchestration](#agent-orchestration)\n- [Agent Testing and Evaluation](#agent-testing-and-evaluation)\n- [Game-Playing Agents](#game-playing-agents)\n- [Training Data Tools](#training-data-tools)\n- [Screen Understanding](#screen-understanding)\n- [Browser Extension Agents](#browser-extension-agents)\n- [Desktop-Native AI Assistants](#desktop-native-ai-assistants)\n- [IDE-Based Agents](#ide-based-agents)\n- [Research Papers](#research-papers)\n- [The Leaderboard](#the-leaderboard)\n- [Related Awesome Lists](#related-awesome-lists)\n\n## Desktop Agents\n\nThe full package. These see your screen, click things, type things, and occasionally crash things.\n\n- [OpenInterpreter](https://github.com/openinterpreter/openinterpreter) - Coding agent for open models like Kimi K3. The OG. `68.0k ⭐`.\n- [OpenHands](https://github.com/OpenHands/OpenHands) - Software dev agents that browse, code, and run commands. `83.7k ⭐`.\n- [Self-Operating Computer](https://github.com/OthersideAI/self-operating-computer) - Framework for multimodal models to operate a computer. `10.3k ⭐`.\n- [Bytebot](https://github.com/bytebot-ai/bytebot) - Self-hosted AI desktop agent in Docker. `11.1k ⭐`.\n- [Agent S](https://github.com/simular-ai/agent-s) - SOTA on OSWorld 72.6%, beats OpenAI CUA. `12.1k ⭐`.\n- [Open Computer Use (E2B)](https://github.com/e2b-dev/open-computer-use) - Cloud sandbox plus open-source LLMs. `2.2k ⭐`.\n- [GenericAgent](https://github.com/lsdefine/GenericAgent) - Minimal 3,300 LOC self-evolving agent. `13.7k ⭐`.\n- [Coworker](https://github.com/accomplish-ai/coworker) - Open-source AI desktop coworker that lives on your desktop (formerly Accomplish). `10.9k ⭐`.\n- [Fara](https://github.com/microsoft/fara) - Microsoft's family of frontier computer-use agent models (Fara 1.5). `6.1k ⭐`.\n- [Magentic-UI](https://github.com/microsoft/magentic-ui) - Human-centered web agent with approval workflows. `10.1k ⭐`.\n- [Cuse](https://github.com/cuse-dev/cuse) - Open framework for building CUA agents. `87 ⭐`.\n- [Clevrr Computer](https://github.com/Clevrr-AI/Clevrr-Computer) - Open-source Anthropic CUA with PyAutoGUI. `318 ⭐`.\n- [Surf](https://github.com/e2b-dev/surf) - OpenAI-powered CUA on E2B's virtual desktop (formerly Cuse Agent). `843 ⭐`.\n- [AskUI Vision Agent](https://github.com/askui/python-sdk) - Automate desktop, mobile, and HMI devices. `551 ⭐`.\n- [Agent Desktop](https://github.com/lahfir/agent-desktop) - Native Rust CLI, AX-first, no screenshots. `1.0k ⭐`.\n- [TongUI Agent](https://github.com/TongUI-agent/TongUI-agent) - VLA model learning from web tutorials. `114 ⭐`.\n- [IrisGUI](https://github.com/black-yt/IrisGUI) - Lightweight with Dynamic Focus Vision. `6 ⭐`.\n- [Oboto](https://github.com/sschepis/oboto) - Everything assistant with persistent cognitive architecture. `27 ⭐`.\n- [Computer-Use-Agent](https://github.com/nabhpatodi10/Computer-Use-Agent) - Windows CUA with OmniParser, GUI Actor, and OpenAI models. `0 ⭐`.\n- [Gofer](https://github.com/jleuth/gofer) - AI agent that lives on your computer, controllable via Telegram. `3 ⭐`.\n- [VNC-Use](https://github.com/mayflower/vnc-use) - LangGraph agent via VNC, Gemini-powered. `8 ⭐`.\n- [Ghost OS](https://github.com/ghostwright/ghost-os) - macOS AX tree plus local VLM, self-learning recipes. `1.6k ⭐`.\n- [TuriX](https://github.com/TurixAI/TuriX-CUA) - Desktop automation, OpenClaw integration. `3.2k ⭐`.\n- [Atlas](https://github.com/dortanes/atlas) - Electron overlay, Gemini native CUA. `24 ⭐`.\n- [Open Cowork](https://github.com/coasty-ai/open-cowork) - Coasty's full-stack computer-use coworker, multi-provider BYO LLM. `130 ⭐`.\n- [Cua](https://github.com/trycua/cua) - Containers for computer-use agents. `21.2k ⭐`.\n- [Cybergod](https://github.com/james4ever0/agi_computer_control) - AGI computer control framework. `157 ⭐`.\n\n## Browser Agents\n\nFor when your AI only needs to wreck things on the web.\n\n- [Browser-Use](https://github.com/browser-use/browser-use) - SOTA browser automation, makes websites LLM-friendly. `108.8k ⭐`.\n- [Skyvern](https://github.com/Skyvern-AI/skyvern) - Vision-driven web automation, no coded selectors. `22.7k ⭐`.\n- [Nanobrowser](https://github.com/nanobrowser/nanobrowser) - Chrome extension, multi-agent Planner and Navigator. `13.5k ⭐`.\n- [Fuji-Web](https://github.com/normal-computing/fuji-web) - Chrome extension with step-by-step explanations. `597 ⭐`.\n- [Notte](https://github.com/nottelabs/notte) - Cloud browser infrastructure and web automation platform for AI agents. `2.0k ⭐`.\n- [LaVague](https://github.com/lavague-ai/LaVague) - Natural language web automation. `6.4k ⭐`.\n- [UI-Act](https://github.com/TobiasNorlund/UI-Act) - Grounded UI interaction. `88 ⭐`.\n- [WebMarker](https://github.com/reidbarber/webmarker) - Web annotation for agents. `61 ⭐`.\n\n## Mobile Agents\n\nAI that taps, swipes, and scrolls on your phone so you can doom-scroll guilt-free.\n\n- [Open-AutoGLM](https://github.com/zai-org/Open-AutoGLM) - Open phone agent model and framework for Android. `26.0k ⭐`.\n- [AppAgent](https://github.com/TencentQQGYLab/AppAgent) - LLM-based multimodal agent for smartphone apps. `6.8k ⭐`.\n- [Mobile-Use](https://github.com/madeagents/mobile-use) - Any app, any task. NeurIPS 2025. `179 ⭐`.\n- [Mobile-Use](https://github.com/minitap-ai/mobile-use) - Automate Android and iOS with natural language. `2.8k ⭐`.\n- [CoCo-Agent](https://github.com/xbmxb/CoCo-Agent) - Smartphone GUI automation, ACL 2024. `35 ⭐`.\n\n## Vision and Grounding Models\n\nThe eyes of computer-use. These models see and understand what is on screen.\n\n- [OmniParser](https://github.com/microsoft/OmniParser) - Screen parsing for pure vision-based GUI agents. `25.2k ⭐`.\n- [CogAgent](https://github.com/zai-org/CogAgent) - Open-sourced VLM-based GUI Agent, CVPR 2024. `1.2k ⭐`.\n- [ShowUI](https://github.com/showlab/ShowUI) - Lightweight VLM for GUI agent and CUA. `1.9k ⭐`.\n- [UI-Venus](https://github.com/inclusionAI/UI-Venus) - Native UI agent, SOTA on ScreenSpot-Pro. `1.0k ⭐`.\n- [UGround](https://github.com/OSU-NLP-Group/UGround) - Universal Visual Grounding, ICLR 2025 Oral. `316 ⭐`.\n- [OS-Atlas](https://github.com/os-copilot/os-atlas) - Foundation Action Model for Generalist GUI Agents. `453 ⭐`.\n- [ScreenSpot-Pro](https://github.com/likaixin2000/screenspot-pro-gui-grounding) - GUI Grounding benchmark for high-res. `387 ⭐`.\n- [Screen-Point-and-Read](https://github.com/UCSB-AI/Screen-Point-and-Read) - Layout-aware GUI screen reading. `31 ⭐`.\n- [ShowUI-Aloha](https://github.com/showlab/ShowUI-Aloha) - Human-taught computer-use agent. `318 ⭐`.\n- [GUI-G1](https://github.com/Yuqi-Zhou/GUI-G1) - R1-Zero-like training for GUI grounding. `28 ⭐`.\n- [GUI-Libra](https://github.com/GUI-Libra/GUI-Libra) - Post-training framework for GUI agents. `68 ⭐`.\n- [SeeClick](https://github.com/njucckevin/SeeClick) - Harnessing GUI grounding for visual agents. `495 ⭐`.\n\n## Screen Capture and Recording\n\nTaking screenshots for AI so it knows what is going on.\n\n- [Screenpipe](https://github.com/screenpipe/screenpipe) - Continuous screen and audio recording, searchable AI memory. `20.9k ⭐`.\n- [Peekaboo](https://github.com/openclaw/Peekaboo) - macOS screen capture plus AI analysis plus MCP server. `5.0k ⭐`.\n- [macOS Visual Agent](https://github.com/peakmojo/macos-visual-agent) - On-device screen understanding via Apple AX APIs. `6 ⭐`.\n- [CaptiOCR](https://github.com/carlosacchi/captiocr) - Real-time OCR for video conferencing. `19 ⭐`.\n\n## OCR and Screen Reading\n\nTurning pixels into text your AI can actually read.\n\n- [NormCap](https://github.com/dynobo/normcap) - OCR-powered screen-capture, capture text not images. `2.7k ⭐`.\n- [OpenOCR](https://github.com/Topdu/OpenOCR) - General OCR toolkit from Fudan University. `1.4k ⭐`.\n- [AgentOCR](https://github.com/AgentMaker/AgentOCR) - Multi-language OCR with PaddleOCR plus ONNX. `125 ⭐`.\n- [LOCR](https://github.com/dmzlingyin/LOCR) - Lightweight auto-extract OCR, cross-platform Go. `11 ⭐`.\n- [Prism AI](https://github.com/chrismannina/prism-ai-screen-reader) - AI screen reader with OCR, privacy-first. `2 ⭐`.\n\n## Desktop Automation Libraries\n\nThe plumbing behind the magic. Low-level tools that do the actual clicking.\n\n- [RobotJS](https://github.com/octalmage/robotjs) - Node.js Desktop Automation. `12.8k ⭐`.\n- [PyAutoGUI](https://github.com/asweigart/pyautogui) - Cross-platform GUI automation for Python. `12.7k ⭐`.\n- [nut.js](https://github.com/nut-tree/nut.js) - Native UI automation for Node.js. `2.8k ⭐`.\n- [guibot](https://github.com/intra2net/guibot) - GUI automation with computer vision. `187 ⭐`.\n- [libnut-core](https://github.com/nut-tree/libnut-core) - Node-API addon for desktop automation. `55 ⭐`.\n- [qontinui](https://github.com/qontinui/qontinui) - Model-based GUI automation with state management. `3 ⭐`.\n\n## RPA Tools\n\nRobotic Process Automation, but make it AI.\n\n- [TagUI-Python](https://github.com/tebelorg/RPA-Python) - Leading open-source Python RPA tool. `5.5k ⭐`.\n- [rpaframework](https://github.com/robocorp/rpaframework) - Open-source RPA libraries for Robot Framework. `1.5k ⭐`.\n- [astron-rpa](https://github.com/iflytek/astron-rpa) - Enterprise-grade RPA with LLM integration. `5.8k ⭐`.\n- [AIVANE](https://github.com/taiyouqu/aivane) - AI plus RPA cross-platform control. `1 ⭐`.\n\n## MCP Servers for Desktop\n\nModel Context Protocol servers that give your AI desktop superpowers.\n\n- [DesktopCommanderMCP](https://github.com/wonderwhy-er/DesktopCommanderMCP) - Terminal control, file system, diff editing. `9.3k ⭐`.\n- [Computer Control MCP](https://github.com/AB498/computer-control-mcp) - Mouse, keyboard, OCR via PyAutoGUI. `158 ⭐`.\n- [QuickDesk](https://github.com/barry-ran/QuickDesk) - AI-native remote desktop with MCP. `345 ⭐`.\n- [System MCP](https://github.com/aurafriday/system_mcp) - Full desktop automation, multi-platform. `3 ⭐`.\n- [MCP Computer Use](https://github.com/mediar-ai/MCP-server-client-computer-use-ai-sdk) - Screenpipe-based computer control. `195 ⭐`.\n- [Desktop Automation MCP](https://github.com/tanob/mcp-desktop-automation) - RobotJS-based desktop automation. `34 ⭐`.\n- [Automation MCP](https://github.com/ashwwwin/automation-mcp) - macOS automation, mouse, keyboard, screenshots. `415 ⭐`.\n- [Windows Desktop Automation MCP](https://github.com/mario-andreschak/mcp-windows-desktop-automation) - Windows desktop automation via AutoIt. `113 ⭐`.\n- [VNC MCP](https://github.com/volkan-m/vnc-mcp-server) - VNC integration for remote agent control. `1 ⭐`.\n\n## Sandbox and Containerization\n\nKeep your AI agent from burning down the house. Run it in a sandbox.\n\n- [E2B Desktop](https://github.com/e2b-dev/desktop) - Open-source secure virtual desktop. `1.4k ⭐`.\n- [Agent Sandbox](https://github.com/agent-infra/sandbox) - Browser plus Shell plus File plus MCP in one container. `5.7k ⭐`.\n- [Code Sandbox MCP](https://github.com/Automata-Labs-team/code-sandbox-mcp) - Secure code sandbox in Docker. `326 ⭐`.\n- [Cyberdesk](https://github.com/cyberdesk-hq/cyberdesk) - Kubernetes-native virtual desktop orchestration. `313 ⭐`.\n- [EdgeBox](https://github.com/BIGPPWONG/EdgeBox) - Local desktop sandbox with MCP support. `213 ⭐`.\n- [Sandbox0](https://github.com/sandbox0-ai/sandbox0) - General-purpose sandbox with hot pool. `76 ⭐`.\n\n## Voice-Controlled Agents\n\nTalk to your computer and it actually does things. Revolutionary.\n\n- [Fazm](https://github.com/mediar-ai/fazm) - Fully local macOS AI agent via voice. `312 ⭐`.\n- [Voqal](https://github.com/voqal/voqal) - Voice-native AI agent, speech-to-code. `193 ⭐`.\n- [Voice Mirror](https://github.com/contextmirror/voice-mirror) - Voice-controlled AI overlay, Tauri plus Rust. `10 ⭐`.\n- [Colossus](https://github.com/richardanaya/colossus) - Voice-controlled multi-agent dev system. `15 ⭐`.\n- [Tankwork](https://github.com/agenttankos/tankwork) - Computer vision plus voice commands, macOS. `98 ⭐`.\n- [Desktop Agent Voice](https://github.com/ruslankoroy/desktop-agent) - CV plus Whisper voice input. `34 ⭐`.\n\n## Agent Memory\n\nBecause your AI should remember that you hate dark mode (weirdo).\n\n- [Mem0](https://github.com/mem0ai/mem0) - Universal memory layer for AI agents. `63.0k ⭐`.\n- [OpenMemory](https://github.com/CaviraOSS/OpenMemory) - Cognitive memory engine with 4 memory types. `4.4k ⭐`.\n- [A-Mem](https://github.com/agiresearch/A-mem) - Agentic Memory with knowledge networks. `1.1k ⭐`.\n- [ReMe](https://github.com/agentscope-ai/ReMe) - Memory management kit, SOTA on benchmarks. `3.3k ⭐`.\n- [Memori](https://github.com/MemoriLabs/Memori) - Agent-native memory infrastructure, LLM-agnostic. `15.7k ⭐`.\n- [MemMachine](https://github.com/MemMachine/MemMachine) - Long-term episodic plus profile memory. `3.4k ⭐`.\n- [SimpleMem](https://github.com/aiming-lab/SimpleMem) - Semantic lossless compression, MCP server. `3.7k ⭐`.\n- [MemLayer](https://github.com/divagr18/memlayer) - Plug-and-play, under 100ms search. `277 ⭐`.\n- [Agent Memory](https://github.com/rohitg00/agentmemory) - Persistent memory for AI coding agents, auto-capture via hooks. `26.9k ⭐`.\n\n## Agent Orchestration\n\nWhen one agent is not enough and you need a whole crew.\n\n- [Ruflo](https://github.com/ruvnet/ruflo) - Enterprise AI orchestration, 60 plus agents. `67.6k ⭐`.\n- [Claude Code by Agents](https://github.com/baryhuang/claude-code-by-agents) - Multi-agent Claude Code with at-mentions. `890 ⭐`.\n- [AnyTool](https://github.com/HKUDS/AnyTool) - Universal tool-use layer with recording. `685 ⭐`.\n\n## Agent Testing and Evaluation\n\nProve your agent actually works (or laugh at how badly it fails).\n\n- [ScreenSuite](https://github.com/huggingface/screensuite) - 13 benchmarks for GUI agents. `145 ⭐`.\n- [OSUniverse](https://github.com/agentsea/osuniverse) - 160 complex multimodal desktop tasks. `24 ⭐`.\n- [ScaleCUA](https://github.com/OpenGVLab/ScaleCUA) - Cross-platform dataset plus training framework. `1.1k ⭐`.\n- [Agent VCR](https://github.com/ixchio/agent-vcr) - Time-travel debugging for AI agents. `33 ⭐`.\n\n## Game-Playing Agents\n\nAI that plays games while you pretend to work.\n\n- [AI-Player](https://github.com/shasankp000/AI-Player) - Minecraft mod with LLM-powered AI player. `138 ⭐`.\n- [OSRS PVP RL](https://github.com/Naton1/osrs-pvp-reinforcement-learning) - Deep RL for Old School RuneScape PvP. `170 ⭐`.\n- [Bedrock Minecraft Agent](https://github.com/build-on-aws/amazon-bedrock-minecraft-agent) - Minecraft bot with Amazon Bedrock. `59 ⭐`.\n- [OSRS-CV](https://github.com/KaustubhLall/OSRS-CV) - Computer vision for OSRS automation. `3 ⭐`.\n- [Claude Minecraft Use](https://github.com/ObservedObserver/claude-minecraft-use) - Claude playing Minecraft. `25 ⭐`.\n- [Minecraft-RL](https://github.com/otsolappalainen/Minecraft-RL) - Deep RL training AI to play Minecraft. `2 ⭐`.\n\n## Training Data Tools\n\nCollect data so your AI agent can learn from humans who actually know how to use a computer.\n\n- [OpenCUA](https://github.com/xlang-ai/OpenCUA) - 22.6K trajectories, annotation tools. `816 ⭐`.\n- [GUI Dataset Creator](https://github.com/harpreetsahota204/gui_dataset_creator) - Annotate GUI interactions in COCO format. `13 ⭐`.\n- [Interaction Logger](https://github.com/hemangjoshi37a/AIComputerInteractionLogger) - Capture screenshots, mouse, keyboard. `13 ⭐`.\n- [Data Collection](https://github.com/bobcoi03/computeruse-data-collection) - Privacy-first interaction data collection. `1 ⭐`.\n- [Captr](https://github.com/anaishowland/Captr_Windows) - Screen recording plus interaction capture. `8 ⭐`.\n\n## Screen Understanding\n\nParse the chaos of a desktop into structured data your AI can reason about.\n\n- [Screen2AX](https://github.com/MacPaw/Screen2AX) - Vision to macOS accessibility tree, 77 percent F1. `32 ⭐`.\n\n## Browser Extension Agents\n\nYour browser, but smarter. And it does things for you.\n\n- [ChromeClaw](https://github.com/algopian/chromeclaw) - OpenClaw-inspired Chrome extension. `192 ⭐`.\n- [Hanzi Browse](https://github.com/hanzili/hanzi-browse) - Give your AI agent your real browser via MCP. `175 ⭐`.\n- [Dev Browser](https://github.com/SawyerHood/dev-browser) - Claude Code browser automation plugin. `6.5k ⭐`.\n- [WebOperator](https://github.com/KazKozDev/WebOperator) - A small browser agent that lives inside Chrome (formerly Browser Agent Extension). `1 ⭐`.\n- [Izan.io](https://github.com/ekingunoncu/izan.io) - 17 plus providers, no-code action recorder. `29 ⭐`.\n- [Browser Control](https://github.com/dhamaniasad/browser-control) - Gemini multimodal browser automation. `16 ⭐`.\n\n## Desktop-Native AI Assistants\n\nYour always-on AI buddy that lives in your system tray.\n\n- [OpenClaw](https://github.com/openclaw/openclaw) - Multi-channel AI assistant with skills. `385.9k ⭐`.\n\n## IDE-Based Agents\n\nCoding agents that learned to use a mouse.\n\n- [Mastra Code UI](https://github.com/mastra-ai/mastra-code-ui) - Desktop coding agent with multi-model. `53 ⭐`.\n- [Elves](https://github.com/mvmcode/elves) - Tauri app orchestrating Claude Code and Codex. `11 ⭐`.\n- [Pilos Agents](https://github.com/pilos-ai/agents) - Visual desktop for Claude Code, multi-agent. `39 ⭐`.\n\n## Research Papers\n\nThe science behind the magic.\n\n- [Agent S](https://arxiv.org/abs/2410.08164) - Open agentic framework that uses computers like a human.\n- [UI-TARS](https://arxiv.org/abs/2501.12326) - Pioneering automated GUI interaction with native agents.\n- [OSWorld](https://arxiv.org/abs/2404.07972) - Benchmarking multimodal agents for open-ended tasks in real computer environments. `NeurIPS 2024`.\n- [OS Agents Survey](https://openreview.net/forum?id=ed2f5ee6b84c3b118cb953b6e750486dbd700419) - Survey on MLLM-based agents for general computing devices use.\n- [Computer Use Agents Survey](https://arxiv.org/abs/2501.16150) - A review of instruction-based computer control, GUI automation, and operator assistants.\n- [Cradle](https://arxiv.org/abs/2403.03186) - Empowering foundation agents towards general computer control.\n- [Aguvis](https://arxiv.org/abs/2412.04454) - Unified pure vision agents for autonomous GUI interaction.\n- [OS-Atlas](https://arxiv.org/abs/2410.23218) - Foundation action model for generalist GUI agents. `ICLR 2025`.\n- [SeeClick](https://arxiv.org/abs/2401.10935) - Harnessing GUI grounding for advanced visual GUI agents. `ACL 2024`.\n- [OmniParser](https://arxiv.org/abs/2408.00203) - Screen parsing tool for pure vision based GUI agent.\n\n## The Leaderboard\n\nWho is actually winning at computer use?\n\n- Agent S3 - OSWorld: 72.6% (Open Source)\n- Anthropic CUA - OSWorld: ~68% (Closed Source)\n- OpenAI Operator - OSWorld: ~65% (Closed Source)\n- Browser-Use - WebVoyager: 89.1% (Open Source)\n- Skyvern - WebVoyager: 85.8% (Open Source)\n- Agent-E - WebVoyager: 73.1% (Open Source)\n\n## Related Awesome Lists\n\nThese folks are doing great work too. Go star them.\n\n- [ranpox/awesome-computer-use](https://github.com/ranpox/awesome-computer-use) - The OG CUA list with papers and projects. `573 ⭐`.\n- [Awesome-GUI-Agent](https://github.com/showlab/Awesome-GUI-Agent) - GUI agent papers and projects. `1.2k ⭐`.\n- [GUI-Agents-Paper-List](https://github.com/OSU-NLP-Group/GUI-Agents-Paper-List) - Comprehensive paper list. `880 ⭐`.\n- [Awesome-GUI-Agents](https://github.com/ZJU-REAL/Awesome-GUI-Agents) - GUI agents with project links. `451 ⭐`.\n- [ACU](https://github.com/trycua/acu) - Awesome Agents for Computer Use. `1.7k ⭐`.\n- [steel-dev/awesome-web-agents](https://github.com/steel-dev/awesome-web-agents) - Web agents. `1.5k ⭐`.\n- [awesome-local-llm](https://github.com/rafska/awesome-local-llm) - Local LLM ecosystem. `2.5k ⭐`.\n","projects_url":"https://awesome.ecosyste.ms/api/v1/lists/12britz%2Fawesome-computer-use/projects"}