Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

awesome-computer-use

139 open-source projects that let AI click, type, scroll, and generally make a mess of your desktop — so you don't have to.
https://github.com/12britz/awesome-computer-use

Last synced: 2 days ago
JSON representation

  • Agent Memory

    • Mem0 - Universal memory layer for AI agents. `63.0k ⭐`.
    • OpenMemory - Cognitive memory engine with 4 memory types. `4.4k ⭐`.
    • A-Mem - Agentic Memory with knowledge networks. `1.1k ⭐`.
    • ReMe - Memory management kit, SOTA on benchmarks. `3.3k ⭐`.
    • Memori - Memory engine with multi-provider support. `150 ⭐`.
    • MemMachine - Long-term episodic plus profile memory. `3.4k ⭐`.
    • SimpleMem - Semantic lossless compression, MCP server. `3.7k ⭐`.
    • MemLayer - Plug-and-play, under 100ms search. `277 ⭐`.
    • Agent Memory - Persistent memory for AI coding agents, auto-capture via hooks. `26.9k ⭐`.
    • Memori - Agent-native memory infrastructure, LLM-agnostic. `15.7k ⭐`.

  • Agent Orchestration

    • Ruflo - Enterprise AI orchestration, 60 plus agents. `67.6k ⭐`.
    • Claude Code by Agents - Multi-agent Claude Code with at-mentions. `890 ⭐`.
    • AnyTool - Universal tool-use layer with recording. `685 ⭐`.

  • Agent Testing and Evaluation

    • ScreenSuite - 13 benchmarks for GUI agents. `145 ⭐`.
    • OSUniverse - 160 complex multimodal desktop tasks. `24 ⭐`.
    • ScaleCUA - Cross-platform dataset plus training framework. `1.1k ⭐`.
    • Agent VCR - Time-travel debugging for AI agents. `33 ⭐`.

  • Browser Agents

    • Browser-Use - SOTA browser automation, makes websites LLM-friendly. `108.8k ⭐`.
    • Skyvern - Vision-driven web automation, no coded selectors. `22.7k ⭐`.
    • Nanobrowser - Chrome extension, multi-agent Planner and Navigator. `13.5k ⭐`.
    • Fuji-Web - Chrome extension with step-by-step explanations. `597 ⭐`.
    • LaVague - Natural language web automation. `6.4k ⭐`.
    • UI-Act - Grounded UI interaction. `88 ⭐`.
    • WebMarker - Web annotation for agents. `61 ⭐`.
    • Notte - Reliable browser AI framework.
    • Notte - Cloud browser infrastructure and web automation platform for AI agents. `2.0k ⭐`.

  • Browser Extension Agents

    • ChromeClaw - OpenClaw-inspired Chrome extension. `192 ⭐`.
    • Dev Browser - Claude Code browser automation plugin. `6.5k ⭐`.
    • Izan.io - 17 plus providers, no-code action recorder. `29 ⭐`.
    • Browser Control - Gemini multimodal browser automation. `16 ⭐`.
    • LLM in Chrome - Give your AI agent your real browser via MCP. `50 ⭐`.
    • Browser Agent Extension - Manifest V3 with tool calling. `30 ⭐`.
    • Hanzi Browse - Give your AI agent your real browser via MCP. `175 ⭐`.
    • WebOperator - A small browser agent that lives inside Chrome (formerly Browser Agent Extension). `1 ⭐`.

  • Desktop Agents

    • OpenInterpreter - Run code locally via natural language. The OG. `55k+ ⭐`.
    • OpenHands - Software dev agents that browse, code, and run commands. `40k+ ⭐`.
    • Self-Operating Computer - Framework for multimodal models to operate a computer. `10.3k ⭐`.
    • Bytebot - Self-hosted AI desktop agent in Docker. `11.1k ⭐`.
    • Agent S - SOTA on OSWorld 72.6%, beats OpenAI CUA. `12.1k ⭐`.
    • Open Computer Use (E2B) - Cloud sandbox plus open-source LLMs. `2.2k ⭐`.
    • GenericAgent - Minimal 3,300 LOC self-evolving agent. `13.7k ⭐`.
    • Accomplish - Open-source AI desktop agent, local-first. `500 ⭐`.
    • Fara - Microsoft's family of frontier computer-use agent models (Fara 1.5). `6.1k ⭐`.
    • Magentic-UI - Human-centered web agent with approval workflows. `10.1k ⭐`.
    • Cuse - Open framework for building CUA agents. `87 ⭐`.
    • Clevrr Computer - Open-source Anthropic CUA with PyAutoGUI. `318 ⭐`.
    • AskUI Vision Agent - Automate desktop, mobile, and HMI devices. `200 ⭐`.
    • Agent Desktop - Native Rust CLI, AX-first, no screenshots. `1.0k ⭐`.
    • TongUI Agent - VLA model learning from web tutorials. `114 ⭐`.
    • IrisGUI - Lightweight with Dynamic Focus Vision. `6 ⭐`.
    • Oboto - Everything assistant with persistent cognitive architecture. `27 ⭐`.
    • Computer-Use-Agent - Windows CUA with OmniParser, GUI Actor, and OpenAI models. `0 ⭐`.
    • Gofer - AI agent that lives on your computer, controllable via Telegram. `3 ⭐`.
    • VNC-Use - LangGraph agent via VNC, Gemini-powered. `8 ⭐`.
    • Ghost OS - macOS AX tree plus local VLM, self-learning recipes. `1.6k ⭐`.
    • TuriX - Desktop automation, OpenClaw integration. `3.2k ⭐`.
    • Atlas - Electron overlay, Gemini native CUA. `24 ⭐`.
    • Open Computer Use - Full-stack platform, multi-provider.
    • Cua - Containers for computer-use agents. `21.2k ⭐`.
    • Cybergod - AGI computer control framework. `157 ⭐`.
    • Cuse Agent - CUA with E2B virtual desktop sandbox. `300 ⭐`.
    • OpenInterpreter - Coding agent for open models like Kimi K3. The OG. `68.0k ⭐`.
    • OpenHands - Software dev agents that browse, code, and run commands. `83.7k ⭐`.
    • Coworker - Open-source AI desktop coworker that lives on your desktop (formerly Accomplish). `10.9k ⭐`.
    • Surf - OpenAI-powered CUA on E2B's virtual desktop (formerly Cuse Agent). `843 ⭐`.
    • AskUI Vision Agent - Automate desktop, mobile, and HMI devices. `551 ⭐`.
    • Open Cowork - Coasty's full-stack computer-use coworker, multi-provider BYO LLM. `130 ⭐`.

  • Desktop Automation Libraries

    • RobotJS - Node.js Desktop Automation. `12.8k ⭐`.
    • PyAutoGUI - Cross-platform GUI automation for Python. `12.7k ⭐`.
    • nut.js - Native UI automation for Node.js. `2.8k ⭐`.
    • guibot - GUI automation with computer vision. `187 ⭐`.
    • libnut-core - Node-API addon for desktop automation. `55 ⭐`.
    • qontinui - Model-based GUI automation with state management. `30 ⭐`.
    • qontinui - Model-based GUI automation with state management. `3 ⭐`.

  • Desktop-Native AI Assistants

    • OpenClaw - Multi-channel AI assistant with skills. `385.9k ⭐`.

  • Game-Playing Agents

  • IDE-Based Agents

    • Mastra Code UI - Desktop coding agent with multi-model. `53 ⭐`.
    • Elves - Tauri app orchestrating Claude Code and Codex. `11 ⭐`.
    • Pilos Agents - Visual desktop for Claude Code, multi-agent. `39 ⭐`.

  • MCP Servers for Desktop

  • Mobile Agents

    • AppAgent - LLM-based multimodal agent for smartphone apps. `6.8k ⭐`.
    • Mobile-Use - Any app, any task. NeurIPS 2025. `179 ⭐`.
    • Mobile-Use - Automate Android and iOS with natural language. `2.8k ⭐`.
    • CoCo-Agent - Smartphone GUI automation, ACL 2024. `35 ⭐`.
    • Open-AutoGLM - Phone agent with vision-language model for Android. `23.6k ⭐`.
    • Open-AutoGLM - Open phone agent model and framework for Android. `26.0k ⭐`.

  • OCR and Screen Reading

    • NormCap - OCR-powered screen-capture, capture text not images. `2.7k ⭐`.
    • OpenOCR - General OCR toolkit from Fudan University. `1.4k ⭐`.
    • AgentOCR - Multi-language OCR with PaddleOCR plus ONNX. `125 ⭐`.
    • LOCR - Lightweight auto-extract OCR, cross-platform Go. `11 ⭐`.
    • Prism AI - AI screen reader with OCR, privacy-first. `2 ⭐`.

  • Research Papers

    • Agent S - Open agentic framework that uses computers like a human.
    • UI-TARS - Pioneering automated GUI interaction with native agents.
    • OSWorld - Benchmarking multimodal agents for open-ended tasks in real computer environments. `NeurIPS 2024`.
    • OS Agents Survey - Survey on MLLM-based agents for general computing devices use.
    • Computer Use Agents Survey - A review of instruction-based computer control, GUI automation, and operator assistants.
    • Cradle - Empowering foundation agents towards general computer control.
    • Aguvis - Unified pure vision agents for autonomous GUI interaction.
    • OS-Atlas - Foundation action model for generalist GUI agents. `ICLR 2025`.
    • SeeClick - Harnessing GUI grounding for advanced visual GUI agents. `ACL 2024`.
    • OmniParser - Screen parsing tool for pure vision based GUI agent.
    • OS Agents Survey - Survey on MLLM-based agents for general computing devices use.

  • RPA Tools

    • TagUI-Python - Leading open-source Python RPA tool. `5.5k ⭐`.
    • rpaframework - Open-source RPA libraries for Robot Framework. `1.5k ⭐`.
    • astron-rpa - Enterprise-grade RPA with LLM integration. `5.8k ⭐`.
    • AIVANE - AI plus RPA cross-platform control. `1 ⭐`.

  • Sandbox and Containerization

    • E2B Desktop - Open-source secure virtual desktop. `1.4k ⭐`.
    • Agent Sandbox - Browser plus Shell plus File plus MCP in one container. `5.7k ⭐`.
    • Code Sandbox MCP - Secure code sandbox in Docker. `326 ⭐`.
    • Cyberdesk - Kubernetes-native virtual desktop orchestration. `313 ⭐`.
    • EdgeBox - Local desktop sandbox with MCP support. `213 ⭐`.
    • Sandbox0 - General-purpose sandbox with hot pool. `76 ⭐`.

  • Screen Capture and Recording

    • Screenpipe - Continuous screen and audio recording, searchable AI memory. `20.9k ⭐`.
    • Peekaboo - macOS screen capture plus AI analysis plus MCP server. `2.9k ⭐`.
    • macOS Visual Agent - On-device screen understanding via Apple AX APIs. `6 ⭐`.
    • CaptiOCR - Real-time OCR for video conferencing. `19 ⭐`.
    • Peekaboo - macOS screen capture plus AI analysis plus MCP server. `5.0k ⭐`.

  • Screen Understanding

    • Screen2AX - Vision to macOS accessibility tree, 77 percent F1. `32 ⭐`.
    • AX Tree Parsers - Cross-platform AX tree extraction scripts. `50 ⭐`.
    • AX Tree Parsers - Cross-platform AX tree extraction scripts. `50 ⭐`.

  • Training Data Tools

    • OpenCUA - 22.6K trajectories, annotation tools. `816 ⭐`.
    • GUI Dataset Creator - Annotate GUI interactions in COCO format. `13 ⭐`.
    • Interaction Logger - Capture screenshots, mouse, keyboard. `13 ⭐`.
    • Data Collection - Privacy-first interaction data collection. `1 ⭐`.
    • Captr - Screen recording plus interaction capture. `8 ⭐`.

  • Vision and Grounding Models

    • OmniParser - Screen parsing for pure vision-based GUI agents. `25.2k ⭐`.
    • ShowUI - Lightweight VLM for GUI agent and CUA. `1.9k ⭐`.
    • UI-Venus - Native UI agent, SOTA on ScreenSpot-Pro. `1.0k ⭐`.
    • UGround - Universal Visual Grounding, ICLR 2025 Oral. `316 ⭐`.
    • OS-Atlas - Foundation Action Model for Generalist GUI Agents. `453 ⭐`.
    • ScreenSpot-Pro - GUI Grounding benchmark for high-res. `387 ⭐`.
    • Screen-Point-and-Read - Layout-aware GUI screen reading. `31 ⭐`.
    • ShowUI-Aloha - Human-taught computer-use agent. `318 ⭐`.
    • GUI-G1 - R1-Zero-like training for GUI grounding. `28 ⭐`.
    • GUI-Libra - Post-training framework for GUI agents. `68 ⭐`.
    • Screen-Point-and-Read - Layout-aware GUI screen reading. `100 ⭐`.
    • SeeClick - Harnessing GUI grounding for visual agents.
    • CogAgent - Open-sourced VLM-based GUI Agent, CVPR 2024. `1.2k ⭐`.
    • SeeClick - Harnessing GUI grounding for visual agents. `495 ⭐`.

  • Voice-Controlled Agents

    • Fazm - Fully local macOS AI agent via voice. `200 ⭐`.
    • Voqal - Voice-native AI agent, speech-to-code. `193 ⭐`.
    • Voice Mirror - Voice-controlled AI overlay, Tauri plus Rust. `10 ⭐`.
    • Colossus - Voice-controlled multi-agent dev system. `15 ⭐`.
    • Tankwork - Computer vision plus voice commands, macOS. `98 ⭐`.
    • Desktop Agent Voice - CV plus Whisper voice input. `34 ⭐`.
    • Voice Mirror - Voice-controlled AI overlay, Tauri plus Rust. `100 ⭐`.
    • Fazm - Fully local macOS AI agent via voice. `312 ⭐`.
Programming Languages
Python 61 TypeScript 20 JavaScript 6 Rust 4 HTML 4 Swift 3 Go 3 Java 2 Jupyter Notebook 2 C 2
Categories
Desktop Agents 33 Vision and Grounding Models 14 Research Papers 11 Agent Memory 10 MCP Servers for Desktop 10 Browser Agents 9 Voice-Controlled Agents 8 Browser Extension Agents 8 Desktop Automation Libraries 7 Related Awesome Lists 7 Sandbox and Containerization 6 Game-Playing Agents 6 Mobile Agents 6 OCR and Screen Reading 5 Training Data Tools 5 Screen Capture and Recording 5 Agent Testing and Evaluation 4 RPA Tools 4 IDE-Based Agents 3 Agent Orchestration 3 Screen Understanding 3 Desktop-Native AI Assistants 1
Sub Categories
Keywords
ai 23 agent 20 llm 20 computer-use 18 python 15 automation 15 ai-agents 14 mcp 12 openai 9 computer-use-agent 8 browser-automation 8 agents 8 browser-use 6 ai-agent 6 ocr 6 chatgpt 6 macos 5 artificial-intelligence 5 claude 5 memory 5 claude-code 4 vision-language-model 4 playwright 4 gpt 4 browser 4 model-context-protocol 4 desktop-automation 4 gui-agents 4 chrome-extension 4 cua 4 anthropic 4 rag 4 gemini 4 gui-agent 4 web-automation 3 nlp 3 operator 3 codex 3 computer 3 gui 3 cli 3 developer-tools 3 javascript 3 reinforcement-learning 3 ai-assistant 3 awesome 3 openclaw 3 browser-extension 3 computer-vision 3 llm-agent 3