An open API service indexing awesome lists of open source software.

awesome-web-agents

🔥 A list of tools, frameworks, and resources for building AI web agents
https://github.com/steel-dev/awesome-web-agents

Last synced: about 1 month ago
JSON representation

AI Web Automation Tools

Computer-use Agents

  • Asteroid.ai - Hosted browser agents for SMEs to automate complex workflows.
  • PulsarRPA - AI-powered browser automation for data extraction.
  • VimGPT - Experimental project using GPT-4 Vision to browse the web via the Vimium extension.
  • Cekura.io - An AI browser agent that helps companies maintain up-to-date documentation.
  • Dex by Dexterity - An AI coworker embedding into and controlling your browser.
  • Runcopycat - A no-code browser automation platform that turns screen recordings into reusable automated workflows.
  • Starizon.ai - Browser assistant for web task automation.
  • BrowserGPT - Browser extension for page summaries and Q&A.
  • Browse.ai - Chrome extension webscraping that can leverage AI for structured data extraction.
  • Deta.surf - An integrated platform that combines a browser, file manager, and AI assistant with browser-level context.
  • Comet by Perplexity - An AI-powered browser by Perplexity. Not much more details out yet.
  • Dia Browser - AI-first web browser envisioned by The Browser Company (Arc).
  • Autobrowser - A free, experimental Chrome extension that leverages Claude Computer Use to automate tasks in your browser.
  • Komos - AI browser workflow automation platform that turns recorded web tasks into reusable runs with API triggers, schedules, credentials, logs, and human review.
  • agent-qa - Self-improving QA harness for natural-language web test runs with execution memory and self-healing actions.
  • Onpiste - Chrome extension that uses AI to control and read webpages, including auto summaries, web automation, scraping, and MCP support.

Dev Tools

  • Omniparser - Tool for parsing GUIs for vision based agents.
  • LaVague - Framework for natural language web automation.
  • LangChain Playwright Toolkit - Toolkit integration with AI agents.
  • Tarsier - Vision utilities library for web interaction agents.
  • Stagehand - AI web browsing framework.
  • AutoGPT - Experimental agent for task completion and web browsing.
  • Bytebot - Containerized computer use agent framework with a virtual desktop environment.
  • Steel.dev - Open-source headless browser API built specifically for AI agents and apps.
  • Lumen - Vision-first browser agent with self-healing deterministic replay. Screenshot → model → action loop over CDP, multi-provider (Anthropic, Google, OpenAI), action caching for zero-token reruns.
  • BrowserTrace - Local-first trace viewer for debugging Playwright, Browser Use, Stagehand, and other web-agent runs with redacted shareable exports.
  • invisible-playwright - Playwright wrapper for a stealth-patched Firefox 150 build. Drop-in replacement returning native Playwright Browser objects; spoofing happens in C++ source with no JS-level overrides.
  • Webwright - Browser agent framework from Microsoft Research where the agent writes and runs Playwright scripts in a terminal workspace; supports OpenAI, Anthropic, and OpenRouter backends.
  • BrowserAct - Browser automation CLI and skills for AI agents to operate real browsers, manage sessions, support human handoff, and capture screenshots and evidence.
  • Agent Browser Shield - Browser extension that sits between an AI agent and the page, stripping prompt injection, masking PII/credentials, and removing dark patterns before content reaches the model.
  • Atlas - Open-source deep research harness for building cited web research agents with ledger-based coverage audits, pluggable search providers, and Steel-backed browser fetches.
  • HUD - Open-source SDK for building browser and computer-use RL environments to evaluate and train web agents, with task-based verifiable rewards runnable as evals or RL training across any model.
  • Webcmd - Self-learning browser infrastructure for AI agents that records how a site is navigated, then compiles that context into deterministic CLI adapters and reusable sitemap memory for later runs.
  • Hermes Connector - Unofficial open-source Chrome extension and local companion that let Hermes Agent control only explicitly attached tabs in a user's Chrome session.
  • TinyFish - Remote web agents that execute tasks on any website and return structured JSON via a single API call.
  • Webagent-cloud - Open-source API for browser agents to automate repetitive workflows. Works with multiple browsers/LLM providers, and minimizes costs with a self-healing action cache.
  • Webfuse - Configurable web proxy and browser-as-a-service for deploying and operating AI agents in a sandbox layer on top of any third-party website, using client-side extensions and without source-code access.

AI Web Scrapers/Crawlers

Dev Tools

  • FireCrawl - APIs for turning websites into LLM-friendly markdown.
  • WebAgent (OpenAgents) - The web-browsing agent module of the OpenAgents platform (HKU). Enables autonomous navigation of websites via natural language, as part of a larger multi-modal agent framework.
  • Expand.ai - Turns any website into a type-safe API you can rely on.
  • LLM Scraper - Uses LLMs for intelligent scraping and content understanding.
  • SpiderCreator - Create complex Playwright spiders with natural language prompts.
  • Plasmate - Open-source headless browser engine for AI agents. Compiles HTML to Semantic Object Model (SOM) with 17.5x token compression. 13 MCP tools. First browser tool on the MCP Registry. Rust, Apache-2.0.
  • Context.dev - Web APIs and MCP tools for search, scraping, crawling, schema-based extraction, document parsing, monitoring, and batch jobs.
  • Crawl4AI - Open-source LLM Friendly Web Crawler & Scraper.
  • DataLens - AI web data agent that plans browser scraping tasks, extracts structured web data, and exposes MCP tools for dataset workflows.

Autonomous Web Agents

Computer-use Agents

  • Anthropic Computer Use - Computer use agent that can control your browser.
  • Self-Operating Computer Framework - A framework to enable multimodal models to operate a computer.
  • Highlight - Desktop activity layer that helps models understand your workflow and complete tasks faster.
  • OpenInterpreter - An open-source CLI based agent that can write & execute code as well as control your browser.
  • UI-TARS - A GUI agent model designed to interact seamlessly with GUIs using human-like perception, reasoning, and action capabilities.
  • OpenAI Operator - OpenAI's AI agents that can browse the web for you.
  • Skyvern-AI - Framework to automate browser-based workflows.
  • WebVoyager (Agent) - Vision-enabled web agent.
  • AgentGPT - Deploy autonomous AI agents in your browser.
  • Agent-E - Agent & framework with HTML DOM distillation.
  • Manus - A general AI agent that can execute long running tasks across tools like browsers, terminals, and text editors.
  • WebSurfer (Autogen) - MultimodalWebSurfer is a multimodal agent that can search the web and visit web pages.
  • Magentic-One - A generalist multi-agent system for solving complex tasks including surfing the web via Autogen's MultimodalWebSurfer.
  • Yutori - A multi-agent system that executes browser-based tasks in parallel given a natural language prompt.
  • rtrvr.ai - AI web agent Chrome extension that autonomously does tasks, scrapes to Sheets, and calls APIs with prompts in your own browser.
  • Tongyi WebAgent - WebAgent for information seeking built by Tongyi Lab, Alibaba Group.
  • Google Project Mariner - A research prototype exploring the future of human-agent interaction, starting with your browser.
  • Openwork - An MIT-licensed, open alternative to Anthropic's Cowork built with Opencode and dev-browser. Supports multiple LLM providers for launching computer-use agents to automate browser workflows.
  • Dassi - An AI coworking agent in your browser that automates tasks, navigates pages, and works with files and 2000+ apps from a side panel.
  • Caesar - Autonomous research agent that traverses the web to build a knowledge graph, then synthesizes an answer through adversarial redrafting.
  • doBrowser - An AI-powered Chrome extension that understands natural language and takes actions in your browser on your behalf.
  • Nanobrowser - An open-source & local-first AI web agent Chrome extension with flexible LLM options and multi-agent system.

Benchmarks & Research

Dev Tools

  • Web Games by Convergence - A collection of challenges designed for testing general-purpose web-browsing AI agents.
  • Bananalyzer - An open-source evaluation framework for web-based AI agents.
  • World of Bits: An Open-Domain Platform for Web-Based Agents - OpenAI's research paper that introduces World of Bits: a platform where agents complete tasks on the internet by performing low-level keyboard and mouse actions.
  • WebCanvas - An online evaluation framework for dynamic web environments. Tests agents on live websites.
  • WebGPT - OpenAI's browser-assisted question-answering research project.
  • WorkArena - A suite of 33 browser-based tasks for enterprise "knowledge worker" scenarios.
  • BrowserGym by ServiceNow - A gym environment for web task automation.
  • WebVoyager (Benchmark) - Vision-enabled benchmark for real-world website interaction with large multimodal models.
  • WebTaskBench - 51-URL benchmark comparing HTML vs Markdown vs SOM representations for AI agents. Measures token efficiency, latency, and accuracy across GPT-4o and Claude Sonnet 4.
  • ClawBench - 283 everyday tasks (V1 153 + V2 130) on 163 live production websites across 15 categories. Two-stage scoring (final HTTP-request interception + LLM judge) blocks only the write request so real sites stay clean. Public leaderboard with 5-layer execution traces (recording, action log, request log, agent messages, interception).
  • Web Agent Leaderboard - Leaderboard compiling AI agent products and their performance on widely used WebVoyager benchmarks.
  • MiniWoB++ - A classic suite of 104 mini web browser tasks in a synthetic environment. It is an extension of the OpenAI MiniWoB benchmark.
  • WebArena - A realistic, self-hostable web environment for autonomous agents. Includes official leaderboard tracking agent performance.
  • WebShop - A simulated e-commerce shopping environment with 1.18M real Amazon products.
  • TimeWarp - A benchmark on historical versions of web UI.

Featured (new releases)

  • Opera Agentic Feature - Opera announces a new agentic feature for its browser, showcasing innovative web agent integration.

Interested in implementing Steel?

Dev Tools

Join the Community

Dev Tools

Tutorials & Guides

Dev Tools

Web Search & Query Tools

Dev Tools

  • AgentQL - A query language and toolkit that makes the web AI-ready.
  • SerpAPI - Search API that provides Google Search results for your agents.
  • Serper.dev - Performant and cost effective search API that provides Google Search results for your agents.
  • Jina.ai - Neural search platform for web data.
  • Exa.ai - The fastest and most accurate web search API for AI agents.
  • Zoom Search - Open-source web search and evidence tool for AI agents with query rewriting, source-domain zoom-in, structured sourced outputs, and MCP and LangGraph integrations.
  • Not Human Search - Search engine that indexes 1,750+ agent-first tools ranked by agentic readiness. Available as an MCP server with tools for searching, scoring, and monitoring agent infrastructure.
  • Superhighway - Web search API for AI agents with five tools (search, news, images, scrape, research); agents pay per call in USDC via the x402 protocol, or use a free API key.
  • SocialCrawl - Unified social and commerce data API covering 50+ platforms as one GET and one JSON schema.