https://github.com/steel-dev/awesome-web-agents
🔥 A list of tools, frameworks, and resources for building AI web agents
https://github.com/steel-dev/awesome-web-agents
List: awesome-web-agents
ai ai-agents browser-automation llms llms-benchmarking
Last synced: 5 months ago
JSON representation
🔥 A list of tools, frameworks, and resources for building AI web agents
- Host: GitHub
- URL: https://github.com/steel-dev/awesome-web-agents
- Owner: steel-dev
- License: other
- Created: 2025-03-06T05:23:47.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-07-25T19:07:33.000Z (11 months ago)
- Last Synced: 2025-08-07T01:01:59.042Z (11 months ago)
- Topics: ai, ai-agents, browser-automation, llms, llms-benchmarking
- Homepage:
- Size: 259 KB
- Stars: 670
- Watchers: 9
- Forks: 56
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
- Contributing: contributing.md
- License: license
- Code of conduct: code_of_conduct.md
Awesome Lists containing this project
- ultimate-awesome - awesome-web-agents - 🔥 A list of tools, frameworks, and resources for building AI web agents. (Other Lists / TeX Lists)
- awesome-code-agents - awesome-web-agents - dev/awesome-web-agents?label=stars)](https://github.com/steel-dev/awesome-web-agents/stargazers) <a href="https://img.shields.io/github/last-commit/steel-dev/awesome-web-agents?color=green"><img src="https://img.shields.io/github/last-commit/steel-dev/awesome-web-agents?color=green" alt="Last Commit"></a> (🙏 Acknowledgements / 🧪 Frontier Labs and Teams)
- awesome-computer-use - steel-dev/awesome-web-agents - Web agents. (Related Awesome Lists)
- awesome-openclaw-skills - steel-dev/awesome-web-agents
README
# Awesome Web Agents
A curated list of tools, frameworks, and resources for building AI agents that can browse and interact with the web.
About Steel
Steel is an [open-source](https://github.com/steel-dev/steel-browser) browser API built specifically for AI agents. We make it easy to build AI applications that can effectively interact with the web.
✨ Get started for free [here](https://app.steel.dev).
Contents
- [Awesome Web Agents](#awesome-web-agents)
- [Autonomous Web Agents](#autonomous-web-agents)
- [Computer-use Agents](#computer-use-agents)
- [AI Web Automation Tools](#ai-web-automation-tools)
- [Dev Tools](#dev-tools)
- [AI Web Scrapers/Crawlers](#ai-web-scraperscrawlers)
- [Web Search \& Query Tools](#web-search--query-tools)
- [Benchmarks \& Research](#benchmarks--research)
- [Tutorials \& Guides](#tutorials--guides)
- [Interested in implementing Steel?](#interested-in-implementing-steel)
- [Join the Community](#join-the-community)
- [Contributing](#contributing)
- [Contributors](#contributors)
## Autonomous Web Agents
AI agents that autonomously navigate and interact with the web through a user-friendly interface. (a.k.a Browser Agents)
- [Surf.new](https://surf.new) - An open-source playground for chatting with different web agents. 
- [OpenAI Operator](https://openai.com/index/introducing-operator/) - OpenAI's AI agents that can browser the web for you.
- [Browser-Use](https://www.browser-use.com) - SOTA agent and framework that makes the web LLM-friendly. 
- [Skyvern-AI](https://www.skyvern.com/) - Framework to automate browser-based workflows. 
- [Proxy by Convergence](https://convergence.ai) - Proxy is your AI-powered digital assistant that explores the web and executes tasks through simple conversation.
- [Google Project Mariner](https://deepmind.google/technologies/project-mariner/) - A research prototype exploring the future of human-agent interaction, starting with your browser.
- [Runner H](https://www.hcompany.ai/) - Runner H is a state-of-the-art AI agent that will allow anyone to automate complex, cumbersome, multi-step tasks without repetitive and manual input.
- [WebVoyager (Agent)](https://github.com/MinorJerry/WebVoyager) - Vision-enabled web agent. 
- [AgentGPT](https://github.com/reworkd/AgentGPT) - Deploy autonomous AI agents in your browser. 
- [Agent-E](https://github.com/EmergenceAI/Agent-E) - Agent & framework with HTML DOM distillation. 
- [Kura](https://www.trykura.com/) - Web Agents for the Enterprise.
- [Manus](https://manus.im/) - A general AI agent that can execute long running tasks across tools like browsers, terminals, and text editors.
- [doBrowser](https://www.dobrowser.io) - An AI-powered Chrome extension that understands natural language and takes actions in your browser on your behalf.
- [WebSurfer (Autogen)](https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.agents.web_surfer.html#autogen_ext.agents.web_surfer.MultimodalWebSurfer) - MultimodalWebSurfer is a multimodal agent that can search the web and visit web pages. 
- [Magentic-One](https://www.microsoft.com/en-us/research/articles/magentic-one-a-generalist-multi-agent-system-for-solving-complex-tasks/) - A generalist multi-agent system for solving complex tasks including surfing the web via Autogen's MultimodalWebSurfer.
- [Harpa.ai](https://harpa.ai/) - An AI-powered Chrome extension & browser agent that understands natural language and takes actions on your behalf.
- [Yutori](https://yutori.com/) - A multi-agent system that executes browser-based tasks in parallel given a natural language prompt.
- [Automina](https://automina.app/) - AI browser automation tool with natural language control.
- [rtrvr.ai](https://www.rtrvr.ai/) - AI Web Agent Chrome Extension that autonomously does tasks, scrapes to Sheets, and calls API's – all with just prompts and your own browser!
- [Nanobrowser](https://nanobrowser.ai) - An open-source & local-first AI web agent Chrome extension with flexible LLM options and multi-agent system. 
- [Browserable](https://browserable.ai) - An open-source & self-hostable browser automation library for AI agents. 
- [Tongyi WebAgent](https://github.com/Alibaba-NLP/WebAgent) - WebAgent for Information Seeking bulit by Tongyi Lab, Alibaba Group. 
### Computer-use Agents
- [Anthropic Computer Use](https://www.anthropic.com/news/3-5-models-and-computer-use) - Computer use agent that can control your browser.
- [Self-Operating Computer Framework](https://github.com/OthersideAI/self-operating-computer) - A framework to enable multimodal models to operate a computer. 
- [Highlight](https://highlightai.com/) - Highlight AI lets models understand your desktop activity. Get stuff done faster.
- [OpenInterpreter](https://github.com/openinterpreter/open-interpreter) - An open-source CLI based agent that can write & execute code as well as control your browser. 
- [UI-TARS](https://github.com/bytedance/UI-TARS?tab=readme-ov-file) - A GUI agent model designed to interact seamlessly with GUIs using human-like perception, reasoning, and action capabilities. 
## AI Web Automation Tools
Tools, frameworks and libraries that translate natural language instructions into web interactions.
- [Asteroid.ai](https://asteroid.ai/) - Hosted Browser Agents for SMEs to automate complex workflows. 
- [PulsarRPA](https://github.com/platonai/pulsarRPA) - AI-powered browser automation for data extraction. 
- [VimGPT](https://github.com/ishan0102/vimGPT) - Experimental project using GPT-4 Vision to browse the web via the Vimium extension. 
- [Cekura.io](https://www.cekura.io/) - An AI browser agent that helps companies maintain up-to-date documentation.
- [Dex by Dexterity](https://getdexterity.com/) - An AI coworker embedding into and controlling your browser.
- [Autobrowser](https://autobrowser.ai/) - A free, experimental Chrome extension that leverages Claude Computer Use to automate tasks in your browser.
- [Bytebot](https://bytebot.ai) - Bytebot provides AI-powered scraping automations that evolve with your target sites.
- [Runcopycat](https://www.runcopycat.com/) - A no-code browser automation platform that turns screen recordings into reusable automated workflows.
- [Bardeen.ai](https://bardeen.ai) - A Chrome extension that enables AI-powered browser automations, allowing users to automate tasks and workflows directly within the browser.
- [Starizon.ai](https://starizon.ai/) - Browser assistant for web task automation.
- [BrowserGPT](https://browsegpt.ai/) - Browser extension for page summaries and Q&A.
- [Browse.ai](https://www.browse.ai/) - Chrome extension webscraping that can leverage AI for structured data extraction.
- [Strawberry Browser](https://www.strawberrybrowser.com/) - A personal assistant that sits in your browser, automates repetitive web actions, learns your workflows.
- [Deta.surf](https://deta.surf/) - An integrated platform that combines a browser, file manager, and AI assistant with browser-level context.
- [Comet by Perplexity](https://www.perplexity.ai/comet) - An AI-powered browser by Perplexity. Not much more details out yet.
- [Dia Browser](https://www.diabrowser.com/) - Dia Browser is envisioned as an entirely new web browser built with AI at the center by The Browser Company (Arc).
- [Reworkd](https://reworkd.ai) - No-code web data extraction solution using agentic AI.
- [Ottogrid](https://ottogrid.ai/) - Spreadsheet based web agents to automate manual research.
### Dev Tools
- [Steel.dev](https://steel.dev) - Open-source headless browser API built specifically for AI agents and apps. 
- [Omniparser](https://microsoft.github.io/OmniParser/) - Tool for parsing GUIs for vision based agents. 
- [LaVague](https://www.lavague.ai/) - Framework for natural language web automation. 
- [Langchain Playwright toolkit](https://python.langchain.com/docs/integrations/tools/playwright/#use-within-an-agent) - Toolkit integration with AI agents.
- [Browserbase](https://browserbase.com) - A headless browser API for AI workflows.
- [Stagehand](https://www.stagehand.dev/) - AI web browsing framework. 
- [Tarsier](https://github.com/reworkd/tarsier) - Vision utilities library for web interaction agents. 
- [AutoGPT](https://github.com/Significant-Gravitas/AutoGPT) - Experimental agent for task completion and web browsing. 
- [Bytebot](https://github.com/bytebot-ai/bytebot) - Containerized computer use agent framework with a virtual desktop environment. 
## AI Web Scrapers/Crawlers
Web crawlers & scrapers that leverage AI to navigate websites and extract content.
- [FireCrawl](https://www.firecrawl.dev/) - APIs for turning websites into LLM-friendly markdown. 
- [Crawl4AI](https://crawl4ai.com) - Open-source LLM Friendly Web Crawler & Scraper. 
- [ScrapeGraphAI](https://scrapegraphai.com/) - Python scraper based on AI. 
- [WebAgent (OpenAgents)](https://github.com/xlang-ai/OpenAgents) - The web-browsing agent module of the OpenAgents platform (HKU). Enables autonomous navigation of websites via natural language, as part of a larger multi-modal agent framework. 
- [Expand.ai](https://www.expand.ai/) - Turns any website into a type-safe API you can rely on.
- [LLM Scraper](https://github.com/mishushakov/llm-scraper) - Uses LLMs for intelligent scraping and content understanding. 
- [SpiderCreator](https://github.com/carlosplanchon/spidercreator) - Create complex Playwright spiders with natural language prompts. 
## Web Search & Query Tools
Utilities that help agents search the web or query web data via natural language.
- [AgentQL](https://www.agentql.com/) - A query language and toolkit that makes the web AI-ready. 
- [SerpAPI](https://serpapi.com/) - Search API that provides Google Search results for your agents.
- [Serper.dev](https://serper.dev/) - Performant and cost effective search API that provides Google Search results for your agents.
- [Jina.ai](https://jina.ai/) - Neural search platform for web data.
- [Exa.ai](https://exa.ai) - Semantic Search Engine for AI.
## Benchmarks & Research
Datasets, benchmarks, and notable research efforts for evaluating and advancing web-capable AI agents.
- [Web Agent Leaderboard](https://leaderboard.steel.dev) - Web agent leaderboard compiling different AI agent products and how they perform on the widely used WebVoyager benchmarks. 
- [Web Games by Convergence](https://webgames.convergence.ai/) - a collection of challenges designed for testing general-purpose web-browsing AI agents. 
- [Bananalyzer](https://github.com/reworkd/bananalyzer) - An open-source evaluation framework for web-based AI agents. 
- [Mind2Web](https://osu-nlp-group.github.io/Mind2Web) - A large-scale dataset for generalist web agents. 
- [World of Bits: An Open-Domain Platform for Web-Based Agents](https://proceedings.mlr.press/v70/shi17a/shi17a.pdf) - OpenAI's research paper that introduces World or Bits: a platform where agents complete tasks on the internet by performing low-level keyboard and mouse actions.
- [MiniWoB++](https://miniwob.farama.org) - A classic suite of 104 mini web browser tasks in a synthetic environment. It's is an extension of the OpenAI MiniWoB benchmark. 
- [WebArena](https://webarena.dev) - A realistic, self-hostable web environment for autonomous agents. Includes official leaderboard tracking agent performance. 
- [WebCanvas](https://github.com/iMeanAI/WebCanvas) - An online evaluation framework for dynamic web environments. Tests agents on live websites. 
- [WebGPT](https://openai.com/research/webgpt) - OpenAI's browser-assisted question-answering research project.
- [WebShop](https://webshop-pnlp.github.io) - A simulated e-commerce shopping environment with 1.18M real Amazon products. 
- [WebVoyager (Benchmark)](https://github.com/MinorJerry/WebVoyager) - Vision-enabled web agent using GPT-4V for real-world website interaction. 
- [WorkArena](https://github.com/ServiceNow/WorkArena) - A suite of 33 browser-based tasks for enterprise "knowledge worker" scenarios. 
- [BrowserGym by ServiceNow](https://github.com/ServiceNow/BrowserGym) - A gym environment for web task automation. 
## Tutorials & Guides
Resources for learning how to build, deploy, or utilize AI web agents.
- [LangGraph WebVoyager Tutorial](https://langchain-ai.github.io/langgraph/tutorials/web-navigation/web_voyager/) - Tutorial demonstrating how to build a web navigation agent using LangGraph Agents, Vision Models, and Web Voyager.
- [Build an AI Browser Agent](https://dzone.com/articles/build-ai-browser-agent-llms-playwright-browser-use) - Step-by-step guide to create an AI that browses the web using Playwright and the Browser-Use library.
- [Install & Run Browser-Use Locally](https://aleksandarhaber.com/install-and-run-browser-use-ai-agents-locally-using-ollama/) - Instructions on installing the open-source Browser-Use agent with a local LLM.
- [Build a Browser Agent with DeepSeek](https://nodeshift.com/blog/build-a-browser-use-agent-with-deepseek-a-step-by-step-guide) - Walks through deploying a Browser-Use web UI agent powered by the DeepSeek model on a cloud VM.
## Interested in implementing Steel?
Feel free to reach out at [team@steel.dev](mailto:team@steel.dev?subject=Hello%20from%20github!) or on [Discord](https://discord.gg/steel-dev).
Steel is an [open-source](https://github.com/steel-dev/steel-browser) browser API built specifically for AI agents. Get started for free [here](https://app.steel.dev).
## Join the Community
- Follow [@steeldotdev](https://x.com/steeldotdev) on X.
- Join the [Discord community](https://discord.gg/steel-dev).
- Feel free to reach out to us at [team@steel.dev](mailto:team@steel.dev?subject=Hello%20from%20github!)
## Contributing
[Contributions of any kind welcome, just follow the guidelines](contributing.md)!
### Contributors
[Thanks goes to these contributors](https://github.com/steel-dev/awesome-web-agents/graphs/contributors)!