awesome-interface-agents
List of AI tools that can interact with user interfaces
https://github.com/lectrician1/awesome-interface-agents
Last synced: about 19 hours ago
JSON representation
-
Complete solutions
-
Operating system
- Manus AI
- ScreenAgent
- UI-ACT
- OpenInterpreter
- OpenAdapt.AI - First Process Automation with Large ([Language (LLMs) / Action (LAMs) / Multimodal (LMMs)] / Visual Language (VLMs)) Models
- Mobile-Agent
- Qwen 2.5-VL Cookbook
- AIOS
- Claude 3.5 Computer Use Cookbook
-
Web browser
- Skyvern
- AgentLLM
- LaVague
- Google Project Mariner
- HyperWrite AI Agent
- OpenAI Operator - Using Agent (CUA) model to interact with the user interface and ask for clarification from the user in your browser.
-
-
Models
-
Segmenters
-
VLMs
- CogAgent - source visual language model that can identify regions and points of UIs to interact with.
- Llama 3.2 - level understanding including charts and graphs, captioning of images, and visual grounding tasks such as directionally pinpointing objects in images based on natural language descriptions.
- Molmo - 4V performance with pointing ability.
- Florence 2 - based representation for a variety of computer vision and vision-language tasks including producing bounding boxes.
- Claude 3.5 Computer Use
- Qwen 2.5-VL
-
-
Papers
Programming Languages
Categories
Sub Categories