Browse The Shelves

Agent evals

• Agent Eval Framework - grade evaluation and continuous improvement of AI agents across multiple quality dimensions. | Framework · Library | Repo |
• Agentic Security - source vulnerability scanner and red-teaming kit for LLM agents and workflows focused on security evaluation. | CLI · Framework · Library | Repo |
• Async Labs LLM Eval (PHP) - Labs/llm-eval) |
• AWS Agent Evaluation - powered framework from AWS Labs to test virtual agents via multi-turn conversations and CI-friendly workflows. | CLI · Framework · Library | Docs / Repo |
• DeepTeam - source LLM red teaming framework for simulating attacks and evaluating safety of LLM systems using DeepEval metrics. | CLI · Framework · Library | Repo |
• Guardrails AI
• LangChain OpenEvals - source collection of ready-made evaluators for LLM applications in the LangChain ecosystem. | Framework · Library | Repo |
• LangFair - case level LLM bias and fairness assessment using bring-your-own-prompts workflows. | Framework · Library | Docs / Repo |
• LangSmith Evaluations - based and online eval workflows. | API · Framework · Web | Website / Docs |
• LLM Evaluation Framework (Inference-Time) - time methods to evaluate LLM reliability across reasoning strategies. | Framework · Library | Repo |
• nuclia-eval - eval/) / Repo |
• Open RAG Eval - RAG style metrics. | CLI · Framework · Library | Website / Docs / Repo |
• OpenAI HumanEval Harness - eval) |
• PyHDL-Eval - specific LLM evaluation framework for assessing hardware design code generation using Python-embedded DSLs and RTL tasks. | CLI · Framework · Library | Repo |
• RAG Evaluation Harnesses - evaluation-harness for retrieval-augmented generation tasks. | CLI · Framework · Library | Repo |
• Rhesis AI - source collaborative testing platform for LLM and agentic applications with UI, SDK, and CI integration. | API · CLI · Framework · Web | Website / Docs / Repo |
• Unitxt - backed library and catalog for textual data preparation and evaluation of generative language models across many tasks. | CLI · Framework · Library | Website / Docs / Repo |
• Vectara Open Evaluation UI - evaluation) / Docs |
• whatbroke - arumugam-git/whatbroke) |
• DeepEval - backed metrics. | CLI · Framework · Library | Website / Repo |
• YYLO Benchmark - repository attempt workspaces with blinded per-step judging. | CLI · Local | Docs / Repo |

Agent observability

• Amazon CloudWatch GenAI Observability - observability.html) / Repo |
• Arize AX - agents/agent-observability/) / Docs |
• Datadog LLM Observability - observability/) / Docs / Repo |
• Dunetrace - source anomaly detection layer that monitors agent traces for structural failures and alerts in real time. | API · Self-hosted | Website / Docs / Repo |
• Flexible GraphRAG Observability Stack - hosted | Docs / Repo |
• LangSmith - platform) / Docs |
• Monocle
• Noveum AI Agent Monitoring - time tracing of LLM calls, RAG steps, and tool interactions. | API · Web · Hosted | Website |
• Okahu - based applications. | API · Web · Hosted | Website / Docs / Repo |
• OpenAI Traces and Evals - in OpenAI platform capabilities for tracing agent workflows and running evaluations on traces and datasets. | API · Web · Hosted | Website / Docs / Repo |
• OpenClaw Monitor - time monitoring dashboard for OpenClaw agents — track task execution, token usage, and agent activity with an intuitive UI. | Web · Self-hosted | Repo |
• OpenInference - source OpenTelemetry-based specification and instrumentations for tracing AI and LLM applications. | Library · Local | Docs / Repo |
• OpenObserve LLM Observability - source observability platform that unifies infrastructure and LLM observability with logs, metrics, and traces. | API · Web · Hybrid | Website / Docs / Repo |
• Zenity AI Observability Platform - focused AI observability platform for discovering and monitoring agents across SaaS, endpoints, and environments. | API · Web · Hosted | Website |
• LiteLLM - hosted | Docs / Repo |

Agent skill packs

• Agent Skills Specification
• AgentSkills MCP - compatible agent with progressive disclosure architecture. | MCP · Local | Repo |
• Anthropic Knowledge Work Plugins - source plugins for Claude Cowork and Claude Code with skills, commands, and MCP connectors for knowledge workers. | MCP · Local | Repo |
• Astronomer Agents
• Awesome Agent Skills - curated directory of agent skills, tutorials, and guides for Claude, Copilot, Codex, and other AI assistants. | Skill Pack | Repo |
• Awesome Cursorrules - cursorrules) |
• Awesome GitHub Copilot - copilot) |
• Awesome Rules - rules) |
• Azure DevOps Skills - devops-skills) |
• CC DevOps Skills - ozer/cc-devops-skills) |
• Cursor Rules - rules) |
• Cursorrules Collection - ok/cursorrules-collection) |
• Custom Modes for Roo Code - first principles. | Skill Pack · Template · Local | Repo |
• dbt Agent Skills - labs/dbt-agent-skills) |
• dotskills
• helderberto/skills
• Invariant Testing - ai/testing) |
• Microsoft Skills
• Pulumi Agent Skills - as-code workflows teaching AI assistants Pulumi migrations, components, and secrets management. | Skill Pack · Template · Local | Repo |
• roomode
• rules
• Superpowers Lab - demand MCP server usage. | Skill Pack · Local | Repo |
• Swarms - agent orchestration skills for Claude Code and Codex with dependency-aware planning and wave-based parallel execution. | Framework · Skill Pack · Local | Repo |
• wshobson/agents

Browser agents

• Agent Browser - based headless browser automation CLI designed for AI agents, wrapping Chromium control behind a simple command-line tool. | CLI | Repo |
• Agent-E - source agent system focused on browser automation, offering a natural-language web automation API and local agent runtime. | Framework · Library | Repo |
• Agent-WebVoyager - style agent that navigates and extracts data from sites via visual interactions. | Framework · Library | Repo |
• Browser MCP
• BrowserPilot - driven browser. | Library | Repo |
• Charlotte MCP Server - source MCP server that renders web pages into structured, token-efficient representations using headless Chromium for agents. | MCP · Self-hosted | Docs / Repo |
• Firecrawl Web Agent - use to execute web tasks over an HTTP API for downstream applications. | API · Framework | Repo |
• HyperAgent
• Komos - automation-tools) / Docs |
• LaVague - based browser interactions for AI web agents. | Framework · Library | Repo |
• OpenAI Computer Use - computer-use) |
• Skyvern - step workflows across websites via an API and SDK. | API · Library · Hybrid | Docs / Repo |
• Steel Browser - source browser API that lets AI apps and agents perform programmatic browsing tasks without managing browser processes directly. | API · Library | Website / Repo |
• Tarsier - backed views of pages by tagging interactable elements. | Library | Repo |
• Webagent (webagent-cloud) - cloud/webagent) |
• WebVoyager - powered web agent that completes end-to-end tasks on real websites using visual and textual signals in a Selenium environment. | Framework · Library | Repo |
• Agent QA - language web and mobile tests through a CLI, dashboard, MCP server, and agent skills. | CLI · MCP · Skill Pack · Web · Local | Website / Docs / Repo |

Coding agents

• Amazon Q Developer - ug/what-is.html) |
• Amp - centric coding agent with deep codebase context, editor links, and automation-oriented SDK features. | API · CLI · Hybrid | Website / Docs |
• Augment Code - aware coding agent for editors and terminal that edits files, uses tools, and understands large codebases. | CLI · IDE · Hybrid | Website / Docs |
• Claude Code
• Cursor - assisted development workflows. | CLI · Desktop · IDE · Hybrid | Website / Docs |
• Devin - started/devin-intro) |
• Ellipsis
• Factory Droid
• Ivy Tendril - source AI coding orchestrator that manages Claude Code, Codex, Antigravity, Copilot and more through a plan-based lifecycle with verification gates and self-improving memory. | Desktop · Local | Website / Docs |
• Jules
• Junie - assistant/junie-agent.html) / Repo |
• OpenAI Codex CLI
• Plandex - source terminal agent for long-running tasks across large projects and real repositories. | CLI · Local | Website / Docs / Repo |
• Qwen Code - source terminal coding agent optimized for Qwen models and large repository tasks. | CLI · Local | Website / Repo |
• Refact.ai - vscode) |
• SWE-agent - source agent for resolving real repository issues and automating software engineering tasks. | CLI · Local | Website / Docs / Repo |
• Sweep - focused coding assistant with agent mode, repo edits, AI code review, and MCP integration. | IDE · Hybrid | Website / Docs / Repo |
• Windsurf Editor - aware agent workflows for multi-file edits and developer automation. | Desktop · Hybrid | Website / Docs |
• cursor-bridge - bridge) |
• Kolega Code - source terminal coding agent where the model writes its own multi-agent workflows (Gigacode) for repo-wide tasks. | CLI · Local | Docs / Repo |

Documentation agents

• AI Document Creator - generates and updates multi-level repository documentation via pull requests. | GitHub app · Web · Hosted | Website / Docs |
• ai-doc-gen - agent CLI that analyzes codebases and generates repository documentation and READMEs using LLMs. | CLI · Local | Repo |
• AIGNE DocSmith - driven documentation generator that builds structured multi-language docs directly from code repositories. | CLI | Website / Repo |
• AutoCodeDocs
• AWS Documentation MCP Server - documentation-mcp-server) / Repo |
• Code2Docs - ai) / Repo |
• Docs.dev - assisted docs-as-code platform that generates and audits Markdown documentation from your codebase. | GitHub app · Web · Hosted | Website / Docs |
• Docudoodle - focused AI documentation generator that creates Markdown docs from existing codebases. | CLI · Local | Repo |
• DocWhizz
• GitCrafts
• Gitdocs AI - powered README generator and developer documentation platform integrated with Git hosting. | Web · Hosted | Website / Docs |
• Google Developer Knowledge API & MCP Server
• RepoGPT - source GitHub assistant that can generate documentation and answer questions about repositories. | Web | Repo |
• write-the - powered Python tool that auto-generates documentation and tests from code, and assists with refactoring. | CLI · Library · Local | Docs / Repo |
• DeepDocs - native AI agent that keeps READMEs, API docs, SDK guides, and tutorials in sync with code changes. | GitHub app · Web · Hosted | Website / Docs |
• AIGNE DocSmith - driven documentation generator that builds structured multi-language docs directly from code repositories. | CLI | Website / Repo |
• AI Document Creator - generates and updates multi-level repository documentation via pull requests. | GitHub app · Web · Hosted | Website / Docs |
• AutoCodeDocs
• DocuWriter.ai - documentation) |
• Gitdocs AI - powered README generator and developer documentation platform integrated with Git hosting. | Web · Hosted | Website / Docs |
• Mintlify - native documentation platform that generates and maintains docs and API references from repositories. | GitHub app · Web · Hosted | Website / Docs |
• RepoMix - friendly file using heuristics and optional LLM support. | CLI · Local | Docs / Repo |

IDE assistants

• GitHub Copilot - of-github-copilot/about-github-copilot) |
• Qodo

Local LLM developer tools

• AutoGPTQ - only quantization of Transformer language models for efficient local inference. | Library · Local | Docs / Repo |
• Docker Model Runner - integrated local model runtime that runs GGUF models via an OpenAI-compatible API. | API · CLI · Local | Website / Docs |
• ExLlamaV2 - quantized LLMs on modern consumer GPUs with high performance. | CLI · Library · Local | Docs / Repo |
• Haystack - source framework for building RAG pipelines, agents, and semantic search over private data. | API · Framework · Local | Docs / Repo |
• Intel Neural Compressor - source compression toolkit supporting quantization and pruning for deep learning and LLM workloads. | Library · Local | Docs / Repo |
• LiteLLM Gateway - in proxy that unifies many LLM providers behind an OpenAI-compatible API, with logging and rate limiting. | API · CLI · Self-hosted | Docs / Repo |
• LLM Compressor - compressor/en/latest/) / Repo |
• llm-llamafile - llamafile#readme) / Repo |
• Mistral Inference - hosted | Docs / Repo |
• MLX Engine for LM Studio - based inference engine used by LM Studio to run quantized text and vision models locally on macOS. | Library · Local | Docs / Repo |
• mlx-lm - based Python package for running and fine-tuning LLMs on Apple silicon, with an HTTP server similar to OpenAI chat API. | API · CLI · Library · Local | Docs / Repo |
• PydanticAI - based framework for building typed LLM apps and agents with validation, tools, and structured outputs. | Framework · Library · Local | Docs / Repo |
• TabbyAPI - based backend server for ExLlamaV2 that exposes local text generation over HTTP for coding and chat workloads. | API · Self-hosted | Docs / Repo |
• text-generation-webui - hosted Gradio/Web UI and API for running diverse local text generation models on CPUs and GPUs. | CLI · Web · Self-hosted | Docs / Repo |
• WebLLM - browser LLM inference engine using WebGPU that exposes an OpenAI-compatible API surface for local models. | Library · Web · Local | Website / Docs / Repo |
• mistral.rs - based engine for fast, flexible LLM inference with OpenAI-compatible APIs, web UI, and extensive quantization options. | API · CLI · Library · Web · Self-hosted | Docs / Repo |
• MLC LLM - hosted | Website / Docs / Repo |

MCP servers

• Anki MCP Server - repetition workflows. | MCP · Local | Website / Repo |
• ArXiv MCP Server - mcp-server) |
• Exa MCP Server - labs/exa-mcp-server) |
• Google MCP Servers
• Next.js DevTools MCP - devtools-mcp) |
• Playwright MCP Server - started-mcp) / Repo |
• Supabase MCP Server - community/supabase-mcp) |
• UIZZE - gate reviews for coding agents. | MCP · Skill Pack · Web · Hosted | Website / Docs |

MCP tooling

• cheqd MCP Toolkit - toolkit#readme) / Repo |
• MCP C# SDK - sdk) |
• MCP Clients Registry - maintained registry mapping MCP client identifiers to official client names and metadata. | Library | Repo |
• MCP Framework (QuantGeekDev) - first framework and CLI for building MCP servers with automatic discovery and multi-transport support. | CLI · Framework | Docs / Repo |
• MCP Registry Server (KBB99) - hosted | Website / Repo |
• Model Context Protocol TypeScript SDK - sdk) |
• Smithery MCP Installer Server - hosted | Repo |
• PulseMCP Registry - focused registry and API that aggregates, enriches, and exposes metadata about public MCP servers. | API · Web · Hosted | Website / Docs |
• MCP Registry - driven registry service and REST API for publishing and discovering Model Context Protocol servers. | API · Hybrid | Website / Docs / Repo |

Repo automation tools

• auto-changelog (cookpete) - changelog) |
• automerge-action - pull-requests-automerge-action) / Repo |
• Bito AI Code Review Agent - code-review-agent/) / Docs |
• changelog-generator (jaywcjlove) - generator) |
• dbt-semguard - semguard#readme) / Repo |
• GitHub Dependabot - native service that configures and runs automated security and version dependency update pull requests. | GitHub app · Hosted | Docs |
• GitHub Stacked PRs (gh stack) - stack/) |
• GitHub Stale Action
• Pull Request Labeler (actions/labeler)
• Qodo-Cover - powered tool that automatically generates unit tests and improves code coverage, runnable as a CLI or in CI. | CLI · GitHub app · Hybrid | Docs / Repo |
• Release Please - please-action) / Repo |
• release-it - it/release-it) |
• Renovate
• semantic-release - release.gitbook.io/semantic-release/recipes/ci-configurations/github-actions) / Repo |
• Shelfi AI GitHub Bot - powered GitHub bot that analyzes pull requests and suggests documentation updates tied to code changes. | GitHub app · Web · Hosted | Docs |
• stack-attack - attack) |
• stack-pr - pr) |
• stale-repos Action - repos) / Repo |
• UpdateBot - x/updatebot) |
• Renovate

Self-hosted AI dev stacks

• Clawix AI Platform - source, self-hosted multi-agent AI orchestration platform running agents in isolated Docker containers. | CLI · Web · Self-hosted | Docs / Repo |
• DevBox AI Dev Stack - trust dev environment repository including an AI dev stack installer for Docker-based tools on a VPS. | CLI · Self-hosted | Docs / Repo |
• Local LLM Docker (mythrantic) - hosted | Docs / Repo |
• OpenRAG - source RAG stack combining Langflow, Docling, and OpenSearch with single-command Docker Compose deployment. | API · Web · Self-hosted | Docs / Repo |
• Paddler - source LLMOps platform for hosting and scaling AI models in your own infrastructure. | API · CLI · Web · Self-hosted | Docs / Repo |
• Pleng - hosted AI platform engineer that installs a Docker-based PaaS with AI agents and automation on your own cloud. | CLI · Web · Self-hosted | Website / Docs / Repo |
• Scalable Agentic RAG Platform (FareedKhan-dev) - grade agentic RAG platform with Kubernetes, Ray, LangGraph, and multi-plane architecture. | CLI · Self-hosted | Docs / Repo |
• Self-hosted AI Package - source Docker Compose starter kit bundling Ollama, n8n, Supabase, Open WebUI, Flowise, Neo4j, Langfuse, and related tools. | API · Web · Self-hosted | Website / Repo |
• Synaplan AI Platform - source, self-hosted AI platform with full stack deployment via Docker Compose and local inference via Ollama. | API · Web · Self-hosted | Website / Docs |
• Winy RAG Sample Stack - hosted | Docs / Repo |
• Gateway From Scratch - compatible API gateway teaching scaffold with multi-provider fallback, written in Node.js for learning gateway internals. | API · Template · Self-hosted | Docs / Repo |
• Pleng - hosted AI platform engineer that installs a Docker-based PaaS with AI agents and automation on your own cloud. | CLI · Web · Self-hosted | Website / Docs / Repo |

Terminal agents

• agenttrace
• codex-profiles - profiles/) / Docs / Repo |
• Tree Ring Memory - first Rust CLI/TUI for agent memory lifecycle with SQLite/FTS recall, audit, forgetting, and source-linked evidence. | CLI · Skill Pack · Local | Website / Docs / Repo |
• agent-watch - agent runs and distinguish transport failures from credential failures before launch. | CLI · Local | Docs / Repo |
• YYLO - line orchestrator for coding agents where each task runs in a dedicated branch/worktree with typed task, validation, merge, and release-readiness boundaries. | CLI · Local | Docs / Repo |

Comparison Matrix

Automate repo work

• Athenaeum - hosted AI dev stacks | Yes | Yes | Yes | No | No | Yes | Docs / Repo |
• Gemini CLI - code-assist/docs/gemini-cli) / Repo |
• Goose - docs.ai/) / Docs / Repo |
• IBM MCP CLI - cli) |
• KoboldCpp
• llama-cpp-python - cpp-python.readthedocs.io) / Repo |
• llama.cpp - org/llama.cpp/tree/master/examples) / Repo |
• MCP Bundles (MCPB) - with-agent-skills) / Repo |