awesome-prompts
Curated prompts, frameworks and papers for LLMs and agents, with an engineering bias. Prompt engineering, context engineering, agent harnesses, red teaming.
https://github.com/ai-boost/awesome-prompts
Last synced: about 20 hours ago
JSON representation
Agent Ecosystem
A2A — Agent-to-Agent Protocol
Agentic Payments
- x402-foundation/x402 - reference implementation + SDKs (TypeScript/Python/Go); Apache-2.0
- google-agentic-commerce/AP2 - Google's Agent Payments Protocol (Sept 2025 launch, 60+ payment-network partners) — cryptographically signed mandates (SD-JWT / ECDSA) proving a human authorized a spend; donated to the FIDO Alliance (Apr 28, 2026), whose Payments WG now builds on AP2; v0.2 reorganized the mandate model into Checkout + Payment Mandates with Open (pre-authorized constraints) / Closed (finalized) states and added "Human Not Present" autonomous payments; Python SDK (Pydantic) + reference scenarios in Python/Go/Kotlin and a sample Android app (Apache-2.0, 3.2k+ stars, active 2026)
-
Universal-Commerce-Protocol/ucp
- Universal Commerce Protocol (Google + Shopify, announced at NRF Jan 11, 2026) — the open commerce layer atop the payment rails: merchants host a signed, versioned capability profile at a well-known discovery URL (
/.well-known/ucp, transport-agnostic across REST and MCP), and agents run four standardized stages — product discovery, capability negotiation, checkout, post-purchase — with composable primitives for Checkout (carts, tax sessions), Identity Linking (OAuth 2.0), Order lifecycle webhooks, and Payment Token Exchange between PSPs; official machine-readable JSON schemas, Python SDK (Pydantic, auto-generated from schemas), and a conformance suite, spec versioned by date (current2026-08-25); Shopify enabled UCP by default for all merchants with Shop Pay tokenized checkout (June 2026), with Etsy, Target, Walmart, Wayfair, Best Buy and Visa/Mastercard/Amex/Stripe/Adyen/PayPal in the ecosystem; powers agentic checkout in Google AI Mode, Gemini apps, and Microsoft Copilot (Apache-2.0, 3.4k+ stars, actively maintained 2026) - bitrefill/awesome-agentic-payments - protocols, specs, SDKs, and tools for the agentic commerce stack (2026)
Agent Identity
- agntcy/identity - AGNTCY Identity (the identity workstream of Cisco-donated AGNTCY, a Linux Foundation project backed by 75+ companies) — onboard, create, and verify identities for agents, MCP servers, and multi-agent systems on W3C DIDs + Verifiable Credentials 2.0; each agent gets a resolvable ID and Agent Badges (signed VCs binding the agent to its OASF schema definition or A2A Agent Card — capabilities are attested, not claimed), multiple badges enable task-scoped least privilege, bring-your-own-identity via Identity Providers (Okta) or A2A cards, Vault-backed signing keys, identity node + CLI tooling; 44 of 48 AGNTCY repos pushed in 2026 (Go, Apache-2.0, actively maintained)
Agent Memory
Agent Skills
-
anthropics/skills
- Official collection + spec (
/spec/agent-skills-spec.md) - VoltAgent/awesome-agent-skills - 1000+ community skills, works across all major platforms
- vercel-labs/agent-skills - Vercel's official skills
- Agent Skills Docs — Anthropic - Official docs & spec
- Equipping Agents for the Real World — Anthropic - Announcement post
- Skills vs MCP — LlamaIndex - When to use which
- obra/superpowers - Jesse Vincent's complete software-development methodology for coding agents, built on composable auto-triggering Agent Skills — spec elicitation, chunked design sign-off, implementation planning, true red/green TDD, and subagent-driven development; agents can work autonomously for hours without deviating from the approved plan; works with Claude Code, Codex, Cursor, Gemini CLI, OpenCode, Pi, and 10+ harnesses (MIT, 296k+ stars, actively maintained Oct 2026)
-
mattpocock/skills
- Matt Pocock's (Total TypeScript / AI Hero) "Skills for Real Engineers" — the most-starred community Agent Skills collection, straight from his
.agentsdirectory: small, composable, model-agnostic skills as an explicit counterpoint to heavyweight process frameworks (GSD/BMAD/Spec-Kit) that own the loop and take control away; ~20 engineering skills (tdd, code-review, diagnosing-bugs, domain-modeling, to-spec, implement-spec, pr, retro, wayfinder…); installs as a self-updating plugin on Claude Code, Codex, GitHub Copilot (CLI + VS Code), and Gemini CLI, or copies editable files into a project via skills.sh (MIT, 313k+ stars, created Feb 2026, v1.3.2 Oct 2026, pushed daily, actively maintained Nov 2026) -
Archify
- Agent skill that turns an idea, plan, question, or codebase into a beautiful interactive HTML diagram — architecture, sequence, and data-flow views plus explainer pages (conclusion first, one panel per question, claims cite numbered sources, repo excerpts pinned to a commit); a typed intermediate JSON source the agent keeps for targeted iteration (
add Redis,highlight the rollback path), deterministic layout with validation receipts, Architecture Delta reviews (validated Before/Delta/After snapshots with machine receipts), one self-contained shareable HTML file with PNG/WebP/JPG long-image and print-to-PDF export; installs by having the agent read itsllms.txt— works with Claude Code, Codex CLI, Cursor, and OpenCode (also a Kimi Work plugin and Supercode Editor's Choice skill); 130K+ installs via skills.sh (Apache-2.0, 94k+ stars, created Apr 2026, v3.2.0 Oct 2026, pushed daily, actively maintained Nov 2026) - MetcalfSolutions/Satori - Reflective dialogue prompt and wisdom companion skill, built to be outgrown: single-lens introspection, SQLite memory vault, and trigger discipline that stays silent during pure coding work
Agent Training (RL Post-Training)
Frameworks
- LangGraph - LangChain
- CrewAI - CrewAI
- Magentic-One - Microsoft
- OpenAI Agents SDK - OpenAI
- OpenAI Agents SDK for JS/TS - OpenAI
- GitHub Agentic Workflows (gh-aw) - GitHub
- Google ADK - Google
- Claude Code - Anthropic
- karpathy/autoresearch - Karpathy
- Microsoft Agent Framework - Microsoft
- openai/codex - OpenAI
- DeerFlow 2.0 - ByteDance
- smolagents - HuggingFace
- browser-use - OSS
- Mastra - Gatsby team
- PraisonAI - Mervin Praison
- Portia AI - Portia Labs
- Paperclip - Paperclip AI
- Gemini CLI - Google
- oh-my-codex - Yeachan Heo
- Hermes Agent - Nous Research
- claw-code - UltraWorkers
- PilotDeck - OpenBMB / THUNLP / ModelBest / AI9Stars
- AOS CE - Unicity
- agent-browser - Vercel
- nanobot - HKUDS
- Flue - Astro
- Agno - Agno
- Qwen-MM-Plugins - Alibaba/Qwen
- codebase-memory-mcp - DeusData
- TencentDB Agent Memory - Tencent Cloud
- agentmemory - rohitg00
- eve - Vercel
- kimi-code - Moonshot AI
- Orca - Stably
- herdr - herdr.dev
- OpenSRE - Tracer Cloud
- DeepSeek Harness - DeepSeek
- TrueForge - TrueFoundry
- OpenBot - CopilotKit
- qm - YC Software
- Omnigent - Omnigent AI
- Claude Agent SDK - Anthropic
- commerce-agents - Anthropic
- OpenHuman - TinyHumans
- phone-harness - ShawnPana
- Artemis - Google
- Dormice - BitMiracle AI
- OpenConnector - Oomol
- OpenClaw - OpenClaw Foundation
- OpenHands - All Hands AI
- REA - morluto
- deja-vu - vshulcz
- TradingAgents - Tauric Research
Harness Engineering
- Harness Engineering — OpenAI - Official OpenAI post: "leveraging Codex in an agent-first world"
- The Importance of Agent Harness in 2026 — Philipp Schmid - "The harness is the dataset. Competitive advantage is the trajectories it captures."
- Skill Issue: Harness Engineering for Coding Agents — HumanLayer - Sub-agents as context firewalls, practical patterns
- Effective Harnesses for Long-Running Agents — Anthropic - Long-running agent design
- SethGammon/Citadel - Production harness: 4-tier routing, parallel worktrees, lifecycle hooks, 6 skills
- langchain-ai/deepagents - LangChain's opinionated deep agent harness (used in TerminalBench)
- Building a C Compiler with Parallel Claudes — Anthropic - How Anthropic used parallel Claude sub-agents to build a C compiler — generator/evaluator harness patterns
- strukto-ai/mirage - Unified virtual filesystem for AI agents — mounts S3, GDrive, Slack, Gmail, Redis as one tree; agents use bash across every backend; Python/TypeScript SDKs, cache, snapshots (May 2026)
- QoderAI/better-harness - Open-source loop/harness improvement skill — turns project and session evidence into prioritized improvements and verifiable next steps for Claude Code, Codex, Cursor, and other coding agents (July 2026)
- harness-engineering - Ryan Lopopolo's anthology, field guide, and agent context bundle for harness engineering — shaping context and tools so agents can recover intent, operate systems, respect authority, prove outcomes, and leave the next run better equipped (CC-BY-4.0, July 2026)
-
loop-engineering
- "Stop prompting. Design the loop." — practical patterns, starters & CLI (
loop-audit,loop-init,loop-cost) for systems that discover work, hand it to agents, verify results, and persist state across Claude Code, Codex, Grok, and OpenCode; report-only week one, scores loops on a "Loop Ready" rubric (MIT, 11k+ stars, June 2026, actively maintained) - ECC - The agent harness performance optimization system — skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond (MIT, 242k+ stars, Jan 2026)
- SoL-Pi - NVIDIA's efficiency extension for the Pi coding agent — four opt-in harness mechanisms distilled from scaled auto-research loops (arXiv 2609.20519): Action Fusion (run follow-up validation in the same tool call), ObservationPack (stable handles with exact paged recall for repeated large outputs), evidence-preserving reducer (long logs → compact receipts where every quotation is verifiable against the archive), and online context compaction gated by economics/window pressure; no patches to Pi, evidence always preserved (TypeScript, MIT, 2.9k+ stars, Sept 2026, actively maintained)
- The Anatomy of an Agent Harness — LangChain - Component-by-component breakdown
- code-yeongyu/oh-my-openagent - Multi-harness coding-agent project for OpenCode and Codex CLI, with lifecycle hooks, skills/MCP integration, context tooling, and opt-in team execution in its OpenCode edition
MCP — Model Context Protocol
- modelcontextprotocol.io
- github.com/modelcontextprotocol/servers
-
PrefectHQ/fastmcp
- FastMCP (Prefect) — the de-facto Python framework for building MCP servers and clients — decorator-based
@mcp.toolAPI, server composition (mount + proxy), OpenAPI import/export, OAuth 2.1, OpenTelemetry, hot reload, MCP Apps (interactive UI in tool responses), background tasks, and a built-in test client; tracks the current spec revision (2026-07-28) on MCP SDK 2.x; from Jeremiah Lowin, the original author of the reference Python MCP SDK (Apache-2.0, ~28k stars, v4.1 Oct 2026, actively maintained) -
modelcontextprotocol/registry
- — the official discovery layer (powers registry.modelcontextprotocol.io): ~10,000 server records by mid-2026, reverse-DNS-verified namespaces (
io.github.user/server), REST API for programmatic in-agent tool discovery, and self-hostable for private enterprise catalogs (Go, MIT, preview Sept 2025 → matured through 2026, actively maintained) -
upstash/context7
- Context7 (Upstash) — the de-facto capability MCP server for coding agents: injects current, version-specific library documentation and code examples into the context window, ending hallucinated APIs and year-old code from stale training data —
resolve-library-id+query-docstools over a continuously crawled, version-pinned doc index (ask for "Next.js 16" and get Next.js 16 docs); use it as a hosted remote MCP endpoint (https://mcp.context7.com/mcp, zero install) or self-host the server, and newer setups install thectx7CLI + agent skill vianpx ctx7 setup(no MCP required); native support for Claude Code, Cursor, VS Code, ChatGPT, Gemini, and pi.dev, with config recipes for 30+ clients (TypeScript, MIT, 64k+ stars, v4.5 line Nov 2026, pushed daily, actively maintained)
Sandboxed Execution
- E2B -
- microsandbox -
- OpenShell -
Context Engineering
- Effective Context Engineering for AI Agents — Anthropic
- Context Engineering Guide — Prompt Engineering Guide
- davidkimai/Context-Engineering - — first-principles handbook on context design, orchestration, and optimization
- Meirtz/Awesome-Context-Engineering - curated papers, frameworks, and implementation guides
- NanoNets/Graft - — Open-source codebase context layer for coding agents — builds a persistent, queryable code graph and pulls matching nodes into each prompt; 46% fewer tool calls, 42% token savings, 60% faster on a 162-run benchmark, 66% SWE-bench Verified (vs 54% cold); supports Claude Code, Cursor, Codex, Gemini via MCP/CLI (MIT, July 2026)
- muratcankoylan/Agent-Skills-for-Context-Engineering - — comprehensive, MIT-licensed collection of Agent Skills for context engineering, multi-agent architectures, and production agent systems — context fundamentals/degradation/compression, memory systems, tool design, harness engineering, self-improvement loops; cited in academic research as foundational work on static skill architecture (17.9k+ stars, actively maintained, 2026)
- redhat-et/ripwire - — "The ripgrep of AI context" (Red Hat): zero-dependency C++23 CLI + MCP server that lets coding agents find what they need without reading the whole repo — tree-sitter signatures at 74.7% fewer bytes than bodies, blast-radius queries, tests-to-run suggestions, and post-change verification with quality deltas; every heuristic guess labelled, every context loss published (Apache-2.0, 2.4k+ stars, actively maintained Sept 2026)
- JuliusBrussee/caveman - — "why use many token when few token do trick" — the viral caveman-voice token optimizer for coding agents: six composable parts behind one local install — Say less (ASD-STE100-derived terse answer-first voice; code, commands, and paths stay verbatim), Read less (logs/JSON/CSV/test output shrink before the model reads them, 33.2% fewer input tokens per session), Auto (per-ask model/effort routing, 54% cheaper at the same score), Blocks (agent keeps the scripts it writes and reruns them, 397 → 76 tokens for one test run), cache-warm waste fixes, and compressed web fetches (129.8× smaller pages); works with Claude Code, Codex, Gemini CLI, OpenCode, and 30+ agents via proxy + Agent Skill; user prompts are never rewritten (Adobe Research [CAVEWOMAN, arXiv 2606.24083] shows cavemanning your prompt backfires and cites this repo — 1.4–2.4× cheaper across eight models); externally validated by a JetBrains paired A/B (86 tasks, no measurable quality loss, p = 0.82) and an Elasticsearch Labs remake (63.6% fewer response tokens, "zero information loss") (Go, Apache-2.0, 117k+ stars, v4.0, pushed daily, actively maintained Nov 2026)
Frameworks
Automatic Prompt Optimization
- TextGrad -
- GEPA -
- Hermes Agent Self-Evolution -
- Reef -
Eval & Observability
- DeepEval -
- Langfuse -
- Phoenix -
- Tracely-ai -
Eval & Testing
- OpenAI Evals -
- promptfoo -
- Terminal-Bench - —
Low-Code & Workflow Platforms
Prompt Programming
Red Team & Security
- garak -
- OpenAI: Prompt Injection Defense - —
- The Promptware Kill Chain - —
- Microsoft Agent Governance Toolkit -
- agent-drift -
- Agents of Chaos (2026) - Two-week red-team study of live autonomous agents; 11 real attack categories
- Jailbreaking LLMs & VLMs: Unified Survey (2026) - First unified survey spanning both LLM and VLM jailbreak mechanisms
- T3MP3ST -
- OpenAI Codex Security -
- SkillSpector -
- Orca AI Incident Archive -
- SoK: Rethinking Jailbreaking in the Era of Agentic AI (2026) - Attack and defense taxonomy for agentic systems; SecurityCube evaluation framework
- SoK: GPT vs. DeepSeek Jailbreak Resilience (2026) - Comparative security analysis across GPT and DeepSeek model families
- LLM Jailbreak Field Guide (2026) - Practitioner-oriented taxonomy: roleplay, encoding, multi-turn, many-shot, refusal suppression, fake policy
- 面向大语言模型的越狱攻击与防御综述 (2025) - 中文综述,分人工设计/模型生成/对抗优化三类攻击
- 生成式大模型越狱攻击安全性研究综述 (2026) - 中文综述,涵盖攻击与防御策略
- The Art of the Jailbreak (2026) - Compositional jailbreak dataset + OPTIMUS training-free evaluation metric
- GPT-Red: Automated Red Teaming via Self-Play at Scale (2026) - OpenAI's internal model trained by self-play RL for automated red teaming
- Climbing the Hill: Curriculum RL for Prompt Injection Red-Teaming (2026) - Curriculum reinforcement learning for black-box prompt injection against frontier LLMs
- Where Do Reasoning Models Refuse? (2026) - Maps refusal behavior differences in reasoning vs. non-reasoning models
- Jailbroken Frontier Models Retain Their Capabilities (2026) - Shows that jailbroken models keep full capabilities, making jailbreaks more dangerous
- Breaking Multimodal LLM Safety via Video-Driven Prompting (CVPR 2026) - Video inputs bypass safety more effectively than images
- Improving Alignment and Robustness with Circuit Breakers (2024) - Representation Rerouting defense; foundation for representation-engineering safety
- Best-of-N Jailbreaking (NeurIPS 2025) - Power-law scaling of random-perturbation attacks; 89% ASR on GPT-4o at 10K samples
- DeepSeek Under Attack: Jailbreak Attacks and Prompt-Level Defenses (2026) - R1 more vulnerable than V3; prompt-level defenses analysis
- MultiBreak: Multi-turn Jailbreak Benchmark (2026) - Scalable multi-turn benchmark; +54% ASR vs. prior datasets on DeepSeek-R1-Distill
- NRT-Bench: Nuclear Plant Safety Red-Teaming (2026) - Multi-turn red-teaming in simulated safety-critical environment; 8.7–12.1% critical failure rate
- PandaGuard: Systematic Jailbreak Evaluation (2026) - Systematic evaluation framework covering attack and defense pairings
- Concerns in Benchmarking Jailbreaks (2026) - Judge choice drives ASR: same attack scores 96% (keyword) vs. 2.7% (Qwen3 evaluator)
- TEMPEST Replication at Scale (2025) - 79/100 behaviors succeed against 8+ frontier models; limited protection from safety training
- Adversarial Attacks on Multimodal LLMs: Survey (2026) - Comprehensive survey of adversarial attacks on multimodal LLMs
- Jailbreak Attacks and Defenses against Multimodal Models: Survey (2025) - Survey covering text, image, audio, and video jailbreak attack surfaces
- Prompt Injection on Agentic Coding Assistants (2026) - 78 studies reviewed; 30+ CVEs across major platforms; MCP tool poisoning taxonomy
- MCP Threat Modeling: Tool Poisoning (2026) - Threat model for MCP tool poisoning; compares 7 MCP client implementations
- DARWIN: Evolving Jailbreak Adversary and Guardrail (2026) - Co-evolutionary attack-defense loop; DARWIN-Guard reaches 95% unsafe recall
- AdversaBench: Multi-Judge Confirmed Red Teaming (2026) - Automated pipeline with three-judge confirmation; every seed produces confirmed failure
- SkillAttack: Automated Agent Skill Red Teaming (2026) - Closed-loop adversarial testing of LLM agent skills
- Learning to Attack and Defend via GRPO (2026) - Co-train attacker and defender with GRPO; per-turn rewards in multi-turn attacks
- Autonomous Adversary: Red-Teaming in the Age of LLMs (2026) - Survey of LLM agents for red-team operations; governance and realistic evaluation
- Colluding LoRA: Composite Fine-Tuning Attacks (ICLR 2026) - Multiple safe-looking LoRA adapters jointly dismantle alignment
- Open-Weight Fine-Tuning Defenses Are Susceptible (2026) - Current defenses against alignment removal fail under simple attacks
- Harmful Fine-tuning Attacks and Defenses: Survey (ACM 2026) - Comprehensive ACM survey of fine-tuning-based attacks and defenses
- Guardrail Attribution: 0%, 45%, or 99%? (2026) - A guardrail's own share of the refusals it is credited with is often unclear
- ForesightSafety Bench (2026) - Frontier risk evaluation; DeepSeek-V3.2 11.67% baseline vulnerability, Claude near zero
Tool Use & Reliability
Papers
Agent Evaluation
- Signals: Trajectory Sampling and Triage for Agentic Interactions (April 2026) - Lightweight signal-based taxonomy for sampling informative agent trajectories post-deployment — 82% informativeness vs 54% random; organizes signals across interaction, execution, and environment dimensions; 6.2k HF likes
- Agent Psychometrics: Task-Level Performance Prediction (April 2026) - Shifts evaluation from simple QA to multi-turn agentic assessment; newer benchmarks like SWE-bench Verified and Terminal-Bench test iterative agent behavior with execution feedback
- YC-Bench: Benchmarking AI Agents for Long-Term Planning (April 2026) - Evaluates whether LLM agents maintain strategic coherence over long horizons — simulated startup over one-year horizon spanning hundreds of turns; tests consistent execution
- When Users Change Their Mind: Evaluating Interruptible Agents (April 2026) - Tests agent ability to handle user interruptions during mid-task execution — critical requirement for realistic deployment in dynamic environments
- SWE-CI: Evaluating Agents on Codebase Maintenance via CI (2026) - First CI-loop benchmark for long-term codebase maintainability — 100 tasks spanning 233 days and 71+ consecutive commits; shifts evaluation from static single-fix to dynamic long-horizon reasoning
- SWE-Skills-Bench (2026) - 565 real-world SE tasks measuring whether agent skills actually improve outcomes — 39/49 public skills give zero gain; average improvement only +1.2%; reveals fundamental gap in skill design
- LongCLI-Bench: A Benchmark for Long-Horizon Agentic Programming in the CLI (2026) - Benchmarks terminal-based coding agents on long-horizon programming tasks that require sustained planning, repo navigation, debugging, and recovery over many steps instead of single-fix patches
- ProjDevBench: Benchmarking AI Agents on End-to-End Software Project Development (2026) - Evaluates whether agents can build complete software projects from requirements to implementation and validation, rather than solving isolated bug-fix tasks; targets end-to-end project delivery realism
- LiveClawBench: Benchmarking LLM Agents on Complex, Real-World Assistant Tasks (April 2026) - Evaluates agents on compositional, real-world assistant tasks requiring planning, tool use, and recovery — closer to production deployment scenarios than static QA benchmarks
- RiskWebWorld: GUI Agents in E-commerce Risk Management (April 2026) - Realistic interactive benchmark for GUI agents in high-stakes professional workflows — 100 real-world e-commerce risk scenarios testing sequential decision-making under uncertainty
- OccuBench: Real-World Professional Tasks via Language World Models (April 2026) - 100 professional task scenarios across 10 industries and 65 domains — evaluates AI agents on realistic occupational workflows using language world models for environment simulation
- EpiBench: Multi-turn Research Workflows for Multimodal Agents (April 2026) - Benchmarks multimodal agents on episodic scientific research workflows — literature search, figure extraction, cross-paper synthesis; built on smolagents with persistent memory and tool use
- Ask Early, Ask Late, Ask Right: When Does Clarification Timing Matter for Long-Horizon Agents (May 2026) - First forced-injection framework measuring how clarification value changes over the execution trajectory across goal/input/constraint/context dimensions; 6,000+ runs, 4 frontier models, 3 benchmarks; finds goal clarifications lose nearly all value after 10% execution, input clarifications retain value through ~50%, and deferring any clarification past mid-trajectory degrades performance below never asking; cross-model Kendall tau 0.78–0.87 confirms task-intrinsic timing curves
- Reasoning Is Not Free: Robust Adaptive Cost-Efficient Routing for LLM-as-a-Judge (May 2026) - ICML 2026: controlled comparisons show reasoning judges substantially improve accuracy on structured-verification tasks (math, coding) but yield limited or negative gains on simpler evaluations while costing significantly more compute; proposes RACER, a distributionally-robust routing policy that dynamically selects between reasoning and non-reasoning judges under a fixed budget via a KL-divergence uncertainty set, with theoretical guarantees including uniqueness of the optimal policy and linear convergence of the primal–dual algorithm
Agent Reliability
- Towards a Science of AI Agent Reliability (2026) - 12 concrete reliability metrics across consistency, robustness, predictability, safety — capability gains ≠ reliability gains
- Agentic Reasoning for LLMs (2026) - Comprehensive survey: 3-layer framework (single-agent capabilities → self-evolving agents → multi-agent coordination); 202 Hugging Face likes
- Why Do Web Agents Fail? A Hierarchical Planning Perspective (2026) - Decomposes web agent behavior into high-level planning, low-level grounding, and replanning — PDDL-structured plans outperform NL plans but grounding remains the dominant bottleneck; a single round of exploratory replanning substantially improves task success
- Claw-Eval: Trustworthy Evaluation of Autonomous Agents (April 2026) - End-to-end evaluation suite with 300 human-verified tasks across 9 categories — trajectory-aware grading over 2,159 rubric items; finds vanilla LLM judges miss 44% of safety violations and 13% of robustness failures
- TimeSeek: Temporal Reliability of Agentic Forecasters (April 2026) - Benchmark built from 150 regulated prediction markets evaluated at 5 lifecycle checkpoints — models are most competitive early and on high-uncertainty markets; search improves pooled accuracy but degrades 12% of conditions