awesome-harness-engineering
Awesome list for AI agent harness engineering: tools, patterns, evals, memory, MCP, permissions, observability, and orchestration.
https://github.com/ai-boost/awesome-harness-engineering
Last synced: 5 days ago
JSON representation
-
Acknowledgments
-
Adjacent Collections
-
-
Design Primitives
-
Agent Loop
- ReAct: Synergizing Reasoning and Acting in Language Models - acting cycle.
- LangGraph — Low Level Concepts - loop state for resumption.
- Unlocking the Codex Harness: How We Built the App Server - dive into the Item/Turn/Thread protocol (JSON-RPC/JSONL over stdio) that exposes the Codex harness to every client surface. The most direct first-party account of why approval flows, streaming diffs, and thread persistence demand a purpose-built protocol — and why MCP's tool-oriented model proved insufficient for these requirements.
- Improving Deep Agents with Harness Engineering - only changes moved their coding agent from rank 30 to top 5 on Terminal Bench 2.0 with no model swap: structured verification loops, context injection (directory maps + time budget warnings), loop-detection middleware, and a "reasoning sandwich" concentrating maximum thinking at planning and verification phases. The most concrete published demonstration that harness design is the primary performance lever, not model capability.
- Agents Learn Their Runtime: Interpreter Persistence as Training-Time Semantics - time semantics produces either 80% missing-variable errors (model expects state that doesn't persist) or 3.5× token overhead (model redundantly recomputes state it expects to already have). Persistence is a learned semantic that must be honored at deployment, not a free runtime choice.
- Real-Time Deadlines Reveal Temporal Awareness Failures in LLM Strategic Reasoning - constrained tasks. Indicates temporal semantics as a learned behavior that must be integrated into harness-level context (current time, deadlines, time budgets) rather than assumed from capability alone.
- A Scheduler-Theoretic Framework for LLM Agent Execution - source LLM agent projects showing 60% adopt the Agent Loop pattern. Proposes a formal scheduler framework that maps execution patterns (Agent Loop, Event-driven, State-machine, Graph/flow, Hybrid) onto a unified control model, making the controllability/expressiveness/implementability trade-offs explicit. Essential reading for choosing the right loop architecture rather than defaulting to the simplest pattern.
- Confucius Code Agent (CCA) - grade coding agent from Meta/Harvard built on the Confucius SDK, which structures harness design around three perspectives: Agent Experience (AX), User Experience (UX), and Developer Experience (DX). Features a unified orchestrator with advanced context management, persistent note-taking for cross-session learning, and a meta-agent that automates build-test-improve cycles. Achieves 59% Resolve@1 on SWE-Bench-Pro, exceeding prior research and commercial baselines.
- The Design Space of Today's and Future AI Agent Systems - engineering of Claude Code's architecture revealing five-stage progressive compaction (budget reduction → snip → microcompact → context collapse → auto-compact), subagent isolation with rebuilt permission contexts, and a 27-event-type hook pipeline. The most detailed public analysis of a production agent loop's internal design decisions — essential for understanding how context pressure, safety, and delegation are handled at scale.
- deepclaude - compatible backends while preserving the same UX. The strongest practical evidence that loop architecture — not model identity — determines agent behavior, and a concrete starting point for building backend-agnostic harnesses. 
- The Coding Harness Behind GitHub Copilot in VS Code - provider model routing across Anthropic, Google, OpenAI, xAI, and Mistral, and the VSC-Bench eval suite with PR-gated assessment. The clearest published account of how a major product treats harness changes as first-class code review criteria — "the model is the engine, the harness is the car."
- statewright - ended loops into deterministic state transitions. The research result is striking: local models went from 2/10 to 10/10 passing on a SWE-bench subset purely by shrinking the tool space, proving that loop structure — not model size — is the binding constraint. 
- Hooks – Codex - hook framework for Codex: inject deterministic scripts at `SessionStart`, `PreToolUse`, `PostToolUse`, and other loop events to enforce guardrails, audit actions, and customize agent behavior without relying on prompt-level trust. A concrete reference for programmable harness governance.
- Life-Harness - aware runtime harness that improves frozen LLM agents by adapting the model-environment interface across four layers: environment contract, procedural skills, action realization, and trajectory regulation. The key result is that harness-side adaptation transfers across 18 model backbones, proving that many agent failures are interface mismatches rather than reasoning deficits. 
- Introducing dynamic workflows in Claude Code - line Bun Zig-to-Rust port. The key harness insight is that the plan lives in executable code rather than the model's context window, scaling the agent loop to work that would otherwise exceed a single context window.
- Getting started with loops - based, goal-based (`/goal`), time-based (`/loop`, `/schedule`), and proactive loops. The framework for matching loop primitive to task shape — and the emphasis on deterministic stop conditions and token budgets — makes it a concise reference for choosing the right loop abstraction instead of defaulting to a single conversational turn cycle.
- AgentSPEX - source specification and execution language for LLM-agent workflows: declarative YAML with typed steps, branching, loops, and explicit state management, backed by a Docker sandbox with 50+ MCP tools, checkpointing, and trajectory logging. A concrete reference for turning ad-hoc agent loops into version-controlled, reproducible harness artifacts. 
- Loop Engineering - tool starter kits, and CLI tools that score readiness, scaffold state, estimate cost, detect drift, and isolate worktrees. The clearest open-source resource for moving from one-off prompting to durable, observable agent loops. 
-
Context Delivery & Compaction
- Effective Context Engineering for AI Agents
- Compaction — Claude API Docs - side context compaction: automatically summarizes older context when approaching the window limit. Reduced token consumption by 84% in a 100-turn web search eval while allowing agents to complete workflows that would otherwise hit context limits.
- LLMLingua - 2 adds 3–6x speed gains, making it viable for latency-sensitive agent loops. 
- Autonomous Context Compression - controlled (compacting at a fixed token threshold) to agent-controlled: agents call a dedicated tool to trigger compression when strategically appropriate — between tasks or before consuming large inputs. Eliminates the failure mode where reactive-at-limit compaction interrupts agents mid-subtask and corrupts in-flight reasoning state.
- Active Context Compression: Autonomous Memory Management in LLM Agents - enforced policy to a model-controlled action. Produces 22.7% token reduction with no accuracy loss on long-horizon tasks; the core contribution is making the compression unit semantically coherent (the agent decides what knowledge is worth preserving) rather than mechanically token-budget-driven.
- context-mode - read tool calls with one script execution — is a concrete harness pattern for turning context pressure into a programming problem rather than a compression problem. 
- Making Agent-Friendly Pages with Content Negotiation - facing HTML URL. This is a real harness primitive, not just a docs trick: it removes boilerplate before it ever enters the context window and gives agents cleaner, cheaper inputs without custom scrapers.
- A-RAG: Scaling Agentic Retrieval-Augmented Generation via Hierarchical Retrieval Interfaces - design problem: instead of injecting retrieved documents into context at pipeline time, expose three retrieval tools (keyword search, semantic search, chunk read) and let the agent pull information incrementally as each reasoning step requires it. The key harness decision is architectural — retrieval becomes a tool call in the agent loop, not a preprocessing step — which means the agent's reasoning can adaptively narrow scope rather than processing everything injected upfront.
- LLM Readiness Harness: Evaluation, Observability, and CI Gates for LLM/RAG Applications - grade evaluation harnesses: evaluation gates that block deployment, observability instrumentation that tracks all agent decisions, and CI integration patterns that catch regressions before they reach users. Essential reading for organizations deploying multiple agents in parallel where a single harness failure can cascade.
- ByteRover: Agent-Native Memory Through LLM-Curated Hierarchical Context - curated hierarchical context management for agents where the model itself learns to weight information importance across multiple hierarchy levels. Reduces token overhead through learned relevance filtering without sacrificing comprehension. Directly applicable to any harness where context budget is the limiting factor — letting the model curate what belongs in active memory vs. what can be retrieved on-demand.
- Claude Code Compaction: How Context Compression Works - dive into Claude Code's automatic compaction mechanism: what survives (current task, recent errors, file names) vs. what gets lost (initial instructions, intermediate decisions, style rules). Key harness insight: never rely on compaction for critical rules — move them to `CLAUDE.md` where they live in the system prompt and survive any compression. Essential practical guidance for anyone running long-session agents.
- Token Savior - savior?style=flat-square&label=★&color=yellow)
- Trellis - platform adapter layer turns vendor-specific harness configuration into a portable team practice rather than a per-tool hack. 
- OpenViking - evolving layer that restructures context based on usage patterns makes it a rare example of context infrastructure that improves autonomously rather than requiring constant manual curation. 
- DESIGN.md - readable design tokens (YAML front matter) combined with human-readable design rationale (markdown prose) give agents a persistent, structured understanding of design constraints without requiring custom tool chains. 
- codebase-memory-mcp - performance code intelligence MCP server that full-indexes repositories into a persistent knowledge graph via tree-sitter AST analysis across 66 languages. Replaces dozens of file-read/grep cycles with sub-millisecond structured queries, cutting active tokens by 120× and turning codebase navigation from a context-pressure problem into a pointer-chasing problem. 
- Mirage - by-side as a single virtual filesystem so agents interact with every backend through familiar bash commands instead of learning N distinct APIs. The key harness insight: LLMs are already fluent in `grep`, `cat`, and `cp` — leveraging that vocabulary eliminates tool-schema bloat and makes cross-service pipelines compose as naturally as local shell scripts. 
- dirac - run/dirac?style=flat-square&label=★&color=yellow)
- MinishLab/semble - language retrieval, cutting active tokens by ~98% while keeping 99% of a transformer-based retriever's accuracy. Ships as an MCP server and CLI, runs on CPU with zero external dependencies — the right drop-in for any coding agent harness struggling with context pressure. 
- harness-experimental - level operating harness that turns any software repo into an agent-ready workspace: structured `AGENTS.md`, `HARNESS.md`, and `FEATURE_INTAKE.md` give agents the missing project context — where to start, what the product contract says, how risky the change is, and which decisions future agents should inherit. The most concrete open-source implementation of "coding agents need better repositories, not just better prompts." 
- headroom - in layer for any harness where bulky tool returns are the primary context pressure source. 
- Context7 - to-date, version-specific library documentation directly into agent context, eliminating hallucinated APIs and outdated code examples caused by stale training data. Ships as both a `ctx7` command-line tool and an MCP server with `resolve-library-id` and `query-docs` tools. 
- Context Pruning for Coding Agents via Multi-Rubric Latent Reasoning - turn coding agent tasks while improving Exact Match by up to +3.5, demonstrating that coding-agent context pruning needs domain-specific rubrics rather than generic compression.
- OpenWiki - readable wikis for codebases or purpose memory, turning documentation drift into a versioned, automatable harness artifact. Emits Google Open Knowledge Format bundles so curated context stays portable across agents and can be kept fresh via CI. 
- PRO-LONG - horizon agents: the harness appends all observations to a structured log and lets the agent search it with code instead of relying on fixed summarization or compaction policies. Achieves 4.2–5.8× token reduction and matches or exceeds specialized harnesses on ARC-AGI-3, showing that long-horizon context management can be a programmable retrieval problem rather than a compression problem. 
- Graft - English system explanations and code relationships, then rides along inside Claude Code, Cursor, Codex, and Gemini via MCP and statusline hooks so the agent stops rediscovering the repo every session. The published SWE-bench Verified and efficiency benchmarks make it the clearest recent demonstration that context delivery for coding agents is a navigation problem, not just a compression problem. 
- Autonomous Context Compression - controlled (compacting at a fixed token threshold) to agent-controlled: agents call a dedicated tool to trigger compression when strategically appropriate — between tasks or before consuming large inputs. Eliminates the failure mode where reactive-at-limit compaction interrupts agents mid-subtask and corrupts in-flight reasoning state.
- ktx - improving executable context layer for data and analytics agents: it ingests warehouses, BI tools, and wikis to build a semantic layer with approved metrics, joinable columns, and resolved fan/chasm traps, then serves the result to Claude Code, Codex, and Cursor through MCP. Fills the gap where general-purpose agents invent metric logic on every question — it turns warehouse querying from a prompt-guessing problem into a governed, retrievable context problem. 
-
Debugging & Developer Experience
- AgentOps - source agent engineering platform (YC W24) with session replay, cost tracking, and failure detection across 10+ frameworks including CrewAI, LangGraph, and OpenAI Agents SDK. The step-by-step execution graph and cross-session metrics make it the most practical debugging layer for multi-agent systems in production. 
- claude-devtools - source DevTools for Claude Code that reconstructs hidden session internals from local logs: per-turn token attribution across 7 context categories, full subagent execution trees with cost breakdowns, and syntax-highlighted diffs for every tool call. Essential because Claude Code's default UI deliberately collapses tool details and thinking steps — this tool restores the visibility harness engineers need to debug context pressure, subagent delegation, and token budget leaks. 
- Syncause/debug-skill - based repair with reviewable results. 
- AgentTrace: Causal Graph Tracing for Root Cause Analysis in Multi-Agent Systems - agent execution traces using causal graph analysis rather than LLM inference. Processes traces in 0.12 seconds (69× faster than LLM-based analysis) with 93.6–95.8% accuracy across 550 synthetic failure scenarios. Distinguishes root causes from downstream symptom propagation — the key capability missing from most trace-inspection debugging workflows.
- TraceCoder: A Trace-Driven Multi-Agent Framework for Automated Debugging of LLM-Generated Code - agent debugging loop: Instrumentation Agent injects diagnostic probes, Analysis Agent performs causal trace diagnosis with a Historical Lesson Learning Mechanism (HLLM) that distills insights from prior failures, and Repair Agent executes validated fixes with rollback on regression. Achieves up to 34.43% relative improvement in Pass@1 over baselines.
- AgentRx: Systematic Debugging for AI Agents - cause analysis of agent failures: trajectory normalization, constraint synthesis from tool schemas, and constraint-guided evaluation. Achieves 23.6% better failure localization than existing approaches with a 115-trajectory annotated benchmark. Shifts agent debugging from manual log inspection to systematic constraint-based diagnosis — a reference design for harness-level observability that surfaces *why* an agent failed, not just *that* it failed. 
- Debugging Deep Agents with LangSmith - fetch (CLI for piping trace data to coding agents). Key insight: debugging deep agents requires AI-assisted trace analysis — the volume of data these systems produce exceeds human capacity.
- Where LLM Agents Fail and How They Can Learn From Failures (AgentDebug) - level failures. The AgentDebug framework isolates root-cause failures and provides corrective feedback, achieving +24% higher all-correct accuracy. The Agent Error Benchmark (annotated trajectories from ALFWorld, GAIA, WebShop) is the first systematic failure dataset for agent debugging.
- AgentPrism - source React component library (Evil Martians) that transforms OpenTelemetry trace data into interactive visualizations: tree view, timeline/Gantt view, sequence diagrams, and detail panels. Framework-agnostic — works with any OTEL-compatible agent. Fills the gap between raw OTEL spans and human-comprehensible agent debugging UIs. 
- Characterizing Faults in Agentic AI - world agent systems (AutoGen, CrewAI, OpenAI Agents SDK, LangChain, CAMEL, DB-GPT) to build the first grounded taxonomy of agent-specific faults: initialization failures, role deviation, memory/state deficiencies, orchestration failures, and tool integration errors. Provides architecture-level fault classification that harness engineers can use as a systematic debugging checklist.
- More Visibility into Copilot Coding Agent Sessions - step logs, collapsed subagent traces, and clearer session-stage visibility are exactly the kind of DX improvements that make long-running agent failures debuggable in practice. It is a concrete reminder that trace readability is part of the harness, not an afterthought layered on top.
- AgentStepper: Interactive Debugging of Software Development Agents - by-side conversations (agent↔LLM and agent↔tools). Enables step-through execution, breakpoint manipulation, and mid-trajectory inspection. Developer study shows frustration scores drop from 5.4 to 2.4 (NASA TLX) and comprehension accuracy improves significantly — the first concrete evidence that interactive debugging primitives materially reduce the cognitive load of understanding multi-turn agent behavior.
- Claude Code `/doctor` - checkup skill (alias `/checkup`) that audits harness hygiene: it deduplicates local and checked-in `CLAUDE.md` files, flags unused skills/MCP servers/plugins, identifies slow hooks, and proposes fixes only after confirmation. The clearest first-party example of treating harness configuration drift — not model output — as the debugging target.
- mindwalk - based visualization makes exploration drift and context pressure immediately visible — a concrete debugging primitive for reviewing whether an agent's footprint matched the intended task scope. 
- mcpsnoop - RPC frame between your real client and MCP servers, live in your terminal. Worth including because MCP Inspector tests servers from its own client, so it can't see the calls your actual agent makes, misses, or hangs on — mcpsnoop sits in the real data path and makes those failures visible. 
- Better Harness - agent workflows across five dimensions and turns project and session evidence into prioritized, evidence-bounded findings with scoped repair actions. Runs inside Claude Code, Codex, Cursor, GitHub Copilot, and other hosts, making it a practical cross-tool diagnostic layer for improving the harness rather than just reviewing the final diff. 
- Debugging Deep Agents with LangSmith - fetch (CLI for piping trace data to coding agents). Key insight: debugging deep agents requires AI-assisted trace analysis — the volume of data these systems produce exceeds human capacity.
-
Human-in-the-Loop
- aws-samples/sample-human-in-the-loop-patterns - tool fine-grained), Step Functions (async third-party approval via SNS), and MCP Elicitation (protocol-native real-time interactive approval). The most concrete production guide for choosing the right HITL architecture based on approval latency, trust boundaries, and integration constraints. 
- Dify Human-in-the-Loop Node - and-edit UI mid-flow, and route subsequent execution based on human action (approve/reject/escalate). Demonstrates how HITL transitions from bolt-on approval gates to first-class execution-graph nodes with stateful pause/resume backed by Celery workers and Redis Pub/Sub.
- HITL Protocol - to-end flows including escalation and hybrid approval.
- LangGraph — Human-in-the-Loop Concepts - loop, persist state, and resume after human review. Directly addresses the harness engineering challenge of inserting human gates into long-running workflows.
- AutoGen — Human-in-the-Loop - agent conversation harness.
- Claude Agent SDK — Handle Approvals and User Input - with-changes/suggest-alternative response shapes; `AskUserQuestion` surfaces structured clarifications mid-task; streaming input enables mid-execution redirects. The "approve with changes" pattern — modifying tool input before execution — is the reference design for safe-by-default harnesses that don't simply block or permit.
- HiL-Bench: Do Agents Know When to Ask for Help? - specified tasks into judgment challenges by injecting 3–5 realistic blockers (missing critical information) and giving agents an `ask_human()` tool. Agents from top models achieve ~90% pass@3 with full information but performance drops significantly when blockers are present — the first systematic measure of when agents should escalate to humans rather than proceeding with insufficient context.
- Human Judgment in the Agent Improvement Loop - ups skip: human input is not just an approval gate at execution time, it's also supervision for improving prompts, tools, memory, and evaluators over time. Useful because it treats expert review as a structured data source for harness evolution rather than a one-off manual checkpoint.
- Humans and Agents in Software Engineering Loops - involvement postures — humans outside, in, or on the agent loop — and argues that "humans on the loop" (maintaining the harness rather than reviewing individual outputs) is the only approach that scales with agent throughput. The "agentic flywheel" section — where agents are directed to evaluate results and recommend harness improvements — is the clearest articulation of how HITL evolves from a gate into a feedback mechanism.
- Measuring AI Agent Autonomy in Practice - world Claude Code interactions. Key finding: experienced users shift from per-action approval (20% auto-approve when new) to intervention-only oversight (40% auto-approve at 750+ sessions), and agent-initiated clarification stops grow faster than human interruptions as task complexity increases. The most data-grounded reference for designing adaptive permission models that scale with user trust.
- AutoResearchClaw HITL Co-Pilot - source human-in-the-loop system with six intervention modes (full-auto, gate-only, checkpoint, step-by-step, co-pilot, custom), SmartPause confidence-driven dynamic suspension, and Intervention Learning from human corrections. The cost-guardrail system — aborting runs that exceed budget thresholds — makes it a practical reference for production HITL where human time is as constrained as agent compute.
- Claude Agent SDK — Handle Approvals and User Input - with-changes/suggest-alternative response shapes; `AskUserQuestion` surfaces structured clarifications mid-task; streaming input enables mid-execution redirects. The "approve with changes" pattern — modifying tool input before execution — is the reference design for safe-by-default harnesses that don't simply block or permit.
- Human Judgment in the Agent Improvement Loop - ups skip: human input is not just an approval gate at execution time, it's also supervision for improving prompts, tools, memory, and evaluators over time. Useful because it treats expert review as a structured data source for harness evolution rather than a one-off manual checkpoint.
- agent-chief - first attention orchestration layer that sits between you and every agent, alert, and feed: a three-stage worthiness engine decides whether to interrupt the human, dispatch work to an agent, or curate to memory. Worth including because it treats human attention as a scarce harness resource and turns the flood of agent-generated notifications into a structured, reviewable HITL decision rather than a free side effect. 
-
Memory & State
- Letta (MemGPT) - tier memory (core / archival / recall) maps directly to harness state management design. Their [agent loop redesign post](https://www.letta.com/blog/letta-v1-agent) is the most thorough public analysis of how memory structure shapes the harness. 
- mem0 - in universal memory layer (YC-backed, AWS Agent SDK's exclusive memory provider) that handles cross-session retention without custom harness-level state management code. Lowest integration cost for production-grade persistent memory. 
- Stash - hosted persistent memory layer with an 8-stage consolidation pipeline (episodes → facts → relationships → patterns) and built-in MCP server. The critical gap it fills: production-grade cross-session memory without cloud dependencies or complex infrastructure — a single Docker Compose gives you Postgres, pgvector, and background consolidation. 
- TencentDB-Agent-Memory - tier progressive pipeline (Conversation → Atom → Scenario → Persona) and symbolic short-term memory via Mermaid canvases. The benchmark data is striking: 61% token reduction and 51% relative pass-rate improvement on long-horizon tasks, demonstrating that hierarchical memory architecture outperforms flat vector stores for coding agents. 
- Zep - built agent memory store with automatic conversation summarization, entity extraction, and semantic search over session history. Solves long-session context overflow at the memory layer rather than forcing the harness to manage trimming manually. 
- engram - agnostic, zero-dependency design makes cross-session memory a local harness primitive rather than a managed cloud service. 
- MemPalace - first AI memory system that stores conversation history verbatim and retrieves it with semantic search through a structured palace architecture (wings, rooms, drawers). Achieves 96.6% R@5 on LongMemEval with zero LLM calls, making it the best-benchmarked open-source memory layer for agents that need cross-session persistence without cloud dependencies. 
- agentmemory - built for coding agents with 95.2% retrieval accuracy and 92% token reduction, backed by real-world benchmarks. Its cross-agent architecture — one memory server serving Claude Code, Cursor, Codex, and OpenCode through MCP and hooks — makes cross-session memory a portable harness primitive rather than a vendor-specific add-on. 
- claude-memory-compiler - evolving knowledge base: hooks capture every interaction, the Agent SDK extracts decisions and lessons, and an LLM compiler distills them into structured, cross-referenced articles that improve retrieval quality over time. The most concrete open-source implementation of trace-driven memory evolution for coding agents. 
- How We Built Agent Builder's Memory System - based three-tier memory system (procedural/semantic/episodic) backed by PostgreSQL but exposed to agents as a virtual filesystem. Key harness decisions: human-in-the-loop approval gates every memory write (blocking prompt-injection via malformed writes), validation errors are fed back to the LLM for self-correction, and AGENTS.md serves as the agent's procedural memory anchor.
- Building an Agentic Memory System for GitHub Copilot - up is one of the clearest public discussions of deployed cross-agent memory: repository-scoped memories are shared across coding agent, CLI, and code review, but only after just-in-time verification against the current code state. The core harness lesson is that memory quality is mostly a freshness and invalidation problem — stale, branch-specific memories are often more dangerous than having no memory at all.
- MemArchitect: A Policy-Driven Memory Governance Layer - level eviction policy — not the model — can remove.
- Codified Context: Infrastructure for AI Agents in a Complex Codebase - validated architecture (283 sessions, 108k-line codebase) built on three components: a "hot-memory constitution" encoding conventions and multi-agent coordination protocols, 19 domain-specialist agents, and a "cold-memory knowledge base" of 34 on-demand specification documents. The empirical data distinguishes what must live in always-on context from what should be retrieved on demand — the most concrete published guidance for scaling cross-session memory in a large codebase.
- Facts as First Class Objects: Knowledge Objects for Persistent LLM Memory - context memory at scale: capacity overflow at ~8,000 facts, 60% fact destruction during compaction, and 54% behavioral drift from constraint erosion across cascaded summarizations. Proposes Knowledge Objects (hash-addressed discrete fact tuples) achieving 100% accuracy at 252× lower cost than in-context storage — the quantitative case for moving persistent facts out of the context window into a structured retrieval layer rather than managing them through prompt engineering.
- Recoverability Has a Law: The ERR Measure for Tool-Augmented Agents
- MAGMA: Multi-Graph Agentic Memory Architecture - guided retrieval over relational views. Outperforms MemGPT on long-horizon reasoning benchmarks by 18.5% accuracy improvement. The multi-graph abstraction lets harness engineers compose different retrieval strategies for different task phases — a concrete architecture for memory that scales beyond single-view approaches.
- GAAMA: Graph Augmented Associative Memory for Agents - only approaches for long-horizon reasoning. Practical alternative to full multi-graph systems when adding structure to existing vector-based memory is sufficient.
- Graph-Native Cognitive Memory for AI Agents: Formal Belief Revision Semantics for Versioned Memory Architectures - turn reasoning. Addresses the hard problem of inconsistency resolution in long-lived agent memory: when new information contradicts prior beliefs, how should the agent update its knowledge base?
- Continual learning for AI agents - driven harness and context updates.
- cognee - source memory platform with a hybrid graph-vector-relational poly-store that lets agents recall facts through both semantic similarity and structured graph traversal — the practical middle ground between flat vector stores and full multi-graph research systems. The self-improving pipeline reweights edges from agent feedback, and the 14-tool MCP server makes it a drop-in memory primitive for production harnesses. 
- Hindsight - session persistence from passive storage into an active learning layer inside the harness. 
- ClawVM: Harness-Managed Virtual Memory for Stateful Tool-Using LLM Agents - memory semantics to agent context management, treating the context window as working memory with typed pages, minimum-fidelity invariants, and validated writeback at lifecycle boundaries. A concrete reference for making residency and durability auditable harness-level concerns rather than best-effort side effects.
- MAGE: Memory as Agent-Guided Exploration - horizon memory as execution-state management rather than semantic retrieval: a hierarchical state tree preserves trajectories, enables rollback, and constructs working state from the active root-to-leaf path. Improves task success by 7.8–20.4 percentage points while cutting token consumption 55.1%, making it a concrete reference for memory architectures where execution history — not just semantic similarity — determines recall.
- mex - learned project knowledge into a repo-local, symbol-grounded wiki with drift detection and task-aware routing. The critical gap it fills: most agent memory stores facts, but mex keeps those facts connected to the exact code symbols they describe and flags when code changes invalidate them — making memory freshness a harness-level concern rather than a manual cleanup task. 
- deja-vu - agent sessions already written to disk and serves them back over MCP with no LLM calls, embeddings, or API keys. The zero-dependency binary solves the memory cold-start problem for cross-session persistence and demonstrates that agent memory can be built on existing filesystem traces rather than a separate learning pipeline. 
- How We Built Agent Builder's Memory System - based three-tier memory system (procedural/semantic/episodic) backed by PostgreSQL but exposed to agents as a virtual filesystem. Key harness decisions: human-in-the-loop approval gates every memory write (blocking prompt-injection via malformed writes), validation errors are fed back to the LLM for self-correction, and AGENTS.md serves as the agent's procedural memory anchor.
- Building Self-Correcting Memory in OpenWiki - verified rather than silently trusted. The most concrete published mechanism for memory invalidation: staleness becomes a first-class, inspectable harness state instead of a side effect of periodic regeneration.
- Continual learning for AI agents - driven harness and context updates.
- trajectory - ready record format. Fills a real infrastructure gap — every runtime logs the same concepts (messages, reasoning, tool calls, results) in incompatible formats — and is explicitly designed for *agent* consumption (memory formation, search, training), turning raw session logs into a portable cross-harness substrate rather than yet another observability dashboard. 
-
Observability & Tracing
- OpenLLMetry - based instrumentation for LLM calls and agent steps: adds trace spans to every inference and tool call without modifying business logic. The cleanest way to bring the existing OTEL ecosystem (Grafana, Datadog, Jaeger) to a harness. 
- Arize Phoenix - hostable trace UI and eval runtime for agent workflows. Lets harness engineers audit and replay every reasoning step and tool call offline, without sending data to a third-party cloud. 
- Langfuse - hostable LLM observability platform: traces every agent step, manages prompt versions, and runs evals in one tool. Preferred over cloud-only alternatives when data residency or cost control is a constraint. 
- Weights & Biases Weave - built for agent workflows: automatic call graph capture, dataset versioning, and LLM-as-judge evals that integrate directly with the wandb experiment tracking ecosystem. 
- OTel GenAI Semantic Conventions - compatible backend.
- Pydantic Logfire - queryable (PostgreSQL-compatible), so coding agents can query production observability data directly via the Logfire MCP server. Full-stack OTEL tracing covers both the AI layer and backend — letting you determine whether a failure is in agent logic or infrastructure. The natural observability choice for PydanticAI-based harnesses. 
- Helicone - source LLM observability proxy (YC W23) with the largest open-source pricing database (300+ models). One-line proxy integration provides cost tracking, token monitoring, session tracing, and prompt versioning across providers. The AI Gateway component handles request routing and caching with zero-code changes. SOC 2 and GDPR compliant, self-hostable via Docker — the natural complement to execution-tracing tools when cost attribution is a primary concern. 
- OpenObserve: Unified Observability for LLM Agents - standard platform for LLM tracing with infrastructure log/metric unification. Enables harness engineers to correlate agent decisions with system-level events (network delays, GPU memory pressure) that explain agent failures, going beyond isolated LLM call traces. 
- Building Observable AI Agents: Temporal Now Integrates with Braintrust - executing prior searches, and prompt updates propagate via `braintrust.load_prompt()` without code deployment. The most practical published integration of workflow durability and LLM observability for production agent debugging.
- Introducing BigQuery Agent Analytics
- Distributed Tracing for Agentic Workflows with OpenTelemetry - systems problem, which is exactly how these harnesses fail in production.
- Red-Teaming Anthropic's Internal Agent Monitoring Systems — METR - week adversarial audit of Anthropic's internal agent monitoring and security systems (described in the Opus 4.6 Sabotage Risk Report). Discovered several novel vulnerabilities, some since patched. The most concrete published account of what it takes to stress-test agent monitoring infrastructure — essential reading before trusting any monitoring system as a safety layer.
- Future AGI - source, self-hostable platform unifying tracing, evals, simulations, guardrails, and gateway into a single feedback loop. Worth including because it demonstrates what a unified observability-and-improvement plane looks like rather than stitching together five separate vendor tools. 
- Opik - source AI observability and evaluation platform: deep tracing of LLM calls, conversation logging, and agent activity, plus built-in eval metrics, prompt versioning, guardrails, and the Opik Agent Optimizer. Worth including because it unifies observability, verification, and optimization in one self-hostable stack rather than stitching together separate tools. 
- Introducing BigQuery Agent Analytics
- agentacct - first Agent Work Intelligence for coding agents: ingests existing Claude Code, Codex, and OpenCode session logs, attributes tokens and estimated cost to recorded work steps with confidence labels, and surfaces the evidence on a private dashboard with no cloud sync or API keys. 
-
Permissions & Authorization
- OWASP LLM06:2025 — Excessive Agency - provisioned functions, unnecessary permissions, and missing approval mechanisms. The standard checklist for auditing harness permission scope against principle of least privilege.
- GitHub Enterprise — Governing Agents - protected configurations, agent environment standardization via `copilot-setup-steps.yml`, ephemeral runner enforcement, and cloud-agent firewall allowlisting. The most concrete published reference for governing agent fleets at scale without creating bottlenecks.
- Claude Code Auto Mode: A Safer Way to Skip Permissions - stage classifier: fast single-token gate first, chain-of-thought reasoning only on flagged actions. The design decisions — stripping assistant messages to prevent the agent from rationalizing dangerous actions, deny-and-continue recovery instead of halt — are the reference design for safe-by-default headless agent permissions.
- Claude Agent SDK — Configure Permissions - layer evaluation order (hooks → deny rules → permission mode → allow rules → canUseTool), `allowedTools`/`disallowedTools` declarative scoping, and four permission modes including `dontAsk` (deny-by-default for headless agents). The subagent inheritance warning for `bypassPermissions` alone is worth reading before any multi-agent deployment.
- Two Different Types of Agent Authorization - behalf-of authorization (agent uses end-user credentials, requires cross-channel identity mapping and per-user memory isolation) from fixed-credential authorization (agent owns its own account, requires human-in-the-loop guardrails on high-risk actions). The two models have fundamentally different threat surfaces and determine where authorization enforcement lives in the harness.
- Authorization and Governance for AI Agents: Runtime Authorization Beyond Identity at Scale - protected endpoint. Every agent calls this fabric before tool execution, receiving a deterministic decision: ALLOW / DENY / REQUIRE_APPROVAL / MASK. Addresses the gap that identity alone (who is this agent?) doesn't answer whether a specific action should be executed now, by this agent, for this user, under the current business and regulatory context.
- IETF draft-klrc-aiagent-auth: AI Agent Authentication and Authorization - track specification for AI agent authentication (March 2026, authors from AWS, OpenAI, Zscaler, Ping Identity, Defakto Security). Builds on WIMSE (Workload Identity in Multi-System Environments) and OAuth 2.0 rather than inventing new protocols — agents get SPIFFE-style identifiers, with delegation via OAuth Token Exchange and DPoP for token binding. Essential reference for any harness that needs to authenticate agents across trust domains.
- AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security - dimensional risk taxonomy (source/failure-mode/consequence) with fine-grained agentic safety benchmark (ATBench) and diagnostic guardrail models (4B–8B parameters) achieving 91.8% accuracy. Shifts safety monitoring from binary safe/unsafe checks to root-cause diagnosis: why did an action violate constraints? Where did the violation originate? What are the downstream consequences? Essential for production harnesses where transparency into safety decisions is required for audit trails.
- Open Agent Passport (OAP): Deterministic Pre-Action Authorization for Autonomous AI Agents - action authorization from sandboxed execution and model-based screening as complementary but distinct harness layers.
- nah - name allow/deny lists. The key insight for harness design: the same binary can be benign or destructive depending on its arguments, so intent-level enforcement is the only reproducible safety layer. 
- Agent Vault - source credential broker that sits between AI agents and the APIs they call, injecting real credentials onto outbound requests so agents never possess secrets directly. Eliminates a concrete prompt-injection attack surface — exfiltration of API keys and PATs — by treating credential possession as a harness-layer boundary rather than an agent-side configuration. 
- Aegis - execution firewall that intercepts, classifies, and blocks agent tool calls before they execute, with a compliance cockpit for real-time monitoring, human-in-the-loop approvals, and a tamper-evident audit trail. The zero-code-change integration makes runtime policy enforcement practical for existing agent deployments. 
- Claude Agent SDK — Configure Permissions - layer evaluation order (hooks → deny rules → permission mode → allow rules → canUseTool), `allowedTools`/`disallowedTools` declarative scoping, and four permission modes including `dontAsk` (deny-by-default for headless agents). The subagent inheritance warning for `bypassPermissions` alone is worth reading before any multi-agent deployment.
- Two Different Types of Agent Authorization - behalf-of authorization (agent uses end-user credentials, requires cross-channel identity mapping and per-user memory isolation) from fixed-credential authorization (agent owns its own account, requires human-in-the-loop guardrails on high-risk actions). The two models have fundamentally different threat surfaces and determine where authorization enforcement lives in the harness.
- Nango: Pre-Built Authentication for AI Agents - source platform providing pre-built OAuth and API key authentication for 700+ APIs across 30 categories. Automatically refreshes access tokens, provides webhooks when credentials break, and stores tokens securely so agent code never touches secrets. Solves the "agent needs to call an authenticated API" problem at scale — the authentication layer that complements Composio's tool wrapping. 
- When "Do Not" Is Not Deny: Security Rules in CLAUDE.md vs Built-In Controls - language security rules are backed by a matching built-in control, exposing the gap between documented intent and enforced permissions. A concrete reminder that harness instructions are not guardrails unless they map to deterministic enforcement.
- Do User-Authored Permission Policies Improve Protection Against AI Agent Overreach? - technical participants) comparing per-action HITL approval, automated model review, and user-authored allow/ask/never rules. The key harness finding: pre-authored policies blocked ~20 percentage points *less* overreach than per-action approval — users overwhelmingly set "ask" as their rule and then approve anyway, so static rule pre-commitment degenerates back into runtime prompts without gaining protection. Essential evidence that permission UX design (not just policy expressiveness) determines whether stated intent survives contact with real agent actions.
-
Planning & Task Decomposition
- microsoft/TaskWeaver - first task decomposition framework with a planner/executor split and a plugin system for injecting domain knowledge into the planning layer. The most complete reference implementation of plan-then-execute with stateful task tracking. 
- LATS: Language Agent Tree Search - search signals, trajectory backtracking on failure, and depth-bounded exploration make this the most actionable planning research for harnesses with real environment interaction.
- Agyn: A Multi-Agent System for Team-Based Autonomous Software Engineering - specific agents with different model sizes and tool access produce better outcomes than single-agent approaches, with concrete metrics on task decomposition effectiveness.
- Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks - level planning from low-level execution through synthetic data generation and explicit structured planning. Achieves 57.58% success on WebArena-Lite and 81.36% on WebVoyager. The key harness insight is that planner and executor can be specialized independently — different model sizes, tool access, and reasoning budgets for each layer — improving overall reliability on tasks exceeding context window limits.
- Choosing the Right Multi-Agent Architecture - agent patterns (subagents, skills, handoffs, router) with concrete performance data: subagents process 67% fewer tokens than skills in multi-domain scenarios because context isolation prevents cross-domain bloat. The five-dimension matching table (distributed development, parallelization, multi-hop, user interaction, latency) is the most actionable published guide for deciding when a topology change — not a model change — is the right lever for a performance problem.
- Multi-Agent Workflows Often Fail. Here's How to Engineer Ones That Don't. - agent systems behave like distributed systems, so every handoff needs typed schemas, constrained action schemas, and explicit boundary validation. Worth including because it turns "add more agents" from a vibe into an interface design problem you can actually reason about.
- Effective Harnesses for Long-Running Agents - session state — is the reference design for any harness where a task exceeds a single context window and naïve restarts lose accumulated progress.
- Task-Adaptive Multi-Agent Orchestration (AdaptOrch) - level lever that can improve performance 12–23% over model selection alone.
- Task-Decoupled Planning for Long-Horizon Agents (TDP) - task node independently, and a Self-Revision module updates the graph after execution. The key harness insight is that decoupling planning from execution at the sub-task level enables localized replanning without cascading failures across the entire task chain.
- Choosing the Right Multi-Agent Architecture - agent patterns (subagents, skills, handoffs, router) with concrete performance data: subagents process 67% fewer tokens than skills in multi-domain scenarios because context isolation prevents cross-domain bloat. The five-dimension matching table (distributed development, parallelization, multi-hop, user interaction, latency) is the most actionable published guide for deciding when a topology change — not a model change — is the right lever for a performance problem.
-
Skills & MCP
- modelcontextprotocol/servers - square&label=★&color=yellow)
- microsoft/playwright-mcp - mcp?style=flat-square&label=★&color=yellow)
- A2A Protocol - to-Agent protocol: JSON-RPC over HTTP(S)/SSE with Agent Card service discovery and a task/message/artifact communication model. The emerging standard for cross-framework agent interoperability in multi-agent harnesses. 
- MCP Inspector - square&label=★&color=yellow)
- Shell + Skills + Compaction: Tips for Long-Running Agents - side compaction via explicit `/responses/compact` endpoint. The most concrete first-party documentation of skills-based routing and compaction published in 2026.
- Composio - ready actions with managed OAuth, so tool integration becomes a one-line import rather than a custom harness component per service. The fastest path from "the agent needs to call an external API" to a production-grade, authenticated tool. 
- MCP Streamable HTTP Transport - 11-25 spec, enabling MCP servers to run as remote services rather than local processes. Servers handle multiple client connections using HTTP POST (for client→server messages) and optional GET (for server→client SSE streams). The key harness architecture decision: Streamable HTTP unlocks remote MCP deployment but introduces session management complexity — stateful `Mcp-Session-Id` headers fight with load balancers and horizontal scaling, which the 2026 roadmap aims to resolve by decoupling sessions from the transport layer.
- The 2026 MCP Roadmap - scaling transport without stateful session constraints, `.well-known` discovery for capability advertisement without live connections, Tasks primitive with retry/expiry semantics, and enterprise extensions (audit trails, SSO, gateway behavior). Essential reading before investing heavily in MCP server infrastructure — the transport and discovery changes in particular affect how clients locate and connect to servers.
- Developer's Guide to AI Agent Protocols - agent routing via Agent Card discovery at well-known URLs), UCP (commerce workflows), AP2 (payment authorization with spend limits), A2UI (agent-driven dynamic UIs), AG-UI (streaming event format). The most practical map of which protocol to choose when an agent needs to cross a system boundary.
- AG-UI - driven protocol standardizing how AI agents connect to frontend applications: streaming state updates, tool call rendering, and HITL interrupts over a shared event bus. Fills the layer between MCP (tool access) and A2A (agent-to-agent) — it's the missing protocol for real-time agent-to-UI communication that neither MCP nor A2A was designed to address. 
- Code Execution with MCP: Building More Efficient Agents - call token overhead by having agents write code to interact with MCP servers rather than calling tools directly: up to 98.7% token reduction in experiments. Broadly applicable to any harness where tool schema overhead and intermediate results are consuming context — the pattern is to wrap multi-step tool interaction in a code execution primitive rather than exposing each operation as a discrete tool call.
- Microsoft Skills Framework - level abstraction that makes skills first-class deployment artifacts rather than ad-hoc tool definitions. 
- SkillNet & SkillsBench: Infrastructure for AI Agent Skills at Scale - task benchmark across 11 domains. Demonstrates the harness problem of skill fragmentation and provides infrastructure for standardized skill evaluation across frameworks. 
- AWS Bedrock AgentCore with WebRTC Support - to-peer, UDP-based WebRTC bidirectional streaming to Bedrock Agents for real-time voice interactions. Complements existing WebSocket support with lower latency and better resilience for poor network conditions. Essential harness-level transport choice for agents targeting sub-800ms Total Turn-Around Time voice interactions.
- Hermes Agent: Unified Streaming for Real-Time Agent Workflows - by-token streaming delivery system enabling real-time agent responses; sub-second decision loops on streaming events vs. batch-refreshed data. Critical infrastructure for harnesses where latency (not just throughput) is the constraint — agents must react to events as they arrive, not wait for batch completions.
- Google Developers: Closing the Knowledge Gap with Agent Skills
- What's New with GitHub Copilot Coding Agent - review, built-in security scanning, and CLI handoff concrete as harness primitives rather than abstract ideas. Useful as a current reference for how repository-scoped agent definitions and security checks are being productized in a real coding-agent control plane.
- Announcing Official MCP Support for Google Services - hosted MCP servers: discovery, IAM, audit logging, and Model Armor are provided as platform primitives instead of being rebuilt per server. Worth including because it shows what "enterprise MCP" looks like when the transport, auth, and governance layers are treated as product surface rather than glue code.
- Dataverse Skills: Your Coding Agent Now Speaks Dataverse - specific skills done properly: the agent learns when to use MCP, when to drop to a Python SDK, and when to call a raw API, while the user stays in natural language. Worth adding because it shows that "skills" are not just prompt snippets, but curated execution strategies that hide a multi-tool integration stack behind intent.
- Agent Toolkit for AWS - supported MCP servers, skills, and plugins that let AI agents provision, query, and manage AWS resources through a standardized protocol interface. Worth including as the reference for how a major cloud provider productizes infrastructure access into agent-ready harness primitives rather than leaving teams to hand-roll IAM-scoped tool definitions. 
- agentic-stack - tool harness layer. Adapters translate the same configuration into Claude Code's `CLAUDE.md`, Cursor's rules, OpenCode's `AGENTS.md`, and more — the first practical answer to harness vendor lock-in at the configuration level. 
- mcp-agent - grade framework for building agents with MCP: composable workflows, built-in observability, and provider-agnostic model routing. The clearest reference for turning MCP servers from isolated utilities into a coherent agent harness. 
- Chrome DevTools MCP - class tool interface rather than relying solely on screenshot-driven automation. 
- agent-device - native control layer for iOS and Android devices: snapshots, semantic targeting, typed client access, diagnostics, and replayable workflows. Fills a critical gap in the mobile-agent harness stack — most tool design assumes desktop or browser surfaces, but real-world agents increasingly need to interact with native mobile apps. 
- Announcing the Agentic Resource Discovery specification - owned catalogs and searchable registries. Adds the missing discovery layer that lets agents find MCP servers, A2A agents, and OpenAPI tools at runtime rather than relying on hardcoded integrations, with trust manifests and namespaced URNs for governance.
- SkillOpt - language skills for frozen LLM agents through trajectory-driven edits and validation-gated updates, producing deployable `best_skill.md` artifacts. The key harness insight is that skills should be treated as optimizable parameters that improve with execution feedback, not static prompt fragments written once and forgotten. 
- superpowers - development methodology with automatically-triggered, mandatory skills that work across Claude Code, Cursor, Codex, Gemini CLI, and Copilot CLI. Demonstrates how to package cross-harness workflows — TDD, subagent-driven development, review gates — as reusable skills with an eval harness. 
- Antigravity Awesome Skills - driven skill catalog with an npm installer and role-based bundles — a concrete reference for treating skills as versioned harness artifacts rather than ad-hoc prompts. 
- agentgateway - source agentic proxy that unifies LLM gateway, MCP gateway, and A2A gateway into a single control plane for managing multi-agent, multi-tool connectivity at scale. Provides drop-in security, observability, and governance for agent-to-LLM, agent-to-tool, and agent-to-agent communication — the missing infrastructure layer between agents and the services they touch. 
- AIP: A Graph Representation for Learning and Governing Agent Skills - form skill prose with directed execution graphs: discrete steps as nodes backed by deterministic scripts or natural-language descriptions, connected by explicit typed input/output edges and governed by a schema-validated YAML spec. Compiling skills to AIP improved Claude Sonnet's pass rate from 53% to 67% while making skills queryable, auditable, and repairable at the script level — the key harness advance is turning skill improvement from a prose rewrite into a measurable tuning loop.
- The 2026-07-28 MCP Specification Release Candidate - Session-Id`, the new `ext-*` extension framework formalizes Tasks and MCP Apps, and a twelve-month deprecation policy gives harness builders a stable target. Essential reading before designing remote MCP server infrastructure that must survive load balancers and horizontal scaling.
- You can't whisper at an AI agent
- Ponytail - in solutions, avoid new dependencies, and write the minimum code that works. Benchmarked on real Claude Code sessions with ~54% fewer lines, ~20% lower cost, and preserved safety guards — a rare harness-level incentive that fights over-engineering rather than just adding capability. 
- oomol-lab/open-connector - source auth gateway connecting 1000+ SaaS providers to AI agents through SDK, CLI, MCP, HTTP, and OpenAPI. Treats external-tool onboarding as a unified, self-hosted harness layer: one credential and access model covers direct SDK calls, MCP servers, and OpenAPI discovery, so teams don't rebuild auth per integration. 
- addyosmani/agent-skills - grade engineering skills for AI coding agents, packaged as 24 reusable skills covering the full development lifecycle from `/spec` to `/ship`. The slash-command interface and context-aware auto-activation make it a concrete reference for turning senior-engineering judgment into agent-executable harness artifacts. 
- wshobson/agents - harness plugin marketplace that maintains one source-of-truth `plugins/` directory and generates harness-native artifacts for Claude Code, Codex CLI, Cursor, OpenCode, Gemini CLI, and GitHub Copilot. It is the clearest practical example of treating reusable agent capabilities as a portable distribution format rather than ad-hoc prompt files. 
- mgechev/skillgrade - trial evals against Claude, Codex, Gemini, or OpenCode, and reports pass rates with a CI-ready threshold. Fills the gap between shipping a skill and knowing an agent actually discovers and invokes it correctly. 
- Qwen-MM-Plugins - first coding harness multimodal-native without rebuilding the agent loop. 
- Comet - running task workflow and Skill platform for coding that turns skill creation, evaluation, and release into a single lifecycle with Rubric, Pass@k, and Pass^k scoring. The Native/Classic dual-workflow model is a concrete reference for matching harness constraint strength to model capability rather than using one loop for every task. 
- Dataverse Skills: Your Coding Agent Now Speaks Dataverse - specific skills done properly: the agent learns when to use MCP, when to drop to a Python SDK, and when to call a raw API, while the user stays in natural language. Worth adding because it shows that "skills" are not just prompt snippets, but curated execution strategies that hide a multi-tool integration stack behind intent.
- vurb.ts - aware by default. 
-
Task Runners & Orchestration
- Building a C Compiler with a Team of Parallel Claudes
- LiteLLM - compatible interface, with a Router handling retry/fallback across deployments, per-project cost and rate-limit tracking, and OTEL callback integrations. The right infrastructure layer when your harness needs provider resilience (automatic failover on 429/500 errors), budget guardrails, or the ability to swap models without touching orchestration code. 
- LangGraph - based state machine framework for multi-agent harnesses: models supervisor/subagent topologies, error-recovery branches, and checkpoint persistence as first-class primitives. The most widely adopted harness orchestration layer in production. 
- OpenAI Agents SDK - agent framework built around handoffs and guardrails; the production successor to Swarm. Complements LangGraph for harnesses where delegation patterns are simpler than full graph orchestration. 
- Google ADK - first agent framework with built-in multi-agent orchestration, tool registration, session state, and eval pipeline. Its `Runner` and `AgentTool` patterns are the reference implementation for wrapping sub-agents as tools in a larger harness. 
- AutoGen - agent conversation framework with a complete AgentChat layer covering agent loop, tool integration, termination conditions, and human-in-the-loop. The most comprehensive open-source reference for large-scale multi-agent harness design. 
- CrewAI - layer harness orchestration: Crew handles autonomous agent delegation, Flow provides event-driven deterministic control (branching + shared Pydantic state). The clearest open-source example of mixing autonomous and scripted execution in the same harness. 
- PydanticAI - safe agent framework where tool definitions, parameters, and return values are Pydantic models. Shifts "agent output doesn't match expected structure" from a runtime bug to a type-check failure; its `RunContext` dependency injection pattern is the reference design for passing session-scoped objects through the harness without global state. 
- OmniRoute: Multi-Provider LLM Gateway - turn decision. 
-
Programming Languages
Categories
Sub Categories
Adjacent Collections
101
Skills & MCP
41
Demo Harnesses
40
Task Runners & Orchestration
39
Memory & State
29
Context Delivery & Compaction
28
Generators & Meta-Harnesses
23
Agent Loop
18
Debugging & Developer Experience
17
Permissions & Authorization
17
Tutorials & Educational
16
Observability & Tracing
16
Tool Design
14
Human-in-the-Loop
14
Verification & CI Integration
11
Planning & Task Decomposition
10
Keywords
llm
46
ai-agents
38
ai
33
mcp
28
claude-code
28
agent
22
python
21
agents
20
openai
18
codex
17
developer-tools
16
cli
14
anthropic
14
ai-agent
13
typescript
13
agentic-ai
12
claude
11
langchain
11
open-source
9
llmops
9
harness-engineering
8
mcp-server
8
cursor
8
prompt-engineering
8
rag
8
llms
7
harness
7
agent-framework
7
opencode
7
observability
7
copilot
7
devtools
7
multi-agent
7
model-context-protocol
6
openclaw
6
memory
6
ai-coding
6
agent-harness
6
sandbox
6
coding-agents
6
llm-evaluation
6
automation
6
awesome-list
6
rust
5
context-engineering
5
gpt-4
5
generative-ai
5
genai
5
awesome
5
terminal
5