awesome-prompts
Curated list of chatgpt prompts from the top-rated GPTs in the GPTs Store. Prompt Engineering, prompt attack & prompt protect. Advanced Prompt Engineering papers.
https://github.com/ai-boost/awesome-prompts
Last synced: 15 days ago
JSON representation
-
Agent Ecosystem
-
A2A — Agent-to-Agent Protocol
-
Agent Skills
- anthropics/skills - skills-spec.md`)  |
- VoltAgent/awesome-agent-skills
- vercel-labs/agent-skills
- Agent Skills Docs — Anthropic
- Equipping Agents for the Real World — Anthropic
- Skills vs MCP — LlamaIndex
- Agentic AI Foundation (AAIF) - founded by Anthropic, OpenAI, and Block, backed by Google, Microsoft, and AWS.
-
Frameworks
- **LangGraph** - grade workflows (Nov 2025 stable release) |
- **CrewAI** - based multi-agent teams |
- **Magentic-One** - capability agents (web + file + code + terminal) |
- **OpenAI Agents SDK** - native orchestration (Mar 2025) |
- **OpenAI Agents SDK for JS/TS** - agents-js?style=flat-square) |
- **GitHub Agentic Workflows (gh-aw)** - first agentic workflows for GitHub Actions — Markdown workflow specs, sandboxed execution, structured outputs, approval-aware automation (2026)  |
- **Google ADK** - native development (Apr 2025) |
- **Claude Code**
- **karpathy/autoresearch** - line self-improving agent — reads its own training code, forms hypotheses, runs experiments overnight (Mar 2026)  |
- **Microsoft Agent Framework** - driven actor model, multi-agent orchestration (RC 2026)  |
- **openai/codex** - mini powered, runs in terminal (Apr 2025, active 2026)  |
- **DeerFlow 2.0** - horizon "SuperAgent" — filesystem, sandboxed execution, persistent memory, parallel sub-agents, skill system; LangGraph-based; hit #1 GitHub Trending on launch day (Feb 28, 2026)  |
- **smolagents** - first agent framework (~1000 LOC core) — MCP integration, multi-agent hierarchies, multimodal I/O, 100+ model providers  |
- **browser-use** - driven browser automation — agents control a real browser to complete web tasks; 89% on WebVoyager benchmark  |
- **Mastra** - first AI agent framework — Agent/Workflow/RAG/Evals primitives, 40+ model providers, native MCP server support (YC W25, 2026)  |
- **PraisonAI** - ready multi-agent framework — 100+ LLM providers, MCP integration, memory/RAG/guardrails, 24/7 delivery to Telegram/Discord/WhatsApp, fastest agent instantiation (2026)  |
- **Portia AI** - source predictable agent framework — 1000+ cloud/MCP tools, built-in auth, auditability and security focus for enterprise workflows (2026)  |
- **Paperclip** - human-company multi-agent orchestration — org charts, budgets, goal management, CEO→Manager→Worker delegation; 48k stars in 3 weeks (Mar 2026)  |
- **Gemini CLI** - source terminal AI agent — ReAct loop, MCP support, 1M context window, Gemini 2.5 Pro/3 Flash/3.1 Pro; free tier (60 req/min); Apache 2.0; v2.0 Apr 2026  |
- **oh-my-codex** - agent execution, notification routing; 23k+ stars (2026)  |
- **Hermes Agent** - improving agent framework built on Hermes 3 — persistent memory across sessions, learns from interactions, multi-platform messaging; 32k+ stars (2026)  |
- **claw-code** - development demo in Rust — human sets direction via chat, claws self-coordinate (plan/build/test/review/push); notification routing kept outside agent context; fastest repo to 100K stars (Mar 2026)  |
- **PilotDeck** - isolated agent OS — white-box memory, smart model routing (~70% cost savings), always-on background execution, MCP-native; productivity platform for multi-project agent workflows (May 2026)  |
- **AOS CE** - harness loops, MCP bridge; composable user-space layer for harnesses and agent-native software (July 2026)  |
- **agent-browser** - labs/agent-browser?style=flat-square) |
- **nanobot** - lightweight self-hosted personal AI agent framework in Python — WebUI, CLI, chat apps, tools, memory, MCP, multi-agent workflows, automation, OpenAI-compatible API (Feb 2026)  |
- **Flue** - harness framework — sessions, tools, skills, sandboxes, durability, and subagents; compose the full harness an agent needs to do real work, run locally via CLI or deploy to a hosted runtime (Feb 2026)  |
- **Agno** - first agent framework — memory, knowledge, tools, multi-agent teams, and structured workflows; rebrand of phidata (2026)  |
- **Qwen-MM-Plugins** - native — vision, audio, and video plugins that wire into existing agent frameworks via MCP/tool interfaces (July 2026)  |
- **codebase-memory-mcp** - performance code-intelligence MCP server — tree-sitter + Hybrid LSP knowledge graph, 15 MCP tools, indexes Linux kernel in 3 min, 120× fewer tokens than file-by-file exploration (Feb 2026)  |
- **TencentDB Agent Memory** - level memory hub for AI agents — turns conversations, docs, and code into four reusable memory assets (Chat Memory, Skill, LLM-Wiki, Code-Graph) that are governed, shared, and equipped across agents and frameworks (Apr 2026)  |
- **agentmemory** - scored facts/procedures/sessions, hybrid dense+keyword+graph retrieval, 54 MCP tools, 12 auto hooks, 95.2% R@5, 92% fewer tokens; supports Claude Code, Cursor, Codex, Gemini CLI, Hermes, OpenClaw, pi, OpenCode, and any MCP client (Feb 2026)  |
- **eve** - first framework for durable backend AI agents — instructions, tools, skills, channels, schedules, connections, and subagents as files; path-named capabilities, typed tools, `eve eval` harness (June 2026)  |
- **kimi-code** - source terminal AI coding agent — single-binary TUI, Kimi K3 + OpenAI-compatible providers, `/goal` judge mode, coder/explore/plan subagents, AI-native MCP config, Skills, lifecycle hooks, video input; MIT (May 2026)  |
- **Orca** - backed (Mar 2026)  |
- **herdr** - native runtime for coding agents — background server with persistent sessions, agent-aware pane states (working/blocked/idle), detach/reattach across terminals and SSH; Rust, Apache-2.0, 34k+ stars (Mar 2026)  |
- **OpenSRE** - source AI SRE agent framework — investigate production incidents across 60+ tool integrations, synthetic RCA simulations, real-world e2e tests across Kubernetes/EC2/CloudWatch/Lambda, reversible PII masking, headless CLI and REPL (Jan 2026)  |
- **DeepSeek Harness** - first open-source agent harness — everything (tools, skills, UI, memory, models) is a hot-swappable plugin; Cordis-based composability; ships with Web UI and headless CLI (developer preview, Aug 2026)  |
- **TrueForge** - source agent harness — runtime layer that turns an LLM into a working agent; chat UI, HTTP API + TypeScript SDK, MCP tools, git-backed skills, sandbox-as-tool, approvals, context compaction; local SQLite or hosted Postgres/Redis (MIT, Aug 2026)  |
- **OpenBot** - source AI coworkers that each get a computer of their own — browser, files and tools; every action decided before it happens and recorded after; bring any AG-UI agent (MIT, Aug 2026)  |
- **qm** - agnostic core (Pi, OpenCode, Codex, Claude Code all drive the same loop), admin-gated org security posture, scope-shared skills with pack imports, web apps and background crons (TypeScript, MIT, 14.5k+ stars, Aug 2026)  |
- **Omnigent** - source meta-harness — a common orchestration layer over Claude Code, Codex, Cursor, OpenCode, Hermes, Pi, and custom YAML-defined agents; mix and supervise multiple agents in one session, swap harnesses without rewriting, policies/sandboxing/approvals, cloud sandbox backends (Modal, E2B, K8s, Databricks…), sessions synced across terminal/browser/phone/desktop (Python, Apache-2.0, 9.7k+ stars, June 2026)  |
- **Claude Agent SDK** - agent-sdk-python?style=flat-square) |
- **commerce-agents** - builder Claude Code plugin that scaffolds and reviews your own deployment (Apache-2.0, 2.8k+ stars, Sept 2026)  |
- **OpenHuman** - first personal AI harness built in Rust — a "brain that remembers everything" (persistent memory), plus agent orchestration and deep-research workflows, with the human kept in the loop (GPL-3.0, 40k+ stars, Feb 2026, actively maintained)  |
- **phone-harness** - compatible agents (MIT, 2.7k+ stars, Aug 2026, actively maintained)  |
- **Artemis** - language Android automation that lets AI assistants drive real devices like a human — cross-app workflows, multimodal element targeting (indices + coordinate/visual fallbacks), reactive observe-and-act loop (~3–5s/step) with asynchronous history summaries, proactive exploration with blocked-action recovery; MCP-native diagnostics (Logcat, screenshots) for Claude Code, Codex, and Windsurf; 99%+ task completion on AndroidWorld; Gemini/Claude/GPT-4o/Qwen-VL multimodal (Python, Apache-2.0, 7.5k+ stars, Aug 2026, actively maintained)  |
- **Reef** - Agent Society | Continual-learning infra for self-improving agents — connects agent inference, feedback, learning, and versioned delivery in one serve → observe → grow → commit loop; either train model weights (Slime/SGLang integration) or evolve the harness itself (prompts, rules, skills) with no local training GPUs; versioned artifact history with candidate evaluation and selection policies (Python, Apache-2.0, 2.8k+ stars, Aug 2026, actively maintained)  |
- **Dormice** - hosted, E2B-compatible sandbox platform for AI agents. Inverts cloud sandbox economics: one daemon + one SQLite ledger on a machine you already pay for, and sandboxes are permanent — they cool down an idle ladder (`active → frozen → stopped → archived`) so idle costs nothing (~5 MiB resident frozen, ~50 ms wake, files intact). `acquireSandbox(key)` is the entire mental model (idempotent create/wake/restore). Docker + gVisor-isolated execution, one-binary deploy (no K8s), multi-node fleet mode, signed file URLs, real PTY streaming — and the official `e2b` SDK works unmodified by changing two URLs; ships an Agent Skill so coding agents can drive it directly (TypeScript, Apache-2.0, 1.2k+ stars, July 2026, early development, actively maintained)  |
- **OpenConnector** - source connector gateway for AI agents — an alternative to Pipedream/Composio. Connect user app accounts once, then expose 1,000+ providers / 10,000+ prebuilt Actions through SDK, CLI, MCP, HTTP, and OpenAPI from one inspectable runtime. Credential handling for API keys, OAuth2, and custom credentials; scoped runtime tokens, action allow/block policies, redacted run logs — provider credentials never enter the agent process. Self-host with Docker/Node.js (SQLite or Postgres) or use the hosted runtime (TypeScript, Apache-2.0, 5.8k+ stars, June 2026, actively maintained)  |
-
Harness Engineering
- Aakash Gupta
- Harness Engineering — OpenAI - first world" |
- Improving Deep Agents with Harness Engineering — LangChain
- The Importance of Agent Harness in 2026 — Philipp Schmid
- Skill Issue: Harness Engineering for Coding Agents — HumanLayer - agents as context firewalls, practical patterns |
- Effective Harnesses for Long-Running Agents — Anthropic - running agent design |
- SethGammon/Citadel - tier routing, parallel worktrees, lifecycle hooks, 6 skills |
- langchain-ai/deepagents
- Building a C Compiler with Parallel Claudes — Anthropic - agents to build a C compiler — generator/evaluator harness patterns |
- strukto-ai/mirage - ai/mirage?style=flat-square) | Unified virtual filesystem for AI agents — mounts S3, GDrive, Slack, Gmail, Redis as one tree; agents use bash across every backend; Python/TypeScript SDKs, cache, snapshots (May 2026) |
- QoderAI/better-harness - harness?style=flat-square) | Open-source loop/harness improvement skill — turns project and session evidence into prioritized improvements and verifiable next steps for Claude Code, Codex, Cursor, and other coding agents (July 2026) |
- **harness-engineering** - engineering?style=flat-square) | Ryan Lopopolo's anthology, field guide, and agent context bundle for harness engineering — shaping context and tools so agents can recover intent, operate systems, respect authority, prove outcomes, and leave the next run better equipped (CC-BY-4.0, July 2026) |
- **loop-engineering** - engineering?style=flat-square) | "Stop prompting. Design the loop." — practical patterns, starters & CLI (`loop-audit`, `loop-init`, `loop-cost`) for systems that discover work, hand it to agents, verify results, and persist state across Claude Code, Codex, Grok, and OpenCode; report-only week one, scores loops on a "Loop Ready" rubric (MIT, 11k+ stars, June 2026, actively maintained) |
- **ECC** - m/ECC?style=flat-square) | The agent harness performance optimization system — skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond (MIT, 242k+ stars, Jan 2026) |
- **SoL-Pi** - Pi?style=flat-square) | NVIDIA's efficiency extension for the Pi coding agent — four opt-in harness mechanisms distilled from scaled auto-research loops ([arXiv 2609.20519](https://arxiv.org/abs/2609.20519)): Action Fusion (run follow-up validation in the same tool call), ObservationPack (stable handles with exact paged recall for repeated large outputs), evidence-preserving reducer (long logs → compact receipts where every quotation is verifiable against the archive), and online context compaction gated by economics/window pressure; no patches to Pi, evidence always preserved (TypeScript, MIT, 2.9k+ stars, Sept 2026, actively maintained) |
-
MCP — Model Context Protocol
-
-
Basic Prompt Engineering
-
Context Engineering
-
Prompt Attack & Defense
- MIT Technology Review
- Effective Context Engineering for AI Agents — Anthropic
- Context Engineering Guide — Prompt Engineering Guide
- davidkimai/Context-Engineering - Engineering?style=flat-square) — first-principles handbook on context design, orchestration, and optimization
- Meirtz/Awesome-Context-Engineering
- **NanoNets/Graft** - square) — Open-source codebase context layer for coding agents — builds a persistent, queryable code graph and pulls matching nodes into each prompt; 46% fewer tool calls, 42% token savings, 60% faster on a 162-run benchmark, 66% SWE-bench Verified (vs 54% cold); supports Claude Code, Cursor, Codex, Gemini via MCP/CLI (MIT, July 2026)
- muratcankoylan/Agent-Skills-for-Context-Engineering - Skills-for-Context-Engineering?style=flat-square) — comprehensive, MIT-licensed collection of Agent Skills for context engineering, multi-agent architectures, and production agent systems — context fundamentals/degradation/compression, memory systems, tool design, harness engineering, self-improvement loops; cited in academic research as foundational work on static skill architecture (17.9k+ stars, actively maintained, 2026)
- **redhat-et/ripwire** - et/ripwire?style=flat-square) — "The ripgrep of AI context" (Red Hat): zero-dependency C++23 CLI + MCP server that lets coding agents find what they need without reading the whole repo — tree-sitter signatures at 74.7% fewer bytes than bodies, blast-radius queries, tests-to-run suggestions, and post-change verification with quality deltas; every heuristic guess labelled, every context loss published (Apache-2.0, 2.4k+ stars, actively maintained Sept 2026)
-
-
Frameworks
-
Automatic Prompt Optimization
- **TextGrad** - group/textgrad?style=flat-square) | Treats LLM feedback as "textual gradients" and backpropagates them to optimize prompts. Published in Nature. |
- **GEPA** - ai/gepa?style=flat-square) | Reflective Text Evolution — optimizes prompts, code, and agent configs. Claims +6–20 pts over GRPO on 6 tasks with fewer rollouts. |
- **Hermes Agent Self-Evolution** - agent-self-evolution?style=flat-square) | Evolutionary self-improvement for Hermes Agent — DSPy + GEPA (Genetic-Pareto Prompt Evolution) automatically evolves skills, tool descriptions, system prompts, and code via reflective search over execution traces (understands *why* things fail, not just that they failed); constraint gates (tests, size limits, benchmarks) before a best-variant PR; no GPU training required, ~$2–10 per optimization run (Nous Research, MIT, 5.4k+ stars, Mar 2026) |
-
Eval & Observability
- **DeepEval** - ai/deepeval?style=flat-square) | Unit testing for LLMs — G-Eval, hallucination, RAG faithfulness, agentic task metrics. |
- **Langfuse** - square) | Open-source LLM engineering platform — tracing, evals, prompt management, A/B experiments. |
- **Phoenix** - ai/phoenix?style=flat-square) | Open-source AI observability & evaluation platform (Arize) — OpenTelemetry-native tracing for agents, LLM-as-judge evals, versioned datasets & experiments for prompt regression testing, prompt management with version control and replay, plus an MCP endpoint so Claude Code/Cursor can query traces directly; framework-agnostic (OpenAI Agents SDK, Claude Agent SDK, LangGraph, DSPy, LlamaIndex, Vercel AI SDK); self-hosted, actively maintained (2026) |
- **Tracely-ai** - ai?style=flat-square) | Trace-native CI/CD for AI agents — grades every production trace as it lands (LLM-as-judge evaluators as trace-table columns), clusters failures into issues, freezes failing runs into hermetic replayable regression cases ($0 replay, no API keys), blocks the PR via CI gate, alerts via Slack/email/webhook; OTLP ingest (MIT, 1.4k+ stars, June 2026, actively maintained) |
-
Eval & Testing
- **OpenAI Evals** - square) | Open eval framework and benchmark registry — standardizes LLM performance measurement. |
- **promptfoo** - square) | Test-driven prompt engineering: regression tests, red teaming, model comparison, CI/CD integration. [Acquired by OpenAI (Mar 2026)](https://openai.com/index/openai-to-acquire-promptfoo/) — remains open source. |
- **Terminal-Bench** - terminal agent benchmark (Stanford/Laude) — compile code, train models, set up servers in Docker-sandboxed environments; the de facto benchmark for agentic coding (2026). |
-
Legacy (2023 era — kept for reference)
- dair-ai/Prompt-Engineering-Guide - ai/Prompt-Engineering-Guide?style=flat-square) — the canonical entry point. Covers techniques, adversarial prompting, RAG, agents, papers, and notebooks.
-
Low-Code & Workflow Platforms
- **Dify** - square) | Production-grade RAG and agent workflow platform — visual pipeline builder, multi-model support, plugin architecture. |
- **Langflow** - ai/langflow?style=flat-square) | Drag-and-drop agent and chain builder — good for rapid prototyping of complex pipelines. |
-
Prompt Programming
- **DSPy** - square) | Write LM pipelines declaratively, then *compile* — DSPy auto-optimizes prompts and few-shot demonstrations. The strongest engineering-first approach. |
- **Guidance** - ai/guidance?style=flat-square) | Interleave generation with constraints, regex/CFG, and control flow. Precision output control that goes beyond what prompts alone can achieve. |
-
Red Team & Security
- **garak** - square) | LLM vulnerability scanner by NVIDIA — red teaming, prompt injection, jailbreak, and leakage detection. |
- **OpenAI: Prompt Injection Defense**
- **The Promptware Kill Chain** - stage malware kill chain; 21/36 documented attacks already traverse 4+ stages. Featured at Black Hat 2026. | [PDF](https://arxiv.org/pdf/2601.09625) |
- **Microsoft Agent Governance Toolkit** - governance-toolkit?style=flat-square) | 7 packages (Python/Rust/TS/Go/.NET) — policy enforcement (<0.1ms), zero-trust agent identity (Ed25519 + SPIFFE), sandboxed execution; covers all OWASP Agentic Top 10; adapters for LangChain/CrewAI/ADK/OpenAI Agents SDK (Apr 2026) |
- **agent-drift** - drift?style=flat-square) | Stress-test agents for goal drift and system-prompt violations across 6 value dimensions — multi-turn escalation, LLM-as-judge, interactive HTML reports; inspired by ICLR 2026 workshop paper (Apr 2026) |
- **T3MP3ST** - plinius/T3MP3ST?style=flat-square) | Autonomous red-teaming meta-harness for AI coding agents — recon → exploit → report against authorized targets, multi-agent offensive-security workflows, offline-model support; by elder-plinius (AGPL-3.0, 5.3k+ stars, July 2026) |
- **OpenAI Codex Security** - security?style=flat-square) | Official OpenAI CLI and TypeScript SDK for finding, validating, and fixing security vulnerabilities — standard/deep scans, diff and working-tree targets, SARIF/CSV/JSON export, CI-native exit codes, pre-commit hooks (Apache-2.0, 8k+ stars, July 2026) |
- **SkillSpector** - square) | Security scanner for AI agent skills — detects vulnerabilities, malicious patterns, prompt injection, data exfiltration, and supply-chain risks in Claude Code, Codex, and MCP skills before installation (Apache-2.0, 14.7k+ stars, Mar 2026) |
-
Tool Use & Reliability
- **forge** - square) | Reliability layer for self-hosted LLM tool-calling — guardrails (rescue parsing, retry nudges, response validation), optional workflow constraints (`required_steps`, `prerequisites`, `terminal_tool`), and built-in eval suite. MIT, 2.2k+ stars, Feb 2026 |
- **reverify** - square) | Hallucination gate for agents — the model proposes claims, deterministic tools check each against ground truth and return VERIFIED/REFUTED with evidence; only what survives counts as fact. Ships as MCP server + CLI, with `reverify rollover` for lossless context handoff across resets. Caught every hallucination on a 71-file binary reverse-engineering benchmark (0 wrong claims accepted). MIT, 978 stars, Aug 2026 |
-
-
Official Agent Building & Prompt Engineering Guides
-
Official Guides
-
Harness Engineering
- Prompt Engineering (production-focused)
- GPT-4.1 Prompting Guide
- A Practical Guide to Building Agents
- Claude Code Best Practices
- DeepSeek Prompt Library
- Prompt Engineering Best Practices
- Building Effective AI Agents
- Demystifying Evals for AI Agents
- Quantifying Infrastructure Noise in Agentic Coding Evals
- Harness Design for Long-Running Application Development
- Building Agents with the Claude Agent SDK
- Eval Awareness in Claude Opus 4.6's BrowseComp Performance
- Scaling Managed Agents: Decoupling Brain from Hands
- Claude Code Auto Mode: A Safer Way to Skip Permissions - layer model-based classifier for read vs write approvals |
- Trustworthy agents in practice
- Responsible Scaling Policy
- GPT-5.4 Prompt Guidance
- GPT-5.2 Prompting Guide
- Realtime Prompting Guide - realtime speech-to-speech model |
- From Model to Agent: Equipping the Responses API with a Computer Environment
- Keeping Your Data Safe When an AI Agent Clicks a Link
- Introducing the OpenAI Safety Bug Bounty Program
- Build with Gemini Deep Research
- Agents Companion Whitepaper - page production playbook: multi-agent, AgentOps, agentic RAG, evals |
- Gemini 3 Prompting Guide - step verification, grounding, persona management |
- Developer's Guide to AI Agent Protocols - UI compared |
- Developer's Guide to Building ADK Agents with Skills
- Codex CLI Prompting Guide
- Llama Prompt Engineering Guide
- Llama 4 Prompt Format
- Loop Engineering: Getting Started with Loops - off coding agents |
- Building scalable AI agents with modular prompt transpilation - file drift checks |
- The "think" tool: Enabling Claude to stop and think - use chains and policy-heavy decisions |
- How we contain Claude across products - turn human approval fails (users approve routinely), and how claude.ai, Claude Code, and Claude Cowork each cap blast radius via sandboxes, VMs, and egress controls |
-
-
Papers
-
Agent Evaluation
- Signals: Trajectory Sampling and Triage for Agentic Interactions (April 2026) - based taxonomy for sampling informative agent trajectories post-deployment — 82% informativeness vs 54% random; organizes signals across interaction, execution, and environment dimensions; 6.2k HF likes | [PDF](https://arxiv.org/pdf/2604.00356) |
- Agent Psychometrics: Task-Level Performance Prediction (April 2026) - turn agentic assessment; newer benchmarks like SWE-bench Verified and Terminal-Bench test iterative agent behavior with execution feedback | [PDF](https://arxiv.org/pdf/2604.00594) |
- YC-Bench: Benchmarking AI Agents for Long-Term Planning (April 2026) - year horizon spanning hundreds of turns; tests consistent execution | [PDF](https://arxiv.org/pdf/2604.01212) |
- When Users Change Their Mind: Evaluating Interruptible Agents (April 2026) - task execution — critical requirement for realistic deployment in dynamic environments | [PDF](https://arxiv.org/pdf/2604.00892) |
- SWE-CI: Evaluating Agents on Codebase Maintenance via CI (2026) - loop benchmark for long-term codebase maintainability — 100 tasks spanning 233 days and 71+ consecutive commits; shifts evaluation from static single-fix to dynamic long-horizon reasoning | [PDF](https://arxiv.org/pdf/2603.03823) |
- SWE-Skills-Bench (2026) - world SE tasks measuring whether agent skills actually improve outcomes — 39/49 public skills give zero gain; average improvement only +1.2%; reveals fundamental gap in skill design | [PDF](https://arxiv.org/pdf/2603.15401) |
- LongCLI-Bench: A Benchmark for Long-Horizon Agentic Programming in the CLI (2026) - based coding agents on long-horizon programming tasks that require sustained planning, repo navigation, debugging, and recovery over many steps instead of single-fix patches | [PDF](https://arxiv.org/pdf/2602.14337) |
- ProjDevBench: Benchmarking AI Agents on End-to-End Software Project Development (2026) - fix tasks; targets end-to-end project delivery realism | [PDF](https://arxiv.org/pdf/2602.01655) |
- LiveClawBench: Benchmarking LLM Agents on Complex, Real-World Assistant Tasks (April 2026) - world assistant tasks requiring planning, tool use, and recovery — closer to production deployment scenarios than static QA benchmarks | [PDF](https://arxiv.org/pdf/2604.13072) |
- RiskWebWorld: GUI Agents in E-commerce Risk Management (April 2026) - stakes professional workflows — 100 real-world e-commerce risk scenarios testing sequential decision-making under uncertainty | [PDF](https://arxiv.org/pdf/2604.13531) |
- OccuBench: Real-World Professional Tasks via Language World Models (April 2026)
- EpiBench: Multi-turn Research Workflows for Multimodal Agents (April 2026) - paper synthesis; built on smolagents with persistent memory and tool use | [PDF](https://arxiv.org/pdf/2604.05557) |
- Ask Early, Ask Late, Ask Right: When Does Clarification Timing Matter for Long-Horizon Agents (May 2026) - injection framework measuring how clarification value changes over the execution trajectory across goal/input/constraint/context dimensions; 6,000+ runs, 4 frontier models, 3 benchmarks; finds goal clarifications lose nearly all value after 10% execution, input clarifications retain value through ~50%, and deferring any clarification past mid-trajectory degrades performance below never asking; cross-model Kendall tau 0.78–0.87 confirms task-intrinsic timing curves | [PDF](https://arxiv.org/pdf/2605.07937) |
- Reasoning Is Not Free: Robust Adaptive Cost-Efficient Routing for LLM-as-a-Judge (May 2026) - verification tasks (math, coding) but yield limited or *negative* gains on simpler evaluations while costing significantly more compute; proposes RACER, a distributionally-robust routing policy that dynamically selects between reasoning and non-reasoning judges under a fixed budget via a KL-divergence uncertainty set, with theoretical guarantees including uniqueness of the optimal policy and linear convergence of the primal–dual algorithm | [PDF](https://arxiv.org/pdf/2605.10805) |
-
Agent Reliability
- Towards a Science of AI Agent Reliability (2026)
- Agentic Reasoning for LLMs (2026) - layer framework (single-agent capabilities → self-evolving agents → multi-agent coordination); 202 Hugging Face likes | [PDF](https://arxiv.org/pdf/2601.12538) |
- Why Do Web Agents Fail? A Hierarchical Planning Perspective (2026) - level planning, low-level grounding, and replanning — PDDL-structured plans outperform NL plans but grounding remains the dominant bottleneck; a single round of exploratory replanning substantially improves task success | [PDF](https://arxiv.org/pdf/2603.14248) |
- Claw-Eval: Trustworthy Evaluation of Autonomous Agents (April 2026) - to-end evaluation suite with 300 human-verified tasks across 9 categories — trajectory-aware grading over 2,159 rubric items; finds vanilla LLM judges miss 44% of safety violations and 13% of robustness failures | [PDF](https://arxiv.org/pdf/2604.06132) |
- TimeSeek: Temporal Reliability of Agentic Forecasters (April 2026) - uncertainty markets; search improves pooled accuracy but degrades 12% of conditions | [PDF](https://arxiv.org/pdf/2604.04220) |
- ReliabilityBench: Evaluating LLM Agent Reliability Under Production-Like Stress (2026)
- Shepherd: A Runtime Substrate Empowering Meta-Agents with a Formalized Execution Trace (May 2026) - class object — typed events, Git-like trace, deterministic fork/replay/intervene primitives; 5× faster fork than Docker, >95% prompt-cache reuse; CooperBench pair-coding success 28.8% → 54.7%, 58% lower wall-clock on TerminalBench-2 | [PDF](https://arxiv.org/pdf/2605.10913) |
- EurekAgent: Agent Environment Engineering is All You Need For Autonomous Scientific Discovery (June 2026) - engineering dimensions (permissions, artifacts, budget, human-in-the-loop) enable off-the-shelf CLI agents to set SOTA on math, kernel engineering, and ML tasks at low cost; open source (THU-Team-Eureka/EurekAgent) | [PDF](https://arxiv.org/pdf/2606.13662) |
- AgentAtlas: Beyond Outcome Leaderboards for LLM Agents (May 2026) - state control-decision taxonomy and trajectory-failure vocabulary for separating outcome success from control-decision and trajectory quality; explicit label menus account for 14–40 pp of apparent agent capability | [PDF](https://arxiv.org/pdf/2605.20530) |
- Stop Hand-Holding Your Coding Agent: Engineering the Loops that Replace Step-by-Step Prompting (July 2026) - level verification ladder, architecture, stopping rule, memory), design principles, and anti-patterns from a corpus of 50 real loops | [PDF](https://arxiv.org/pdf/2607.00038) |
- From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents (July 2026) - dominant enterprise prototypes into code-owned, auditable harnesses — source-to-claim pipeline, seven validation dimensions (grounding, routing, trace, hygiene, recommendation language, runtime interfaces, latency), and the principle that "prompts are not guardrails"; validated across 25 listed companies and 270 live-LLM runs | [PDF](https://arxiv.org/pdf/2607.08028) |
-
Agent Safety
- ClawSafety: "Safe" LLMs, Unsafe Agents (April 2026) - privilege domains (SWE/finance/medical/legal/DevOps), 3 injection channels (skill files, email, web); 40–75% attack success rate; safety depends on model + framework stack, not model alone | [PDF](https://arxiv.org/pdf/2604.01438) |
- Supply-Chain Poisoning Attacks Against Agent Skill Ecosystems (April 2026)
- BeSafe-Bench: Behavioral Safety Risks of Situated Agents (2026) - risk categories; even the best agent completes <40% of tasks under full safety constraints | [PDF](https://arxiv.org/pdf/2603.25747) |
- Agents of Chaos (2026) - week red-team study of live autonomous agents (email, Discord, shell, persistent memory) — documents 11 real attack categories including cross-agent unsafe practice propagation, identity spoofing, unauthorized resource consumption, and false task completion (32 HF likes) | [PDF](https://arxiv.org/pdf/2602.20021) |
- LPS-Bench: Long-Horizon Safety Benchmarking for Computer-Use Agents (2026) - use agents focused on long-horizon tasks where risk accumulates across many UI actions — useful for testing confirmation discipline, phishing resistance, and context drift | [PDF](https://arxiv.org/pdf/2602.03255) |
- Internal Safety Collapse in Frontier LLMs (2026) - Bench — frontier models fail at 95.3% rate on dual-use professional tasks where capability and harm co-occur; advanced models are *more* vulnerable than earlier LLMs because their capabilities become liabilities | [PDF](https://arxiv.org/pdf/2603.23509) |
- Jailbreaking LLMs & VLMs: Mechanisms, Evaluation, and Unified Defense (2026) - context, RL, and multimodal attack types; proposes 3-layer defense framework (perception / generation / parameter layers) | [PDF](https://arxiv.org/pdf/2601.03594) |
- Attack and Defense Landscape of Agentic AI (2026)
- Architecting Secure AI Agents: System-Level Defenses Against Indirect Prompt Injection (March 2026)
- Parallax: Why AI Agents That Think Must Never Act (April 2026) - based safety is architecturally insufficient for agents with execution capability; introduces Parallax, a plan-then-execute separation architecture with formal safety guarantees | [PDF](https://arxiv.org/pdf/2604.12986) |
- Safety, Security, and Cognitive Risks in World Models (2026) - model-equipped agents — adversarial attacks, goal misgeneralisation, deceptive alignment, automation bias; extends MITRE ATLAS and OWASP to world model stack | [PDF](https://arxiv.org/pdf/2604.01346) |
- Self-Propagating Attacks Across LLM Agent Ecosystems (March 2026) - like self-spreading malware targeting agent ecosystems via MCP, tool chains, and shared memory | [PDF](https://arxiv.org/pdf/2603.15727) |
- From Untrusted Input to Trusted Memory: A Systematic Study of Memory Poisoning Attacks in LLM Agents (June 2026) - injection defenses are insufficient against cross-session memory manipulation | [PDF](https://arxiv.org/pdf/2606.04329) |
- Agent Data Injection Attacks are Realistic Threats to AI Agents (July 2026) - world attacks on web and coding agents including Claude Code, Codex, and Gemini CLI | [PDF](https://arxiv.org/pdf/2607.05120) |
-
Automatic Optimization
- ProTeGi / Gradient Descent for Prompts (2023) - optimization methods |
-
Programming Languages
Categories
Papers
150
Agent Ecosystem
77
Video courses
62
Table of Contents
48
Official Guides
34
Tools & Libraries
28
Frameworks
25
Prompting libraries & tools (in alphabetical order)
16
Official Agent Building & Prompt Engineering Guides
15
Papers on advanced prompting to improve reasoning
9
Context Engineering
8
System Prompt Leaks
6
Prompting guides
5
Prompt Attack and Prompt Protect
2
Basic Prompt Engineering
1
Sub Categories
Frameworks
50
Harness Engineering
49
Voice & Realtime Agents
32
Reasoning Techniques
16
Agent Evaluation
14
Agent Safety
14
Context & Memory
13
Automatic Optimization
12
Agent Reliability
11
Self-Improving Agents
8
Red Team & Security
8
Prompt Attack & Defense
8
Low-Code & Workflow Platforms
8
Multi-Agent Coordination
8
Instruction Following
7
Agent Skills
7
RAG & Knowledge
7
Multimodal Prompting
7
Foundations
6
Tool Use
6
Medical & Health AI
6
Embodied AI & World Models
6
Surveys
5
Eval & Observability
4
Automatic Prompt Optimization
3
A2A — Agent-to-Agent Protocol
3
Eval & Testing
3
Prompt Programming
2
MCP — Model Context Protocol
2
Tool Use & Reliability
2
Legacy (2023 era — kept for reference)
1
Keywords
llm
32
ai
26
ai-agents
25
python
18
agents
17
openai
15
claude-code
14
prompt-engineering
14
codex
13
chatgpt
12
agent
12
typescript
12
mcp
11
anthropic
11
developer-tools
10
rag
10
langchain
9
cli
7
claude
7
generative-ai
7
framework
7
gemini
7
large-language-models
6
llmops
6
llm-evaluation
6
prompts
5
awesome-list
5
agentic-ai
5
cursor
5
multi-agent
5
context-engineering
5
llms
5
harness
5
mcp-server
4
genai
4
agent-framework
4
workflows
4
coding-agents
4
machine-learning
4
open-source
4
sdk
4
orchestration
3
awesome
3
automation
3
language-model
3
tree-sitter
3
self-hosted
3
javascript
3
nodejs
3
openclaw
3