{"id":50566653,"url":"https://github.com/a2aproject/a2a-itk","last_synced_at":"2026-06-04T15:30:27.015Z","repository":{"id":357816629,"uuid":"1238468033","full_name":"a2aproject/a2a-itk","owner":"a2aproject","description":"Integration Testing Kit for A2A SDKs","archived":false,"fork":false,"pushed_at":"2026-05-14T12:27:58.000Z","size":119,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-14T12:38:20.158Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/a2aproject.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-14T06:35:54.000Z","updated_at":"2026-05-14T12:28:02.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/a2aproject/a2a-itk","commit_stats":null,"previous_names":["a2aproject/a2a-itk"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/a2aproject/a2a-itk","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/a2aproject%2Fa2a-itk","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/a2aproject%2Fa2a-itk/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/a2aproject%2Fa2a-itk/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/a2aproject%2Fa2a-itk/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/a2aproject","download_url":"https://codeload.github.com/a2aproject/a2a-itk/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/a2aproject%2Fa2a-itk/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33912317,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-04T02:00:06.755Z","response_time":64,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-04T15:30:26.051Z","updated_at":"2026-06-04T15:30:27.006Z","avatar_url":"https://github.com/a2aproject.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🛠 ITK: Integration Test Kit\n\n![Platform](https://img.shields.io/badge/Platform-Linux-orange.svg)\n![A2A Protocol](https://img.shields.io/badge/Protocol-A2A-blue.svg)\n![Status](https://img.shields.io/badge/Status-Active-green.svg)\n\nITK is a technical toolkit designed to verify compatibility across different A2A SDK implementations and versions. It uses a multi-hop traversal model to ensure that messages can be routed across a cluster of agents using varied transport protocols (JSON-RPC, gRPC, and HTTP-JSON/REST), including support for streaming.\n\n---\n\n## 🏗 Architecture\n\nThe kit operates by dispatching a single, deeply nested instruction through a chain of agents, structuring the traversal as a complete verification cycle.\n\n### Traversal Cycle Flow\n1. **Dispatch**: The Test Runner initiates execution by sending the nested traversal instruction to the primary entrypoint agent (**Agent 1**) via **JSON-RPC**.\n2. **Consistent Inter-Agent Traversal**: For intermediate hops between agents within a given scenario, messaging evaluates a single, consistent transport protocol. Each receiving agent resolves the next target's agent card, maps the transport, and forwards the remaining payload.\n3. **Cycle Completion \u0026 Trace Verification**: Upon completing the final traversal hop, the execution unwinds, and **Agent 1** returns a JSON-RPC response to the Test Runner across all modes.\n   - **Standard / Streaming Verification**: The Test Runner verifies the traversal trace directly from the returned response payload.\n   - **Push Notification Verification**: In scenarios evaluating asynchronous event delivery (`push_notification`), participating agents asynchronously push trace updates to an isolated Mock Notification Server during traversal. The Test Runner queries this Push Notification Service (`GET /notifications`) to read and verify the accumulated traversal trace.\n\n```mermaid\ngraph TD\n    Runner[Test Runner] --\u003e|1. JSON-RPC Request| Ag1[Agent 1]\n    Ag1 -.-\u003e|2. Configured Transport| Ag2[Agent 2]\n    Ag2 -.-\u003e|2. Configured Transport| AgN[...Agent N]\n    \n    %% Return Path (Always Executed)\n    AgN -.-\u003e|3. Response Unwinding| Ag1\n    Ag1 --\u003e|3. Standard Verification - JSON-RPC Response| Runner\n    \n    %% Push Notification Path \u0026 Verification\n    PNS[Push Notification Service]\n    Ag1 -.-\u003e|Async Push Event| PNS\n    Ag2 -.-\u003e|Async Push Event| PNS\n    AgN -.-\u003e|Async Push Event| PNS\n    PNS --\u003e|4. Push Verification - GET /notifications| Runner\n```\n\n---\n\n## 📈 Graph-Based Traversal\n\nTo achieve comprehensive verification, ITK utilizes graph-based traversal algorithms:\n\n- **Eulerian Circuits**: Implements **Hierholzer's Algorithm** to generate a single linear nested instruction chain that covers 100% of directed edges in the agent cluster exactly once.\n- **Dynamic Topology**: Supports complete digraphs (n-to-n) or custom edge definitions to test specific connection patterns.\n\n---\n\n## 🌟 Key Features\n\n### 🤖 SDK-Agnostic Test Runner\n- **Universal Independence**: Operates completely independently of any underlying A2A SDK version or language implementation.\n\n### 🔌 Extensible SDK Support \u0026 CI/CD Integration\nITK is structured to validate in-development SDK codebases against a cluster of reference stable configurations, basing on released versions of A2A SDKs. It is serving as a verification gate for **Pull Requests** and automated **nightly runs**.\n\n- **Stable Reference Baselines**: Pre-packaged reference implementations for released A2A versions.\n- **Current Agent Mounting**: Dynamically mounts a local SDK source checkout into a designated \"current\" agent process to evaluate compatibility against the stable cluster.\n\n#### SDK Support Matrix\n\n| SDK Language | Stable v0.3 | Stable v1.0 | Current Mount Support |\n| :--- | :---: | :---: | :---: |\n| **Python** | ✅ | ✅ | ✅ |\n| **Go** | ✅ | ✅ | ✅ |\n| **TypeScript** | ❌ | ❌ | ✅ |\n| **.NET** | ❌ | ❌ | ⚠️ |\n| **Java** | ❌ | ❌ | ⚠️ |\n| **Rust** | ❌ | ❌ | ❌ |\n\n\u003e [!NOTE]\n\u003e ⚠️ *Indicates preliminary integration layout utilizing initial placeholders for current SDK state *\n\n### 🛤 Multi-Protocol \u0026 Interaction Modes\nExecutes standalone traversal scenarios dedicated to verifying compatibility across each primary transport protocol:\n- **JSON-RPC**\n- **gRPC**\n- **HTTP-JSON (REST)**\n\nWithin these transport scenarios, the following A2A features can be tested:\n- **Send Message**: Standard request-response messaging.\n- **Send Message (Streaming)**: Streaming message payloads across compatible transport protocols.\n- **Push Notification**: Asynchronous event delivery and ingestion verification.\n- **Task Resubscription**: Initiates a streaming communication lifecycle where the client extracts the active task ID, disconnects, re-subscribes to resume the stream, and finally issues a cancellation request (`cancel_task`) to terminate the task.\n\n---\n\n## 📂 Project Structure\n\n- `agents/`: SDK-specific agent implementations (e.g., Go, Python).\n- `dashboard/`: Static web assets (HTML, JS, CSS) for rendering compatibility matrix test results.\n- `scripts/`: Auxiliary utilities, including result-parsing metrics pipelines.\n- `test_suite/`: Modular agent definitions, launchers, and traversal logic.\n- `itk_service.py`: FastAPI orchestration service for remote test execution.\n- `notifications_app.py`: Dedicated mock server for ingesting and verifying SDK push notifications.\n- `run_tests.py`: CLI orchestrator for running concurrent test scenarios.\n- `testlib.py`: Core logic for cluster lifecycle, port management, and test execution.\n- `Dockerfile`: Container environment definition for the ITK service.\n\n---\n\n## 🚀 Usage\n\n### Prerequisites\n- **uv**: Python package and project manager.\n- **Go 1.25+**: Required for Go agent builds.\n- **Node.js v20**: Required for certain A2A utility components.\n\n### 1. Local Run with Stable SDKs\nRun the standard integration suite locally using purely the stable reference baseline agents:\n```bash\nuv run run_tests.py\n```\n\n### 2. Setting up PR Testing \u0026 Nightly Runs\nTo gate **Pull Requests** or schedule automated **nightly runs** against an in-development SDK repository (e.g., `a2a-python` or `a2a-go`), consuming codebases mount their local source directly into ITK's validation container runtime.\n\n#### Integration Requirements\n\n1. **Instruction Handling Agent Implementation**:\n   - Consuming SDKs must implement an instruction handling agent capable of parsing nested traversal instructions and executing varied agent behavior modes.\n   - **Implementation Reference**: The native stable baselines hosted in this repository ([agents/go](https://github.com/a2aproject/a2a-itk/tree/main/agents/go) and [agents/python](https://github.com/a2aproject/a2a-itk/tree/main/agents/python)) serve as comprehensive production referrals for custom handling logic.\n\n2. **Custom Scenario Definitions**:\n   - Consuming repositories supply customized scenario suites tuned to the desired depth of testing:\n     - **PR Testing (`scenarios.json`)**: Shorter, optimized validation paths focused on rapid compatibility verification.\n     - **Nightly Runs (`scenario_full.json`)**: Comprehensive, multi-hop matrix configurations evaluating edge-case behavior and transport stability across protocol matrix boundaries.\n   - **Scenario Schema \u0026 Fields**: Configuration files define a root object containing a `tests` array. Each scenario object specifies:\n     - `name` *(String, Required)*: Descriptive display title for the test scenario.\n     - `sdks` *(Array of Strings, Required)*: Target agent identifiers participating in the cluster (e.g., `[\"current\", \"python_v10\", \"go_v03\"]`). The array index dictates node IDs for routing.\n     - `protocols` *(Array of Strings, Required)*: Transport mechanisms executed under this topology (`\"jsonrpc\"`, `\"grpc\"`, `\"http_json\"`).\n     - `behavior` *(String, Required)*: Verification interaction mode (`\"send_message\"`, `\"push_notification\"`, `\"resubscribe\"`).\n     - `edges` *(Array of Strings, Optional)*: Custom directed communication edge pairs using zero-based SDK indices (e.g., `[\"0-\u003e1\", \"1-\u003e0\"]`). If omitted, defaults to a complete digraph (n-to-n) topology.\n     - `streaming` *(Boolean, Optional)*: If set to `true`, activates streaming message payload delivery. Defaults to `false`.\n     - `build_subtests` *(Boolean, Optional)*: If set to `true`, instructs the test runner to extract and execute targeted sub-graphs or individual edges as distinct validation subtests. Defaults to `false`.\n\n3. **Automated Orchestration Wrapper**:\n   - The target codebase maintains a runner script (e.g., `run_itk.sh`) that exports `A2A_ITK_REVISION`, clones the test suite, compiles the core test container, dynamically mounts the workspace source as the `current` agent context, and verifies execution outputs.\n\n#### Consuming SDK References\nReview production integration structures, runner scripts, and CI workflow templates directly in the main remote repositories:\n\n- **Python SDK (`a2a-python`)**:\n  - **Integration Setup**: Core integration layout and runner configurations ([itk/](https://github.com/a2aproject/a2a-python/tree/main/itk)).\n  - **PR Validation Workflow**: Continuous integration gating for Pull Requests ([itk.yaml](https://github.com/a2aproject/a2a-python/blob/main/.github/workflows/itk.yaml)).\n  - **Nightly Run Workflow**: Automated scheduled test matrix verification ([nightly.yaml](https://github.com/a2aproject/a2a-python/blob/main/.github/workflows/nightly.yaml)).\n\n- **Go SDK (`a2a-go`)**:\n  - **Integration Setup**: Core integration layout and runner configurations ([itk/](https://github.com/a2aproject/a2a-go/tree/main/itk)).\n  - **PR Validation Workflow**: Continuous integration gating for Pull Requests ([itk.yaml](https://github.com/a2aproject/a2a-go/blob/main/.github/workflows/itk.yaml)).\n  - **Nightly Run Workflow**: Automated scheduled test matrix verification ([itk-nightly.yaml](https://github.com/a2aproject/a2a-go/blob/main/.github/workflows/itk-nightly.yaml)).\n\n---\n\n## 📊 Centralized Dashboard\n\nITK hosts a static centralized visualization dashboard to aggregate and display recurring nightly integration test matrix results.\n\n- **Public Dashboard URL**: [A2A ITK Dashboard](https://a2aproject.github.io/a2a-itk/dashboard)\n\n### Daily Snapshot Processing\n\n\u003e [!NOTE]\n\u003e The centralized dashboard does **not** provide real-time live monitoring. It functions as a daily integration status update reflecting completed overnight matrix executions.\n\nThe data presentation pipeline operates via a decoupled publication model:\n1. **Metrics Artifact Generation**: Consuming SDK repositories execute comprehensive multi-protocol traversal suites overnight. Upon completion, extracted run results are formatted as structured JSON metrics artifacts.\n2. **Rolling Release Ingestion**: Consuming repositories push these extracted JSON artifacts directly to a specially dedicated rolling release tag named **`nightly-metrics`** inside their own GitHub releases environment.\n3. **Aggregated Deployment**: A scheduled daily workflow within the `a2a-itk` repository fetches these static released metrics from each target SDK's `nightly-metrics` tag and triggers a static site compilation, re-deploying the unified frontend to GitHub Pages.\n\n### Onboarding a New SDK to the Dashboard\n\nWhen integrating automated nightly matrix runs for a newly onboarded language library, follow these steps to render its compatibility outputs globally:\n\n1. Ensure the new SDK's nightly continuous integration workflow publishes its final output JSON artifacts to a rolling release tag named `nightly-metrics`.\n2. Modify the automated dashboard deployment workflow within this repository ([.github/workflows/deploy_dashboard.yaml](https://github.com/a2aproject/a2a-itk/blob/main/.github/workflows/deploy_dashboard.yaml)) to fetch the metric payload from the new target SDK's release space alongside existing baseline configurations.\n\n---\n\n## 📋 Task Backlog\n\nTo further expand verification depth and ensure absolute compliance with the growing Agent2Agent protocol standard, future iterations aim to address the following roadmap items:\n\n### 1. Erroneous Behavior \u0026 Fault Tolerance Verification\n- [ ] **Error Assertion Mapping**: Verify that SDK implementations raise structurally correct exceptions under anomalous execution paths.\n- [ ] **Out-of-Order Processing**: Assert failures when attempting to enqueue task status updates prior to task state creation.\n- [ ] **Terminal State Handshakes**: Validate graceful rejections when initiating subscriptions against explicitly completed or failed task instances.\n\n### 2. Protocol Specification \u0026 Schema Validation\n- [ ] **Agent Card Passing Suites**: Establish targeted automated subtests focused exclusively on resolving, exchanging, and validating `AgentCard` payload structures.\n- [ ] **Payload Content Boundaries**: Expand schema adherence gates ensuring message envelopes strictly align with explicit protocol schema definitions.\n\n### 3. Expanded A2A API Capability Coverage\nIncorporate traversal test strategies evaluating additional native client API contracts present in standard baseline models:\n- [ ] `get_task` / `list_tasks`\n- [ ] `create_task_push_notification_config` / `delete_task_push_notification_config`\n- [ ] `get_extended_agent_card`\n\n### 4. Missing Stable Baseline Implementations\nPackage stable agents images for:\n- [ ] **TypeScript** baseline agents\n- [ ] **.NET** baseline agents\n- [ ] **Java** baseline agents\n- [ ] **Rust** baseline agents\n\n### 5. Client SDK Repository Orchestration\nIntegrate full continuous integration orchestration pipelines and custom instruction handlers across client SDK repositories to transition them from placeholders to active validation status:\n- [ ] **TypeScript SDK**: Configure automated scheduled **nightly runs** pipeline publishing validation JSON payloads to dashboard `nightly-metrics` releases.\n- [ ] **Java SDK**: Implement functional instruction handling agents and scenario orchestration scripts to replace existing basic `current` placeholders.\n- [ ] **.NET SDK**: Implement functional instruction handling agents and scenario orchestration scripts to replace existing basic `current` placeholders.\n- [ ] **Rust SDK**: Set up core mounting configuration, custom handlers, and full repository verification workflows.\n\n### 6. Automated Baseline Lifecycle\n- [ ] **Stable Agent Version Bumping**: Implement automated CI/CD workflows to periodically detect new stable upstream A2A SDK releases and automatically bump version configurations for ITK reference baseline agents.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fa2aproject%2Fa2a-itk","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fa2aproject%2Fa2a-itk","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fa2aproject%2Fa2a-itk/lists"}