{"id":44823775,"url":"https://github.com/kafclaw/kafclaw","last_synced_at":"2026-03-15T08:54:48.205Z","repository":{"id":338810825,"uuid":"1159169884","full_name":"KafClaw/KafClaw","owner":"KafClaw","description":"🦾 Ultra-compact OpenClaw in Go + Kafka enabled. Operate a distributed swarm.","archived":false,"fork":false,"pushed_at":"2026-02-21T04:01:25.000Z","size":2044,"stargazers_count":9,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-21T12:52:18.158Z","etag":null,"topics":["ai","ai-agents","ai-assistant","amazon-msk","confluent-kafka","kafka","kafscale","openclaw","own-your-data","redpanda"],"latest_commit_sha":null,"homepage":"https://kafclaw.scalytics.io","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KafClaw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":"NOTICE","maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-16T12:03:05.000Z","updated_at":"2026-02-21T03:26:22.000Z","dependencies_parsed_at":"2026-02-20T07:02:23.404Z","dependency_job_id":null,"html_url":"https://github.com/KafClaw/KafClaw","commit_stats":null,"previous_names":["kafclaw/kafclaw"],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/KafClaw/KafClaw","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KafClaw%2FKafClaw","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KafClaw%2FKafClaw/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KafClaw%2FKafClaw/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KafClaw%2FKafClaw/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KafClaw","download_url":"https://codeload.github.com/KafClaw/KafClaw/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KafClaw%2FKafClaw/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29707520,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-22T05:59:28.568Z","status":"ssl_error","status_checked_at":"2026-02-22T05:58:46.208Z","response_time":110,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","ai-agents","ai-assistant","amazon-msk","confluent-kafka","kafka","kafscale","openclaw","own-your-data","redpanda"],"created_at":"2026-02-16T21:02:43.236Z","updated_at":"2026-03-01T06:03:02.127Z","avatar_url":"https://github.com/KafClaw.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# KafClaw\n[![CI (Smoke+Fuzz+Go)](https://github.com/KafClaw/KafClaw/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/KafClaw/KafClaw/actions/workflows/ci.yml)\n[![Release](https://github.com/KafClaw/KafClaw/actions/workflows/release.yml/badge.svg?branch=main)](https://github.com/KafClaw/KafClaw/actions/workflows/release.yml)\n[![Pages](https://github.com/KafClaw/KafClaw/actions/workflows/pages.yml/badge.svg?branch=main)](https://github.com/KafClaw/KafClaw/actions/workflows/pages.yml)\n[![CodeQL](https://github.com/KafClaw/KafClaw/actions/workflows/codeql.yml/badge.svg?branch=main)](https://github.com/KafClaw/KafClaw/actions/workflows/codeql.yml)\n\n[![Go Report Card](https://goreportcard.com/badge/github.com/KafClaw/KafClaw)](https://goreportcard.com/report/github.com/KafClaw/KafClaw)\n[![License](https://img.shields.io/github/license/KafClaw/KafClaw)](LICENSE)\n\nKafClaw is backed by [Scalytics](https://www.scalytics.io). We do not create, operate, or endorse any crypto tokens. If you see token-based fundraising using the KafClaw name, it is not affiliated with this project.\n\n## Platform Suite\n\nKafClaw is part of a broader infrastructure stack:\n\n- **KafScale**: Kafka-compatible + S3-compatible platform for event transport and large artifact flows.  \n  [kafscale.io](https://kafscale.io) • [github.com/kafscale](https://github.com/kafscale)\n- **GitClaw**: agent-first self-hosted Git platform for repository workflows and automation.\n- **Scalytics Copilot**: open-source operations stack for private AI inference with open models.  \n  [github.com/scalytics/ScalyticsCopilot](https://github.com/scalytics/ScalyticsCopilot)\n- **KafClaw**: agent runtime and coordination layer (local, Kafka-connected, and remote gateway modes).\n\n**Enterprise-grade multi-agent collaboration over Apache Kafka.**\n\nKafClaw is an agent coordination framework built in Go. It connects autonomous AI agents through Kafka-based messaging, giving them group collaboration, hierarchical orchestration, shared memory, and distributed skill routing — without coupling them to any single LLM provider, runtime, or deployment model.\n\nThe name reflects what it does: **Kaf**ka as the backbone, **Claw** as the grip that holds heterogeneous agents together.\n\n---\n\n## Core Contributions\n\n### 1. Enterprise-Ready Agent Communication via Kafka Protocol \n\nAgents communicate through a structured Kafka topic hierarchy. Every message flows through typed envelopes (`announce`, `request`, `response`, `trace`, `memory`, `audit`) with correlation IDs and timestamps, giving full observability out of the box.\n\n```\ngroup.\u003cname\u003e.announce             # join / leave / heartbeat\ngroup.\u003cname\u003e.requests             # task requests\ngroup.\u003cname\u003e.responses            # task responses\ngroup.\u003cname\u003e.traces               # distributed trace spans\ngroup.\u003cname\u003e.control.roster       # topic registry + member capabilities\ngroup.\u003cname\u003e.control.onboarding   # agent onboarding protocol\ngroup.\u003cname\u003e.tasks.status         # task progress updates\ngroup.\u003cname\u003e.observe.audit        # admin audit trail\ngroup.\u003cname\u003e.memory.shared        # persistent shared knowledge (via LFS/S3)\ngroup.\u003cname\u003e.memory.context       # ephemeral context sharing (TTL-based)\ngroup.\u003cname\u003e.orchestrator         # hierarchy discovery + zone coordination\ngroup.\u003cname\u003e.skill.\u003cs\u003e.requests   # per-skill task routing (dynamic)\ngroup.\u003cname\u003e.skill.\u003cs\u003e.responses  # per-skill responses (dynamic)\n```\n\nThis is not a toy pub-sub wrapper. It is a deliberate wire protocol designed for production traceability, auditability, and zero-downtime agent onboarding.\n\n### 2. Group Collaboration — Internal and Inter-Group\n\nAgents form **groups**. Within a group, they discover each other via heartbeats, delegate tasks through request/response topics, and share trace spans for distributed debugging.\n\n**Group-internal collaboration:**\n- Roster management with automatic heartbeat-based liveness detection\n- Task delegation with depth tracking and deadline propagation\n- Skill registration — agents advertise capabilities, others route tasks to them\n- Onboarding protocol (open or gated with challenge/response handshake)\n\n**Inter-group coordination** is handled by the **orchestrator**, which adds:\n- **Hierarchy**: parent-child agent relationships for delegation chains\n- **Zones**: security boundaries with `public`, `shared`, and `private` visibility\n- **Discovery**: agents announce themselves on the orchestrator topic; the hierarchy and zone graph update in real time\n\nA group of agents working on code review does not need to see the agents running customer support — unless a zone explicitly bridges them.\n\n### 3. Species-Independent Inter-Species Nervous System\n\nKafClaw decouples the *what* from the *how*. The Kafka topic layer is the nervous system; agents are the species.\n\nAn agent is anything that can produce and consume Kafka envelopes. It might be:\n- A Go binary running the full KafClaw runtime\n- A Python script using `kafka-python`\n- A Node.js service, a Rust worker, a shell script polling via `kcat`\n- A human operating through the WhatsApp or Telegram channel bridge\n\nThe wire format is JSON. The envelope schema is simple and documented. No SDK lock-in, no runtime dependency beyond Kafka itself.\n\n**Shared memory** reinforces this:\n- Agents publish knowledge artifacts to `memory.shared` (backed by S3/LFS for large payloads)\n- Ephemeral context goes to `memory.context` with TTL expiry\n- Locally, each agent can index received items into its own vector store (SQLite-vec, Qdrant, or custom) for semantic retrieval\n- The result: agents with different architectures, models, and languages can build on each other's work without direct coupling\n\n### 4. Shared Learning\n\nKnowledge does not stay locked inside a single agent's context window.\n\n- **Memory items** published to the group are stored locally in each subscribing agent's vector index, making them available for RAG retrieval in future conversations\n- **Expertise tracking** records what each agent knows and has done, so the group can route questions to the right specialist\n- **Auto-indexing** captures conversation context and indexes it for later recall\n- **Skill channels** let agents register domain expertise as addressable services — other agents don't need to know the implementation, just the skill name\n\nThe learning loop: Agent A discovers something, shares it as a memory item, Agent B indexes it, Agent B uses it to answer a question three days later. No central brain required.\n\n---\n\n## Architecture\n\n```\n                         ┌──────────────────────────────────┐\n                         │           Apache Kafka           │\n                         │                                  │\n                         │  control ─ tasks ─ observe       │\n                         │  memory ─ orchestrator ─ skills  │\n                         └──────┬──────────────┬────────────┘\n                                │              │\n                    ┌───────────┘              └───────────┐\n                    │                                      │\n             ┌──────┴──────┐                        ┌──────┴──────┐\n             │   Agent A   │                        │   Agent B   │\n             │  (Go/Full)  │                        │ (Any lang)  │\n             │             │                        │             │\n             │ Agent Loop  │                        │ Kafka       │\n             │ Tool Reg.   │                        │ Consumer    │\n             │ Memory Svc  │                        │ + Producer  │\n             │ LLM Provider│                        │             │\n             │ Channels    │                        │             │\n             └─────────────┘                        └─────────────┘\n                    │\n        ┌───────────┼───────────┐\n        │           │           │\n   WhatsApp     Telegram     Web UI\n   (whatsmeow)  (bridge)    (:18791)\n```\n\n**Full KafClaw agents** (Go runtime) include:\n- **Agent loop** with LLM provider abstraction (OpenAI, OpenRouter)\n- **Tool registry** (filesystem, shell, memory, web — with security sandboxing)\n- **Message bus** decoupling channels from the agent loop\n- **Timeline DB** (SQLite) for event logging, media, and distributed tracing\n- **Policy engine** for message classification, token quotas, and rate limiting\n- **Scheduler** for cron jobs and deferred tasks\n\n**Lightweight agents** only need a Kafka client and the envelope JSON schema.\n\n---\n\n## Operating Modes\n\n| Mode | Kafka | Orchestrator | Use Case |\n|------|-------|-------------|----------|\n| `standalone` | No | No | Single-agent desktop assistant |\n| `group` | Yes | No | Peer-to-peer agent collaboration |\n| `full` | Yes | Yes | Hierarchical multi-agent with zones |\n| `headless` | Yes | Yes | Server deployment (0.0.0.0 + auth) |\n\n```bash\ncd KafClaw\n\nmake run-standalone    # No Kafka, no orchestrator\nmake run               # Default gateway\nmake run-full          # Group + orchestrator enabled\nmake run-headless      # Server mode (requires KAFCLAW_GATEWAY_AUTH_TOKEN)\n```\n\n---\n\n## Quick Start\n\n```bash\n# Prerequisites: Go 1.24+, Apache Kafka (for group modes)\n\ncd KafClaw\n\n# Build\nmake build\n\n# Run single message (standalone, no Kafka needed)\n./kafclaw agent -m \"hello\"\n\n# Run gateway (standalone mode)\nmake run-standalone\n\n# Run with group collaboration\nmake run-full\n\n# Run tests\ngo test ./...\nmake test-smoke              # critical-path smoke tests\nmake test-critical           # enforce 100% critical-logic coverage gate\nmake test-fuzz               # fuzz critical guard logic\n\n# Kafka diagnostics\n./kafclaw kshark --broker localhost:9092 --test-connection\n```\n\n### Configuration\n\nLoaded in order: environment variables \u003e `~/.kafclaw/config.json` \u003e defaults.\n\n| Variable | Default | Description |\n|----------|---------|-------------|\n| `KAFCLAW_GROUP_ENABLED` | `false` | Enable Kafka group collaboration (legacy: `MIKROBOT_GROUP_ENABLED`) |\n| `KAFCLAW_ORCHESTRATOR_ENABLED` | `false` | Enable hierarchical orchestration (legacy: `MIKROBOT_ORCHESTRATOR_ENABLED`) |\n| `KAFCLAW_ORCHESTRATOR_ROLE` | `worker` | Agent role: `orchestrator`, `worker`, `observer` (legacy: `MIKROBOT_ORCHESTRATOR_ROLE`) |\n| `KAFCLAW_GROUP_KAFKA_BROKERS` | `localhost:9092` | Kafka broker addresses (legacy: `MIKROBOT_KAFKA_BROKERS`) |\n| `KAFCLAW_GATEWAY_HOST` | `127.0.0.1` | API bind address (legacy: `MIKROBOT_GATEWAY_HOST`) |\n| `KAFCLAW_GATEWAY_AUTH_TOKEN` | *(empty)* | Bearer token for headless mode (legacy: `MIKROBOT_GATEWAY_AUTH_TOKEN`) |\n\nGateway ports: **18790** (API), **18791** (dashboard).\n\n---\n\n## Key Packages\n\n```\ninternal/\n├── group/          # Kafka-based group collaboration, onboarding, skills, shared memory\n├── orchestrator/   # Hierarchy, zones, discovery\n├── agent/          # Core agent loop, context builder, soul file loading\n├── bus/            # Async message bus (pub-sub, decouples channels from agent)\n├── channels/       # WhatsApp, Telegram, Discord, Web — Channel interface\n├── config/         # Env / file / default config loading\n├── provider/       # LLM abstraction (OpenAI, OpenRouter, Whisper, TTS)\n├── memory/         # Vector store, semantic search, expertise tracking, auto-indexing\n├── session/        # Per-session conversation history (JSONL persistence)\n├── timeline/       # SQLite event log, media storage, trace/span IDs\n├── tools/          # Registry-based tools (fs, shell, memory, web) with security sandbox\n├── policy/         # Message classification, token quotas, rate limiting\n├── scheduler/      # Cron and deferred task scheduling\n├── kshark/         # Kafka diagnostic tool (connection, topics, partitions, metrics)\n└── approval/       # Task approval workflows\n```\n\n---\n\n## Diagnostic Tooling: KShark\n\nKShark is a built-in Kafka diagnostic tool for verifying connectivity and inspecting group infrastructure:\n\n```bash\n./kafclaw kshark --broker localhost:9092 --test-connection\n./kafclaw kshark --broker localhost:9092 --probe-topics --group mygroup\n./kafclaw kshark --broker localhost:9092 --network-diag\n```\n\n---\n\n## Releases\n\n- Local: `make release-patch` (or `release-minor`, `release-major`) in `kafclaw/`\n- CI: push a tag `vX.Y.Z` to trigger the build workflow\n- Cross-compile: `make dist-go` produces binaries for darwin/linux (amd64/arm64)\n- Electron desktop: `make electron-dist` packages for current platform\n- See `docs/release.md` for details\n\n## License\n\nLicensed under the [Apache License, Version 2.0](LICENSE).\n\nThis project was inspired by [HKUDS/nanobot](https://github.com/HKUDS/nanobot). While originally a fork, KafClaw has been completely rewritten in Go to support enterprise-grade Kafka backends and multi-agent coordination. Original attribution is preserved in the [NOTICE](NOTICE) file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkafclaw%2Fkafclaw","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkafclaw%2Fkafclaw","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkafclaw%2Fkafclaw/lists"}