{"id":49215841,"url":"https://github.com/ericmey/musubi","last_synced_at":"2026-05-17T06:13:56.741Z","repository":{"id":352955570,"uuid":"1198658695","full_name":"ericmey/musubi","owner":"ericmey","description":"Musubi (結び) — Aoi's shared memory and thought layer. The braiding of threads between presences.","archived":false,"fork":false,"pushed_at":"2026-04-21T22:26:48.000Z","size":4264,"stargazers_count":0,"open_issues_count":9,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-21T22:28:00.180Z","etag":null,"topics":["ai-agents","ai-memory","cosign","fastapi","grpc","homelab","hybrid-search","livekit","mcp","memory-system","model-context-protocol","obsidian","ollama","pydantic","python","qdrant","rag","retrieval-augmented-generation","self-hosted","vector-search"],"latest_commit_sha":null,"homepage":"https://github.com/ericmey/musubi","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ericmey.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-04-01T16:23:54.000Z","updated_at":"2026-04-21T21:17:48.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ericmey/musubi","commit_stats":null,"previous_names":["ericmey/musubi"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/ericmey/musubi","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ericmey%2Fmusubi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ericmey%2Fmusubi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ericmey%2Fmusubi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ericmey%2Fmusubi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ericmey","download_url":"https://codeload.github.com/ericmey/musubi/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ericmey%2Fmusubi/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32203362,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-23T20:19:26.138Z","status":"ssl_error","status_checked_at":"2026-04-23T20:19:23.520Z","response_time":53,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-agents","ai-memory","cosign","fastapi","grpc","homelab","hybrid-search","livekit","mcp","memory-system","model-context-protocol","obsidian","ollama","pydantic","python","qdrant","rag","retrieval-augmented-generation","self-hosted","vector-search"],"created_at":"2026-04-24T00:01:39.637Z","updated_at":"2026-05-17T06:13:56.734Z","avatar_url":"https://github.com/ericmey.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003ch1 align=\"center\"\u003eMusubi 結び\u003c/h1\u003e\n  \u003cp align=\"center\"\u003e\n    \u003cem\u003eShared memory for a small fleet of AI agents — three planes, local inference, a lifecycle engine that matures raw captures into a human-reviewable knowledge base.\u003c/em\u003e\n  \u003c/p\u003e\n  \u003cp align=\"center\"\u003e\n    \u003ca href=\"https://github.com/ericmey/musubi/releases/latest\"\u003e\u003cimg alt=\"Latest release\" src=\"https://img.shields.io/github/v/release/ericmey/musubi?sort=semver\u0026color=blue\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/ericmey/musubi/actions/workflows/ci.yml\"\u003e\u003cimg alt=\"CI\" src=\"https://github.com/ericmey/musubi/actions/workflows/ci.yml/badge.svg?branch=main\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/ericmey/musubi/actions/workflows/publish-core-image.yml\"\u003e\u003cimg alt=\"Signed image\" src=\"https://github.com/ericmey/musubi/actions/workflows/publish-core-image.yml/badge.svg?branch=main\"\u003e\u003c/a\u003e\n    \u003ca href=\"LICENSE\"\u003e\u003cimg alt=\"License: Apache 2.0\" src=\"https://img.shields.io/badge/license-Apache%202.0-blue\"\u003e\u003c/a\u003e\n    \u003cimg alt=\"Python 3.12\" src=\"https://img.shields.io/badge/python-3.12-blue\"\u003e\n    \u003cimg alt=\"cosign signed\" src=\"https://img.shields.io/badge/cosign-signed-brightgreen\"\u003e\n  \u003c/p\u003e\n\u003c/p\u003e\n\n---\n\nMusubi (結び — *\"to tie, to join, to bind\"*) is a memory server built for the moment when a single AI assistant is not enough: you're running several, each with its own role — one drafts notes, one answers questions, one cleans up the vault at 3am — and they need a shared substrate so that what one learns, the others can use.\n\nIt is a standalone Python service. Every downstream interface (MCP, LiveKit, a CLI, a browser extension) is an adapter that depends on Musubi's SDK. The core owns the memory model and the API; adapters own the surface.\n\n## The three planes\n\n```\n  ┌──────────────────────────────────────────────────────────────────────────┐\n  │                            MUSUBI CORE                                   │\n  │                                                                          │\n  │    episodic ──────► concept ──────► curated                              │\n  │    (raw captures)   (synthesized    (human-reviewed                      │\n  │                      themes)         Obsidian notes)                     │\n  │                                                                          │\n  │    + artifact plane (binary blobs — images, audio, pdfs)                 │\n  │    + thoughts plane (pub/sub channel for agent ↔ agent messaging)        │\n  │                                                                          │\n  └──────────────────────────────────────────────────────────────────────────┘\n```\n\n- **Episodic** — every sentence, quote, and observation an agent ingests. Written fast, scored by an LLM for importance, matured to `matured` after a dwell window if no contradictions surface.\n\n- **Concept** — a daily pass clusters mature episodics by shared topic + semantic similarity, asks an LLM to summarise each cluster into a `SynthesizedConcept`, checks for contradictions between concepts, and persists the result. Concepts reinforce (not duplicate) when similar clusters recur.\n\n- **Curated** — concepts that clear a promotion gate (reinforcement count + importance + age) are rendered as markdown and written to an [Obsidian](https://obsidian.md) vault. A human reviewer sees them, edits them, moves them around. Edits flow back into Musubi through the vault sync.\n\nA **lifecycle engine** runs five sweeps on cron: maturation (hourly), synthesis (03:00), promotion (04:00), demotion (05:00, sweep unreinforced rows out), reflection (06:00, writes a daily digest back to the vault). Each sweep is file-locked, idempotent, and emits structured events to a SQLite journal.\n\n## Why not a single RAG index?\n\nOne reason: **lifecycle**. A plain vector store keeps everything forever and retrieves by cosine. Musubi's episodic rows expire on a TTL if they aren't matured; mature ones flow up through synthesis; promoted ones become curated rows the human owns. Nothing sits in a \"everything I've ever said\" bucket — the system makes opinions about what's worth keeping and surfaces them for review.\n\nThe other reason: **agent ↔ agent memory**. Thoughts are a first-class message channel — agents can `send` and `subscribe` without a coordinator process. It's not a chat log, it's a shared board of short-lived state.\n\nDesign choices are captured as ADRs in [`docs/Musubi/13-decisions/`](docs/Musubi/13-decisions/).\n\n## Stack\n\n- **Python 3.12**, `pydantic v2`, strict `mypy`, `ruff` format + lint.\n- **Qdrant** for named-vector hybrid search (dense + sparse + rerank).\n- **TEI (text-embeddings-inference)** for BGE-M3 dense + SPLADE sparse + BGE-reranker — all GPU-hostable, CPU-fallback OK.\n- **Ollama** for LLM calls (maturation scoring, synthesis, promotion rendering, reflection). Defaults to Qwen 2.5 7B; any Ollama-tagged model works.\n- **FastAPI + HTTPX** HTTP surface; **gRPC** generated from `proto/` (partial). Both exposed on the same port.\n- **Docker Compose** for local / single-box deploy. **Ansible** playbooks for a managed-host rollout (`deploy/ansible/`). Every published image is [cosign](https://github.com/sigstore/cosign)-signed by digest, Trivy-scanned, and ships with a CycloneDX SBOM attestation.\n\n## Try it\n\n```bash\n# 1. Clone\ngit clone https://github.com/ericmey/musubi \u0026\u0026 cd musubi\n\n# 2. Install (Python 3.12 + uv required — https://docs.astral.sh/uv/)\nmake install\n\n# 3. Run the local test suite\nmake check\n```\n\nA single-box Docker Compose deploy is laid out in [`deploy/ansible/templates/docker-compose.yml.j2`](deploy/ansible/templates/docker-compose.yml.j2); a first-deploy runbook lives at [`deploy/runbooks/upgrade-image.md`](deploy/runbooks/upgrade-image.md). Pin by digest — tags are mutable, digests aren't, and the rest of the repo (ansible, CI, SECURITY.md) verifies by digest:\n\n```\nghcr.io/ericmey/musubi-core@sha256:\u003cdigest\u003e\n```\n\nFind the digest for a specific release on its GitHub Release page (the `publish-core-image` workflow attaches it) or via `docker buildx imagetools inspect ghcr.io/ericmey/musubi-core:\u003cversion\u003e`.\n\nVerify the signature before pinning in production:\n\n```bash\ncosign verify \\\n  --certificate-identity-regexp 'https://github.com/ericmey/musubi/.*' \\\n  --certificate-oidc-issuer https://token.actions.githubusercontent.com \\\n  ghcr.io/ericmey/musubi-core@sha256:\u003cdigest\u003e\n```\n\n## Repository layout\n\n```\nsrc/musubi/                 importable package\n  types/                    shared pydantic types — the schema is the contract\n  store/                    Qdrant layout, collection names, vector specs\n  embedding/                TEI client + Embedder protocol + FakeEmbedder\n  planes/                   episodic / concept / curated / artifact / thoughts\n  retrieve/                 scoring, hybrid search, fast/deep paths\n  lifecycle/                maturation / synthesis / promotion / demotion / reflection / runner\n  llm/                      Ollama client + frozen prompt files (per-name versioned)\n  api/                      FastAPI app, OpenAPI, /v1/* routes\n  sdk/                      Python client\n  adapters/                 MCP, LiveKit, OpenClaw (SDK + types only)\n  vault/                    Obsidian watcher + writer + write-log\n  observability/            structured logging + Prometheus metrics\n\ntests/                      mirrors src/musubi/ path-for-path\ndocs/Musubi/                the architecture vault (Obsidian) — source of truth for design\ndeploy/                     ansible, prometheus, grafana, docker-compose templates\n```\n\n## Status\n\n**v1.0 — released.** The API shape is sealed, agent-as-tenant namespace model is in ([ADR 0030](docs/Musubi/13-decisions/0030-agent-as-tenant.md)), both downstream integrations (openclaw-livekit, openclaw-musubi plugin) are on the canonical API, and the release chain from conventional commit through pinned-digest PR is fully hands-off.\n\nIn v1.0:\n\n- ✅ Three-plane memory (episodic / curated / concept) + artifacts + thoughts, per-plane collections, KSUID-addressed rows\n- ✅ All five lifecycle sweeps (maturation, synthesis, promotion, demotion, reflection) running on cron, SQLite-journaled, with a hard three-strikes rejection cap on promotion\n- ✅ Hybrid retrieval — dense BGE-M3 + sparse SPLADE + BGE-reranker + 2-segment cross-plane fanout in one call\n- ✅ Full HTTP surface (gRPC partial; lives behind the runtime flag `MUSUBI_GRPC`, default off)\n- ✅ Plane-aligned endpoint paths: `/v1/episodic`, `/v1/curated`, `/v1/concepts`, `/v1/artifacts`\n- ✅ Agent-as-tenant namespace model: `\u003cagent\u003e/\u003cchannel\u003e/\u003cplane\u003e` ([ADR 0030](docs/Musubi/13-decisions/0030-agent-as-tenant.md))\n- ✅ `Last-Event-ID` replay on `/v1/thoughts/stream` — reconnect without losing thoughts\n- ✅ MCP + LiveKit + OpenClaw adapters (external repos), static-bearer auth model with per-agent token support\n- ✅ Supply-chain: cosign + SBOM + Trivy on every published image\n- ✅ Fully automated release chain — conventional commit → release PR auto-merges → tag → signed image → digest pin PR auto-merges. Operator runs `ansible-playbook update.yml` from the control host and that is the only human step.\n- ✅ Operator tooling: `musubi promote force|reject` CLI, vault large-file skip-with-warning, rate-limited vault watcher\n\nPost-v1.0:\n- ⏳ Fleet orchestration — single-node today; multi-node HA is a post-1.0 design space and will need its own ADR\n- ⏳ Auto-deploy pipeline (image publish is automated; host rollout is still operator-driven via ansible)\n- ⏳ gRPC transport ADR ([#98](https://github.com/ericmey/musubi/issues/98)) — priority-low, not a 1.0 blocker\n- ⏳ Vault-wide sweep to update illustrative `eric/...` examples to agent-as-tenant (normative specs already flipped; docs carry a banner pointing at [ADR 0030](docs/Musubi/13-decisions/0030-agent-as-tenant.md))\n\nRoadmap detail lives in [`docs/Musubi/12-roadmap/`](docs/Musubi/12-roadmap/).\n\n## Contributing\n\nThis is a personal project that's been opened up for others to follow along, fork, and riff on. Contributions are welcome but the bar is: opened issue → discussion → PR. See [CONTRIBUTING.md](CONTRIBUTING.md) for the workflow and conventions.\n\nThe internal design is captured in an Obsidian-style vault at [`docs/Musubi/`](docs/Musubi/). It's readable as-is on GitHub but renders best in Obsidian. Every architectural decision has an ADR; every working module has a `slice` spec with a Test Contract.\n\n## Security\n\nIf you find a vulnerability, please don't open a public issue. See [SECURITY.md](SECURITY.md) for the disclosure process.\n\n## License\n\n[Apache 2.0](LICENSE) © 2025–2026 Eric Mey.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fericmey%2Fmusubi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fericmey%2Fmusubi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fericmey%2Fmusubi/lists"}