{"id":48401921,"url":"https://github.com/matzalazar/rhizome","last_synced_at":"2026-04-06T02:13:19.945Z","repository":{"id":347528370,"uuid":"1192482902","full_name":"matzalazar/rhizome","owner":"matzalazar","description":"Local-first semantic backlinks for Obsidian and Logseq — embeds your notes with a multilingual sentence transformer and writes   ## Related Notes sections as wikilinks, with no cloud API or database required.","archived":false,"fork":false,"pushed_at":"2026-03-28T12:04:12.000Z","size":3544,"stargazers_count":49,"open_issues_count":0,"forks_count":4,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-03-28T13:43:01.588Z","etag":null,"topics":["local-first","logseq","notes","obsidian","onnx","pkm","semantic-search","wikilinks","zettelkasten"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/matzalazar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-26T09:04:43.000Z","updated_at":"2026-03-28T12:04:16.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/matzalazar/rhizome","commit_stats":null,"previous_names":["matzalazar/rhizome"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/matzalazar/rhizome","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matzalazar%2Frhizome","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matzalazar%2Frhizome/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matzalazar%2Frhizome/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matzalazar%2Frhizome/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/matzalazar","download_url":"https://codeload.github.com/matzalazar/rhizome/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matzalazar%2Frhizome/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31456682,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-05T21:22:52.476Z","status":"online","status_checked_at":"2026-04-06T02:00:07.287Z","response_time":112,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["local-first","logseq","notes","obsidian","onnx","pkm","semantic-search","wikilinks","zettelkasten"],"created_at":"2026-04-06T02:13:18.377Z","updated_at":"2026-04-06T02:13:19.940Z","avatar_url":"https://github.com/matzalazar.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# rhizome\n\n\u003cdiv align=\"center\"\u003e\n  \u003cpicture\u003e\n    \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"assets/banner_dark.png\"\u003e\n    \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"assets/banner_light.png\"\u003e\n    \u003cimg alt=\"rhizome banner\" src=\"assets/banner_light.png\"\u003e\n  \u003c/picture\u003e\n\u003c/div\u003e\n\n[![CI](https://github.com/matzalazar/rhizome/actions/workflows/ci.yml/badge.svg)](https://github.com/matzalazar/rhizome/actions/workflows/ci.yml)\n[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue)](https://www.python.org/downloads/)\n[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n[![runs locally · no cloud](https://img.shields.io/badge/runs%20locally-no%20cloud-brightgreen)](#)\n\n\u003e Semantic backlinks for your notes — generated locally, stored as `[[wikilinks]]`.\n\nRhizome reads your vault, embeds every note with a multilingual sentence transformer, and writes a `## Related Notes` section at the bottom of each file.\nNo cloud API, no database, no daemon — the knowledge graph lives entirely in the filesystem and syncs with the rest of your vault.\n\n```\n## Related Notes\n\n- [[Zettelkasten]]\n- [[Evergreen notes]]\n- [[How I take notes]]\n```\n\n![Rhizome interface demo](assets/demo1.png)\n\n---\n\n## Features\n\n- **Fully local** — ONNX Runtime on CPU, no GPU or network call after first run\n- **Multilingual** — default model handles 50+ languages; swap to any Xenova ONNX export via `MODEL_NAME`\n- **Scales automatically** — exact numpy search for small vaults, approximate HNSW for large ones\n- **Long-document aware** — notes exceeding 512 tokens are split into overlapping chunks; chunk embeddings are averaged so every section of the note influences its semantic representation\n- **Idempotent** — re-running the pipeline replaces the section, never duplicates it\n- **Safe by default** — prompts for a timestamped vault backup before writing anything\n- **Dry-run mode** — preview every proposed link without touching a single file\n- **Extensible** — a four-method `VaultReader` Protocol is the only contract a new adapter needs\n\n---\n\n## Requirements\n\n- Python 3.10+\n- An Obsidian or Logseq vault\n\n---\n\n## Installation\n\n```bash\ngit clone https://github.com/matzalazar/rhizome\ncd rhizome\npip install -e .\n```\n\nFor development tools (pytest, ruff):\n\n```bash\npip install -e \".[dev]\"\n```\n\n---\n\n## Quick start\n\n```bash\n# 1. Copy the example config and fill in your vault path\ncp .env.example .env\n\n# 2. Check vault stats and model cache status\nrhizome status\n\n# 3. Run — the model downloads automatically on first use (~250 MB, once)\nrhizome run\n```\n\nOn first run, rhizome performs a dry-run preview and asks for confirmation:\n\n```\n  Notes to modify  : 47\n  Links to write   : 214\n  (A timestamped backup will be created before writing.)\n\n  Proceed? [Y/n]:\n```\n\nIf you confirm, you will also be asked whether to create a backup before writing.\n\nThe backup is written to `{vault}/../.rhizome_backups/backup_YYYYMMDD_HHMMSS/`\nand can be restored interactively with `rhizome restore`.\n\n---\n\n## Configuration\n\nAll settings are read from environment variables or a `.env` file.\n\n| Variable               | Default                                          | Description                                          |\n|------------------------|--------------------------------------------------|------------------------------------------------------|\n| `VAULT_PATH`           | *(required)*                                     | Absolute path to your vault directory                |\n| `VAULT_APP`            | `obsidian`                                       | Adapter to use: `obsidian` or `logseq`               |\n| `SIMILARITY_THRESHOLD` | `medium` (0.75)                                  | Minimum cosine similarity. Accepts a float in [0, 1] or a named level: `low` (0.60), `medium` (0.75), `high` (0.88) |\n| `TOP_K`                | `5`                                              | Maximum related notes to surface per note            |\n| `MODEL_DIR`            | `./models`                                       | Directory for the cached ONNX model files            |\n| `MODEL_NAME`           | `Xenova/paraphrase-multilingual-MiniLM-L12-v2`  | HuggingFace model identifier (Xenova ONNX exports). If you change this, clear `MODEL_DIR` first to avoid stale cache. |\n| `LOG_DIR`              | `./logs`                                         | Directory where per-run JSON logs are written        |\n| `DRY_RUN`              | `false`                                          | Preview proposed links without modifying any files   |\n| `MANUAL_OVERRIDE_FIELDS` | `top_k,similarity_threshold,chunk_size,chunk_overlap,related_notes_header` | Comma-separated list of runtime prompts to show in `rhizome run --manual`. Accepts `top_k`, `similarity_threshold`, `chunk_size`, `chunk_overlap`, `related_notes_header`, plus aliases `threshold` and `header`. |\n| `EXCLUDE_DIRS`         | *(empty)*                                        | Comma-separated list of directories (relative to `VAULT_PATH`) to skip. Uses prefix matching: `journal` excludes `vault/journal/` but not `vault/project/journal/`. |\n| `INCLUDE_DIRS`         | *(empty)*                                        | Comma-separated whitelist of directories to scan exclusively. When set, only files under these paths are processed. `EXCLUDE_DIRS` is applied afterwards, so you can narrow within the whitelist (e.g. `INCLUDE_DIRS=projects` + `EXCLUDE_DIRS=projects/drafts`). Leave empty to process the entire vault. |\n| `CHUNK_SIZE`           | `512`                                            | Maximum tokens per chunk when embedding long notes. Notes exceeding this limit are split into overlapping windows and their embeddings averaged into one vector. Set to `0` to disable chunking (notes truncated at 512 tokens). |\n| `CHUNK_OVERLAP`        | `32`                                             | Tokens shared between adjacent chunks. Preserves sentence context across chunk boundaries. Must be less than `CHUNK_SIZE`. Ignored when `CHUNK_SIZE=0`. |\n\n\u003e [!IMPORTANT]\n\u003e **`CHUNK_SIZE` directly affects embedding time.**\n\u003e Each chunk requires a full forward pass through the model.\n\u003e With the default (`512`), chunking only triggers for notes longer than ~400 words — typical PKM notes are unaffected.\n\u003e Lowering `CHUNK_SIZE` increases the number of chunks per long note and embedding time grows proportionally:\n\u003e a 2 000-token note with `CHUNK_SIZE=256` produces ~8 chunks instead of 4.\n\u003e Set `CHUNK_SIZE=0` to disable chunking entirely and restore the original truncation behaviour,\n\u003e at the cost of losing content beyond the first ~400 words of each note.\n\n---\n\n## Choosing a model\n\nThe default model works well for most vaults. If your vault is entirely in one\nlanguage, or you want a different speed/quality trade-off, you can switch to any\ncompatible model by setting `MODEL_NAME`.\n\n| Model | Size | Languages | Best for |\n|-------|------|-----------|----------|\n| `Xenova/paraphrase-multilingual-MiniLM-L12-v2` | ~250 MB | 50+ | **Default** — mixed-language or unknown vault |\n| `Xenova/all-MiniLM-L6-v2` | ~90 MB | English | Fast and lean: English-only vault, low-RAM hardware |\n| `Xenova/all-MiniLM-L12-v2` | ~130 MB | English | Balanced: English vault, better recall than L6 |\n| `Xenova/all-mpnet-base-v2` | ~430 MB | English | High precision: English vault where quality matters most |\n| `Xenova/paraphrase-multilingual-mpnet-base-v2` | ~1.1 GB | 50+ | Best multilingual quality, if disk space allows |\n\n\u003e **When you change `MODEL_NAME`, delete `MODEL_DIR` first.**\n\u003e Embeddings from different models are not comparable — mixing them produces\n\u003e meaningless similarity scores.\n\n```bash\n# Example: switch to a leaner English-only model\nMODEL_NAME=Xenova/all-MiniLM-L6-v2\nMODEL_DIR=./models   # delete contents of this directory first\n```\n\nSee [docs/models.md](docs/models.md) for a detailed comparison, a decision guide,\nand notes on compatibility requirements.\n\n---\n\n## CLI reference\n\n```\nrhizome run              Execute the full pipeline (dry-run preview + confirmation)\nrhizome run --yes        Skip confirmation and auto-confirm backup (CI / scripted)\nrhizome run --manual     Interactively choose one or more notes to update while matching against the full vault\nrhizome audit            Analyze vault connectivity without modifying any file\nrhizome status           Show vault stats and model cache status\nrhizome clean            Remove all generated ## Related Notes sections\nrhizome download-model   Pre-cache the ONNX model (useful for CI / Docker)\nrhizome backups          List available backups with metadata\nrhizome restore          Interactively restore a previous backup\n```\n\nEvery command accepts `--verbose` / `-v` to enable debug logging.\n\n### Log output format\n\nAll log output uses a symbol-based format:\n\n| Symbol | Level    |\n|--------|----------|\n| `[.]`  | debug    |\n| `[i]`  | info     |\n| `[-]`  | success  |\n| `[!]`  | warning  |\n| `[x]`  | error    |\n| `[!!]` | critical |\n\n### Verbose mode\n\n```bash\nrhizome run --verbose\n```\n\nPrints one debug line per modified note showing the note title and the links that were written:\n\n```\n05:33:45 [.] Zettelkasten → [[Evergreen notes]], [[How I take notes]], [[PKM]]\n05:33:45 [.] Atomic habits → [[Deep work]], [[Focus]], [[GTD]]\n```\n\n### Dry run\n\n`rhizome run` always shows a dry-run preview before writing.  Confirm at the\nprompt to proceed, or press `n` (or Ctrl-C) to abort without touching any file.\n\nFor fully non-interactive execution (CI / scripts):\n\n```bash\nrhizome run --yes        # skip all prompts, auto-confirm backup\n```\n\nTo update one or more notes interactively while still comparing them against the\nfull vault:\n\n```bash\nrhizome run --manual\n```\n\nRhizome will ask you to search by filename or path, add notes to a manual target\nlist, and optionally override the runtime settings configured in\n`MANUAL_OVERRIDE_FIELDS` for that run only. The `.env` file remains unchanged.\n\nFor example, if you usually only tweak the threshold:\n\n```bash\nMANUAL_OVERRIDE_FIELDS=similarity_threshold\n```\n\nTo preview proposed links without writing anything at all (no prompt):\n\n```bash\nDRY_RUN=true rhizome run\n```\n\n### Auditing the vault\n\n```bash\nrhizome audit\n```\n\nAnalyzes your vault and reports its connectivity state without modifying anything:\n\n```\n[i] Vault audit — /home/you/notes (312 notes)\nConnectivity distribution\n─────────────────────────\nNo connections       :  47 notes  ( 15%)\n1–2 connections      :  83 notes  ( 27%)\n3–5 connections      :  92 notes  ( 30%)\n6+  connections      :  90 notes  ( 29%)\nPotential new links  : 214  (dry-run to preview them)\nEst. notes affected  :  98\n[i] Run `rhizome run` to generate links.\n```\n\n\"Connections\" counts existing `[[wikilinks]]` in the `## Related Notes` section only.\n\"Potential new links\" runs the full embedding + similarity pipeline in memory — no files\nare written.\n\n### Pre-caching the model\n\n```bash\nrhizome download-model\n```\n\nDownloads and exports the model to `MODEL_DIR` without touching the vault.\nUseful as a dedicated step in a Docker build or CI pipeline so the model layer\nis cached separately from the application code.\n\n---\n\n## How it works\n\n- [docs/architecture.md](docs/architecture.md) — pipeline walkthrough: embedding strategy, similarity backends, section writing logic\n- [docs/models.md](docs/models.md) — model selection guide: compatibility, comparison table, decision criteria, switching instructions\n\n---\n\n## Backup and restore\n\nBefore modifying any files, `rhizome run` prompts to create a timestamped backup.\nBackups are stored at `{vault}/../.rhizome_backups/backup_YYYYMMDD_HHMMSS/` and\ninclude a manifest with vault path, timestamp, note count, and rhizome version.\n\n```bash\nrhizome backups    # list all backups\nrhizome restore    # select and restore interactively\n```\n\nRestore overwrites files present in the backup but leaves files created after the\nbackup untouched.\n\n---\n\n## Supported apps\n\n| App      | Status | Link format      |\n|----------|--------|------------------|\n| Obsidian | Stable | `[[wikilinks]]`  |\n| Logseq   | Stable | `[[wikilinks]]`  |\n\nBoth adapters generate the same `## Related Notes` section with `[[wikilinks]]`.\nLogseq renders markdown headers and bullet lists natively in both outline and\ndocument view, so the format works without any special configuration.\n\n---\n\n## Project structure\n\n```\nsrc/rhizome/\n├── config.py            pydantic-settings — validated at startup\n├── pipeline.py          orchestration (run / clean / status)\n├── main.py              CLI bootstrap (load_dotenv → delegate to cli/)\n│\n├── inference/\n│   ├── model.py         ONNX export, mean pooling, L2 normalisation\n│   └── similarity.py    SimilarityStrategy Protocol · Numpy · HNSW\n│\n├── vault/\n│   ├── base.py          Note dataclass · VaultReader Protocol\n│   ├── obsidian.py      Obsidian adapter + module-level helpers\n│   ├── logseq.py        Logseq adapter\n│   ├── backup.py        create / list / restore backups\n│   └── __init__.py      get_vault_reader() factory · public re-exports\n│\n└── cli/\n    └── commands.py      Typer app with all command definitions\n```\n\n---\n\n## Contributing\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for setup, testing, and how to add a new vault adapter.\n\n---\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatzalazar%2Frhizome","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmatzalazar%2Frhizome","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatzalazar%2Frhizome/lists"}