{"id":50528442,"url":"https://github.com/chris-santiago/ml-wiki","last_synced_at":"2026-06-03T10:30:39.657Z","repository":{"id":355835282,"uuid":"1229383952","full_name":"chris-santiago/ml-wiki","owner":"chris-santiago","description":"A self-contained research wiki powered by Claude Code. Ingests papers, PDFs, Zotero libraries, and markdown into interlinked wiki pages with semantic search, automated cross-references, and LLM-generated summaries.","archived":false,"fork":false,"pushed_at":"2026-05-05T12:22:49.000Z","size":174,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"public","last_synced_at":"2026-05-05T14:28:33.714Z","etag":null,"topics":["claude-code","knowledge-management","llm","openrouter","personal-knowledge-base","research-wiki","zotero"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chris-santiago.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-05T01:44:25.000Z","updated_at":"2026-05-05T12:22:53.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/chris-santiago/ml-wiki","commit_stats":null,"previous_names":["chris-santiago/ml-wiki"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/chris-santiago/ml-wiki","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chris-santiago%2Fml-wiki","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chris-santiago%2Fml-wiki/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chris-santiago%2Fml-wiki/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chris-santiago%2Fml-wiki/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chris-santiago","download_url":"https://codeload.github.com/chris-santiago/ml-wiki/tar.gz/refs/heads/public","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chris-santiago%2Fml-wiki/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33860962,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-03T02:00:06.370Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["claude-code","knowledge-management","llm","openrouter","personal-knowledge-base","research-wiki","zotero"],"created_at":"2026-06-03T10:30:38.819Z","updated_at":"2026-06-03T10:30:39.649Z","avatar_url":"https://github.com/chris-santiago.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ml-wiki\n\nA personal research wiki that ingests papers, Zotero libraries, PDFs, markdown notes, images, and ML experiment logs into a structured directory of interlinked markdown pages. It provides Map of Content (MOC) pages, semantic fragment search, contradiction detection, and staleness tracking -- all orchestrated through Claude Code slash commands.\n\nAfter setup, use `/ingest --sync` to populate from your sources, `/render --all-stubs` to generate wiki pages, and `/build` to create topic indexes and cross-references.\n\n---\n\n## How It Works\n\nml-wiki uses a three-layer architecture where each layer has a single responsibility:\n\n| Layer | Lives in | Responsibility |\n|---|---|---|\n| **CLI layer** (orchestration) | `.wiki/scripts/wiki_cli.py` | Python Click CLI that orchestrates everything. Each subcommand replaces a former SKILL.md procedure -- resolves config, calls scripts, dispatches LLM, validates output, and writes results. |\n| **LLM layer** (judgment) | `.wiki/scripts/wiki_llm.py` + `.wiki/agents/*.md` | OpenRouter API calls with JSON output. Agent prompts request structured JSON, validated by jsonschema schemas in `.wiki/schemas/` and domain-specific post-validators in `wiki_validate.py`. |\n| **Script layer** (mechanical) | `.wiki/scripts/wiki_index.py`, `wiki_render.py`, etc. | Does the work. Python scripts for all file I/O, index CRUD, source extraction, page assembly, and batch operations. No script calls another -- they communicate via stdin/stdout/files/exit codes. |\n\nSkills still exist as thin wrappers (10-line SKILL.md files that call `uv run .wiki/scripts/wiki_cli.py \u003ccommand\u003e`) for slash-command access inside Claude Code. Three skills remain native Claude Code operations: `howto`, `process`, and `sync-docs`.\n\n---\n\n## Prerequisites\n\n- **Claude Code** -- the [Anthropic CLI for Claude](https://docs.anthropic.com/en/docs/claude-code). Provides the slash-command interface and runs the 3 native skills (`/howto`, `/process`, `/sync-docs`).\n- **uv** -- the Python package manager. Scripts use [PEP 723 inline metadata](https://peps.python.org/pep-0723/), so there is no `requirements.txt` or `pyproject.toml`. Running `uv run .wiki/scripts/wiki_cli.py` resolves and installs dependencies automatically.\n- **OpenRouter API key** -- set the `OPENROUTER_API_KEY` environment variable. All LLM calls (render, build, query, lint, normalize-tags, idea improve) go through OpenRouter. The model tiers are configured in `.wiki/config.yaml`.\n- **Zotero + BetterBibTeX plugin** (for the primary ingest path) -- you need BetterBibTeX JSON exports (not Better CSL JSON, which uses a different schema). Each Zotero collection is exported as a `.json` file into `sources/`.\n- **Python 3.11+**\n\n---\n\n## Setup\n\n### 1. Clone and initialize\n\n```bash\ngit clone \u003crepo-url\u003e\ncd ml-wiki\n```\n\nRun `/wiki-init` inside Claude Code to interactively generate `.wiki/config.yaml`:\n\n```\n/wiki-init\n```\n\nIt asks for:\n- Zotero library names and BBT JSON export paths (you can have multiple)\n- Optional: ml-journal names and JSONL paths (for ML experiment logs)\n- Optional: Zotero storage path (for PDF extraction)\n\nThe command scaffolds all required directories (`wiki/_pages/`, `wiki/topics/`, `wiki/projects/`, `.wiki/scripts/`, `.wiki/sources/`, `.wiki/schemas/`), creates an empty index if one does not exist, and validates that configured paths are reachable.\n\n### 2. Configuration\n\nThe config file lives at `.wiki/config.yaml`:\n\n```yaml\nresearch_wiki:\n  wiki_dir: ./wiki\n  index_path: ./.wiki/index.jsonl\n  default_render_depth: shallow\n  llm:\n    base_url: https://openrouter.ai/api/v1\n    api_key_env: OPENROUTER_API_KEY\n    models:\n      nano: openai/gpt-5.4-nano      # Fast: render-shallow, query-p1, build-rank, normalize-tags, search-terms\n      mini: openai/gpt-5.4-mini      # Mid: render-deep, build-map, lint, idea-improve\n      full: openai/gpt-5.4           # Best: build-moc, query-p2\n    default_params:\n      max_tokens: 16384\n      temperature: 0\n    model_overrides: {}               # Per-agent overrides (e.g. \"render-deep\": \"anthropic/claude-sonnet-4\")\n  tag_blocklist: []                   # Tags excluded from normalize-tags proposals\n  tag_aliases:\n    # Cumulative map of non-canonical -\u003e canonical tags.\n    # Auto-managed by /normalize-tags. Do not hand-edit.\n    lstm-networks: lstm\n    nlp: natural-language-processing\n    # ... (1700+ entries)\n  sources:\n    - name: my-papers\n      type: zotero\n      bbt_export_path: ./sources/MyPapers.json\n      zotero_storage: /Users/you/Zotero/storage\n    - name: my-references\n      type: zotero\n      bbt_export_path: ./sources/MyReferences.json\n      zotero_storage: /Users/you/Zotero/storage\n    - name: my-experiments\n      type: ml-journal\n      journal_path: /path/to/.project-log/journal.jsonl\n```\n\n`wiki_config.py` is the PyYAML boundary. It reads `config.yaml` and emits JSON to stdout. All other scripts are config-unaware -- they receive values as CLI flags from `wiki_cli.py`. This keeps the YAML dependency isolated.\n\n### 3. Populate from sources\n\n```\n/ingest --sync\n```\n\nThis diffs all configured Zotero and ml-journal sources against the index and adds new entries as stubs. Then render them:\n\n```\n/render --all-stubs\n```\n\n---\n\n## Philosophy\n\n### Why a wiki, not RAG\n\nMost LLM-and-document workflows are retrieval-based: you upload files, the LLM retrieves relevant chunks at query time, and generates an answer from scratch. Nothing accumulates. Ask a subtle question that requires synthesizing five papers, and the LLM has to find and piece together the fragments every time.\n\nThis system takes a different approach. Instead of re-deriving knowledge on every query, the LLM incrementally builds a persistent wiki -- a structured, interlinked collection of markdown files that sits between you and the raw sources. When you add a new paper, the LLM reads it, extracts the key knowledge, and integrates it into the existing wiki. The cross-references are already there. The contradictions have already been flagged. The synthesis already reflects everything you have read.\n\n### The maintenance problem\n\nPersonal knowledge bases die not because people stop reading but because the bookkeeping becomes unbearable. Updating cross-references, keeping summaries current, noting when new evidence contradicts old claims, maintaining consistency across dozens of pages. The maintenance burden grows faster than the value.\n\nLLMs solve this. They do not get bored, do not forget to update a cross-reference, and can touch every file in the wiki in one pass. The cost of maintenance drops to near zero, so the wiki stays alive.\n\n### Separation of concerns\n\nEach tool in the stack does exactly one thing:\n\n- **Zotero** is the library -- it manages PDFs, citations, and bibliographic metadata\n- **ml-journal** is the experiment log -- it tracks what you tried and what happened\n- **The wiki** is the synthesis layer -- it captures *why*, connecting papers to claims to experiments to open questions\n- **Obsidian** is the viewer -- it renders the wiki for human browsing\n\nNone of these systems write to each other. The wiki reads from Zotero and ml-journal but never modifies them. Obsidian reads the wiki but never authors in it. Data flows in one direction: sources -\u003e index -\u003e wiki -\u003e viewer.\n\n### Two speeds\n\nThe system has a clear cost boundary. Operations that touch only the JSONL index -- ingesting, syncing, filtering, diffing -- are fast and free. Operations that invoke the LLM -- rendering, querying, narrative generation -- cost time and tokens. These are always decoupled so you are never blocked. You can ingest your entire Zotero library in seconds; you render papers on demand, when you are ready to engage with them.\n\n### Compounding knowledge\n\nQuery results feed back into the wiki. When you synthesize an answer across multiple papers and experiments, that synthesis is valuable -- it should not vanish into chat history. `/save` files it as a first-class wiki page. Your explorations compound in the knowledge base just like ingested sources do. Over time, the wiki becomes a layered artifact: raw summaries at the bottom, cross-references in the middle, and your evolving synthesis on top.\n\n---\n\n## Core Concepts\n\n### Entry types\n\n| Type | Prefix | Directory | Created by | Description |\n|---|---|---|---|---|\n| **paper** | (none) | `wiki/_pages/` | `/ingest` + `/render` | A research paper, imported from Zotero, PDF, or pasted text |\n| **experiment** | `exp-` | `wiki/experiments/` | `/ingest --sync` (ml-journal source) | An ML experiment record with hypothesis, setup, results |\n| **synthesis** | `synthesis-` | `wiki/syntheses/` | `/save` or `/ingest` (authored markdown) | Your own analysis, query answer, or blog post |\n| **image** | `img-` | `wiki/images/` | `/ingest` (image file) | A diagram or figure with vision-generated caption and description |\n| **idea** | `idea-` | `wiki/ideas/` | `/idea create` or `/idea ingest` | A forward-looking proposal, research plan, or methodology design |\n| **moc** | `topic-` or `project-` | `wiki/topics/` or `wiki/projects/` | `/build` | Map of Content -- navigational page grouping entries by tag or project |\n\n### Fragments\n\nFragments are the atomic units of knowledge in the wiki. Each one is a self-contained sentence or two that captures a single claim, method, finding, dataset, metric, question, or definition from a source.\n\n- **Created during `/render`**: The LLM reads the source material and extracts 2-4 fragments (shallow) or 6-12 fragments (deep). These are written to the index as `frag-\u003centry-id\u003e-\u003cseq\u003e` records.\n- **Power search**: `/query` runs keyword search across fragment titles and tags. Because each fragment title is a self-contained statement (e.g., \"Attention mechanisms outperform CNNs for sequential fraud detection on the Kaggle IEEE-CIS dataset\"), the search engine can match on meaning rather than just keywords.\n- **Power lint**: `/lint` detects contradictions by comparing `claim` and `finding` fragment titles across entries that share a tag.\n- **Not stable across re-renders**: When a page is re-rendered, old fragments are deleted and new ones are created. Reference the source entry ID, not fragment IDs.\n\nFragment types (closed list): `claim`, `method`, `finding`, `dataset`, `metric`, `question`, `definition`, `description`.\n\n### The JSONL index\n\nThe index at `.wiki/index.jsonl` is the single source of truth for what the wiki contains. It holds two kinds of records:\n\n\u003cdetails\u003e\n\u003csummary\u003eEntry record example\u003c/summary\u003e\n\n```json\n{\n  \"id\": \"fischerDeepLearning2024\",\n  \"type\": \"paper\",\n  \"source_type\": \"zotero\",\n  \"source_path\": \"/Users/you/Zotero/storage/.../fischer2024.pdf\",\n  \"wiki_path\": \"wiki/_pages/fischerDeepLearning2024.md\",\n  \"title\": \"Deep Learning for Fraud Detection\",\n  \"citation\": \"Fischer, A. (2024). Deep Learning...\",\n  \"year\": 2024,\n  \"tags\": [\"deep-learning\", \"fraud-detection\"],\n  \"project\": null,\n  \"source_name\": \"my-papers\",\n  \"references\": [],\n  \"status\": \"rendered\",\n  \"locked\": false,\n  \"ingested\": \"2026-05-02T05:07:54Z\",\n  \"rendered\": \"2026-05-02T05:21:37Z\",\n  \"last_improved\": null\n}\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eFragment record example\u003c/summary\u003e\n\n```json\n{\n  \"id\": \"frag-fischerDeepLearning2024-01\",\n  \"type\": \"method\",\n  \"title\": \"GRU with mean-pooled FastText embeddings for sequential fraud detection.\",\n  \"tags\": [\"deep-learning\", \"fraud-detection\", \"gru\"],\n  \"references\": [\"fischerDeepLearning2024\"],\n  \"ingested\": \"2026-05-02T05:21:31Z\"\n}\n```\n\n\u003c/details\u003e\n\nAll index mutations use atomic writes (write to `.tmp`, then `os.rename()`).\n\n### Page structure\n\nEvery rendered wiki page follows a consistent structure:\n\n```markdown\n---\nid: fischerDeepLearning2024\ntype: paper\nyear: 2024\nsource_name: my-papers\n---\n# Paper Title\n\n**Citation:** Fischer, A. (2024). Deep Learning...\n**PDF:** [Open PDF](zotero://open-pdf/library/items/ABC123)\n\n## Summary\n[LLM-generated content]\n\n## Key Claims\n[LLM-generated content]\n\n## Methods / Results / Limitations\n[LLM-generated content]\n\n## Notes\n\u003c!-- user-owned section -- preserved across re-renders --\u003e\nYour notes go here. This section survives /render and /render --force.\n\u003c!-- end-notes --\u003e\n\n## Connections\n\u003c!-- managed by /build -- do not edit manually --\u003e\n**References:** [[cited-entry-1]], [[cited-entry-2]]\n**Cited by:** [[topic-fraud-detection]]\n**Related:** [[related-entry-1]], [[related-entry-2]]\n```\n\nImportant boundaries:\n- Everything between `## Notes` and `\u003c!-- end-notes --\u003e` is yours. It survives all re-renders.\n- Everything from `## Connections` to EOF is managed by `/build`. Do not edit manually.\n- Staleness banners (marked with `\u003c!-- wiki-staleness-banner --\u003e`) are added/removed automatically by `/build`.\n\n### Locking\n\n`/lock \u003cid\u003e` prevents `/render` from overwriting a page -- use it when you have manually edited generated sections. `/build` still updates the `## Connections` section and staleness banners on locked pages. `/unlock \u003cid\u003e` reverses it. The `## Notes` section is always preserved regardless of lock state.\n\n### Staleness tracking\n\n`/build` detects stale entries -- syntheses whose source entries have been re-rendered since the synthesis was written, and ideas that have never been reviewed or whose shared-tag entries have been updated. Stale pages get a banner:\n\n- Syntheses: `\u003e **Stale synthesis** -- N new entries with shared tags since last render. Run /render \u003cid\u003e to refresh.`\n- Unreviewed ideas: `\u003e **Unreviewed idea** -- Run /idea improve \u003cid\u003e to generate an evidence review.`\n- Stale ideas: `\u003e **Stale idea** -- N new entries with shared tags since last improve. Run /idea improve \u003cid\u003e to refresh.`\n\nStale syntheses can be refreshed programmatically via the `synthesis-refresh` CLI command (see Scripts Reference), which reads the existing synthesis, gathers new evidence, and dispatches an LLM to integrate it.\n\n### MOC pages\n\nMap of Content pages are navigational pages that group entries by tag or project. They are generated by `/build` for any tag with 10+ entries (configurable via `--min-entries`), or any project with entries.\n\nA full MOC page (generated by `/build` without `--fast`) contains:\n- **Narrative**: A paragraph-level overview of the topic\n- **Subgroups**: How entries cluster within the topic\n- **Tensions**: Where papers disagree or offer competing approaches\n- **Evolution**: How the field has developed over time\n- **Listing**: A formatted list of every entry in the group\n\nA fast MOC page (generated by `/build --fast`) contains only the listing. `/build --fast` skips groups that already have a rendered narrative page, making it safe to run without clobbering existing LLM-generated content.\n\n`wiki/TOPICS.md` is a tiered alphabetical index: Major (10+ entries), Minor (3-9), Specialist (1-2). `wiki/MAP.md` (generated with `--map`) clusters all major topics into 8-12 thematic research domains.\n\n---\n\n## Canonical Workflows\n\n### Batch ingest (new Zotero papers)\n\nThe standard sequence after exporting a Zotero collection:\n\n```\n/ingest --sync              # Register new entries as stubs (fast, no LLM)\n/render --all-stubs         # Generate wiki pages for every stub (LLM)\n/normalize-tags             # Consolidate tag drift across the new batch\n/build                      # Reconcile backlinks, staleness, and MOC pages\n```\n\nEach step is independent and idempotent. `/ingest` has no LLM cost. `/normalize-tags` uses LLM calls for tag clustering and junk detection (mini + nano tiers). `/render` is the expensive step. `/build` runs after any render batch to sync the navigational layer.\n\n### Single paper\n\nFor a one-off addition:\n\n```\n/process fischerDeepLearning2024\n```\n\nThis runs ingest + render + partial backlink update in one step. Run `/build` afterwards if you want full MOC regeneration.\n\n### Query and save\n\n```\n/query \"What loss functions work best for heavily imbalanced fraud detection?\"\n/save\n```\n\nThe answer is saved as a synthesis page in `wiki/syntheses/` with extracted fragments. Or combine into one step:\n\n```\n/query \"What loss functions work best for heavily imbalanced fraud detection?\" --save\n```\n\n### Web-enriched query\n\n```\n/query \"What are recent advances in tabular deep learning?\" --web\n```\n\nAfter answering, `--web` searches arXiv for relevant papers not yet in the wiki, shows a selection panel, and ingests your picks. The query then re-runs against the enriched index.\n\n### Idea workflow\n\n```\n/idea create \"GRU-Mamba hybrid for fraud sequence modeling\"\n# ... fill in your content in the wiki page ...\n/idea improve idea-gru-mamba-hybrid\n```\n\nOr ingest an existing document and immediately critique it:\n\n```\n/idea ingest plans/my-research-plan.md --improve\n```\n\nThe improve pipeline searches the wiki (and optionally the web) for supporting and contradicting evidence, then splices a structured critique into the idea page.\n\n### Maintenance pass\n\nAfter a period of active ingestion:\n\n```\n/normalize-tags             # Resolve tag drift\n/build                      # Full rebuild: connections + MOCs + banners\n/lint                       # Structural + semantic audit\n```\n\n---\n\n## Commands\n\nAll commands are Claude Code slash commands. Run them in a Claude Code session. Most delegate to `wiki_cli.py` -- the CLI handles all orchestration, LLM calls, and output validation.\n\n### /ingest\n\nRegister a source in the JSONL index. Fast, no LLM cost.\n\n```\n/ingest fischerDeepLearning2024          # Zotero citekey\n/ingest paper.pdf                        # Local PDF\n/ingest notes.md                         # Markdown (asks: synthesis or reference?)\n/ingest screenshot.png                   # Image (vision pass for captioning)\n/ingest import/                          # Batch-ingest all files in a directory\n/ingest --sync                           # Diff all configured sources against index\n/ingest --sync --source my-papers      # Sync a single source\n```\n\nSource type detection is automatic: Zotero citekeys are matched against configured BBT exports, files are classified by extension, and ambiguous inputs get one clarifying question.\n\n\u003cdetails\u003e\n\u003csummary\u003eFlags\u003c/summary\u003e\n\n| Flag | Effect |\n|---|---|\n| `--text \"\u003ccontent\u003e\"` | Ingest pasted text directly (no file needed) |\n| `--id \u003cslug\u003e` | Explicit entry ID override |\n| `--title \"\u003ctitle\u003e\"` | Entry title |\n| `--tags \"t1,t2\"` | Comma-separated tags |\n| `--project \u003cname\u003e` | Project name |\n| `--source-kind reference\\|synthesis` | For markdown: skip the interactive question |\n| `--sync` | Diff all configured sources and bulk-add stubs |\n| `--source \u003cname\u003e` | With `--sync`: only sync this source |\n| `--caption \"\u003ctext\u003e\"` | Image caption (non-interactive) |\n| `--description \"\u003ctext\u003e\"` | Image description (non-interactive) |\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eSpecial cases\u003c/summary\u003e\n\n- **Markdown files**: You are asked whether it is an authored synthesis (your own writing) or a reference document. Syntheses get a `synthesis-` prefix and route to `wiki/syntheses/`. If the source has relative figure/image links, it is symlinked rather than copied so figure references resolve correctly.\n- **Images**: The file is copied to `wiki/assets/`, Claude Code's vision is used to generate a caption and description, and you confirm before the entry is created.\n- **Pasted text**: If the input is not a file path, the content is saved to `.wiki/sources/\u003cid\u003e.txt`.\n- **Directory ingest**: Each file is ingested individually with full interactive flow. Successfully ingested files are removed from the directory; failed or skipped files are left in place.\n- **Sync**: Diffs all configured Zotero sources (via `zotero_reader.py diff`) and ml-journal sources (via `wiki_render.py list-experiments`) against the index and bulk-adds new stubs.\n\n\u003c/details\u003e\n\n---\n\n### /render\n\nGenerate a markdown wiki page from source material. This is where LLM cost lives.\n\n```\n/render fischerDeepLearning2024                        # Shallow render (default)\n/render fischerDeepLearning2024 --depth deep           # Comprehensive extraction\n/render fischerDeepLearning2024 --focus \"fraud detection\"  # Steer extraction emphasis\n/render fischerDeepLearning2024 --force                # Render even if locked\n/render --all-stubs                                    # Batch render every stub\n/render --tag fraud-detection                          # Batch render stubs with a tag\n/render --all-stubs --concurrency 5                    # Limit parallel API calls\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eRender depths\u003c/summary\u003e\n\n| Depth | Model tier | Output |\n|---|---|---|\n| **shallow** (default) | `nano` | 3-5 sentence summary, 3-4 key claims, 2-4 fragments |\n| **deep** | `mini` | Detailed methods/results, 6-12 fragments across all types |\n\n\u003c/details\u003e\n\nBatch mode (`--all-stubs` / `--tag`) dispatches LLM calls concurrently. Use `--concurrency N` to control parallelism (default 10).\n\nThe `## Notes` section is always preserved across re-renders, even with `--force`. Idea entries are rejected -- they use `/idea improve` instead.\n\n---\n\n### /process\n\nConvenience wrapper: ingest + render + partial backlink update in one step.\n\n```\n/process fischerDeepLearning2024\n/process fischerDeepLearning2024 --depth deep --focus \"transformers\"\n```\n\nUse for one-off additions. For bulk operations, use `/ingest` and `/render` separately, then `/build`. Does not regenerate MOC pages.\n\n---\n\n### /build\n\nReconcile the entire wiki: backlinks, staleness banners, and MOC pages.\n\n```\n/build                              # semantic ranking + full LLM narratives\n/build --rank fast --fast           # fastest possible, no LLM, no model download\n/build --rank deep                  # best link quality, LLM-ranked\n/build --fast --map                 # listing-only MOCs + thematic map\n/build --force                      # rebuild all MOC groups, even up-to-date ones\n/build --min-entries 5              # lower the topic page threshold (default: 10)\n/build --only topic-fraud-detection # rebuild a single MOC page (skips ranking/staleness)\n/build --only topic-fraud-detection --only topic-deep-learning  # multiple targets\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eRanking modes (--rank)\u003c/summary\u003e\n\n| Flag | Behavior | Cost |\n|---|---|---|\n| `--rank fast` | Top-8 by shared tag count. Stdlib only, instant. | None |\n| `--rank semantic` (default) | MiniLM cosine similarity via `wiki_embed.py`. ~80s one-time model download (~80MB), then ~80s encode/score. | No LLM calls |\n| `--rank deep` | Parallel LLM calls reading entry content. Most accurate, highest cost and latency. | LLM |\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eMOC generation modes\u003c/summary\u003e\n\n| Flag | Behavior |\n|---|---|\n| (no flag) | LLM-generated narrative pages (Narrative, Subgroups, Tensions, Evolution sections). Only dirty groups (membership changed or entries re-rendered) are rebuilt unless `--force` is passed. |\n| `--fast` | Listing-only pages, no LLM calls. Skips groups with existing narrative pages unless `--force` is also passed. |\n| `--map` | Additionally generates `wiki/MAP.md` thematic landscape overview (one extra LLM call). |\n| `--min-entries N` | Minimum entries for a tag to get a topic page (default: 10). Lower it to generate MOC pages for smaller clusters. |\n| `--only MOC_ID ...` | Rebuild only the named MOC pages (e.g. `topic-fraud-detection`). Skips connection ranking, staleness checks, and topic index generation. Repeatable -- pass multiple `--only` flags for multiple targets. |\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eFull build pipeline\u003c/summary\u003e\n\n1. Resolve the full connection map (shared tags, shared fragments, backlinks, journal refs)\n2. Rank related links for each entry using the selected ranking mode\n3. Write `## Connections` sections on every page (including locked pages)\n4. Detect stale synthesis and idea entries and add/remove staleness banners\n5. Load tag aliases from config and group entries by canonical tag\n6. Filter to dirty MOC groups (unless `--force`)\n7. Generate MOC pages (full narratives or listings depending on `--fast`)\n8. Write `wiki/TOPICS.md` (tiered alphabetical index)\n9. Optionally write `wiki/MAP.md` (`--map`)\n\n\u003c/details\u003e\n\n---\n\n### /query\n\nAnswer a research question from the wiki's knowledge base.\n\n```\n/query \"What loss functions work best for heavily imbalanced fraud detection?\"\n/query \"How do state-space models compare to transformers?\" --save\n/query \"What are recent advances in tabular deep learning?\" --web\n/query \"What are recent advances in tabular deep learning?\" --web --auto\n/query \"What are recent advances in tabular deep learning?\" --web --auto --no-requery\n/query \"What are recent advances in tabular deep learning?\" --web --select \"1,3,5\"\n```\n\n| Flag | Effect |\n|---|---|\n| `--save` | File the final answer as a synthesis wiki page |\n| `--web` | After answering, search arXiv for relevant uningested papers and offer to ingest them, then re-query |\n| `--auto` | With `--web`: ingest all suggestions without prompting |\n| `--no-requery` | With `--web`: ingest without re-running the query |\n| `--select \"1,3\"` | With `--web`: non-interactive paper selection by index |\n\n\u003cdetails\u003e\n\u003csummary\u003eQuery pipeline\u003c/summary\u003e\n\n1. **Pass 1** (`nano` tier): Keyword search across fragments, generate semantic rephrasings, evaluate routing rubric. Either answers directly from fragments or identifies which full pages need reading.\n2. **Pass 2** (`full` tier, only if needed): Reads full wiki pages and synthesizes a comprehensive answer with inline citations. Contradictions are surfaced explicitly -- conflicting claims are called out, not blended.\n\nThe result is saved to `.wiki/last-query-result.md` as JSON for follow-up with `/save` (which reads it via `json.load`).\n\nWith `--web`, after the answer is displayed, a panel shows arXiv papers not yet in the wiki. You select which to ingest (or `--auto` ingests all, or `--select` picks by index). Each selected paper is downloaded, ingested, and rendered via `/web-ingest` + `/render`. Then the query re-runs against the enriched index (unless `--no-requery`). Web-acquired entries become first-class wiki entries -- they are never injected directly into synthesis.\n\n\u003c/details\u003e\n\n---\n\n### /save\n\nFile the last query result as a synthesis wiki page.\n\n```\n/save\n/save --id synthesis-loss-functions-imbalanced-fraud\n```\n\nCreates a synthesis page in `wiki/syntheses/` with fragments extracted from the answer. The `synthesis-` prefix is added automatically if omitted.\n\n---\n\n### /idea\n\nCreate, ingest, and improve wiki idea entries. Ideas are forward-looking proposals that can be iteratively improved with evidence-based critique.\n\n```\n/idea create \"GRU-Mamba hybrid for fraud modeling\"\n/idea create \"GRU-Mamba hybrid\" --tags \"gru,mamba\" --project ato\n/idea ingest plans/research-plan.md\n/idea ingest plans/research-plan.md --improve\n/idea ingest plans/research-plan.md --improve --local\n/idea improve idea-gru-mamba-hybrid\n/idea improve idea-gru-mamba-hybrid --local\n```\n\n| Subcommand | Effect |\n|---|---|\n| `create \"\u003ctitle\u003e\"` | Create a new empty idea page at `wiki/ideas/idea-\u003cslug\u003e.md` |\n| `ingest \u003cpath\u003e` | Create an idea from an existing markdown file or pasted text |\n| `ingest \u003cpath\u003e --improve` | Ingest and immediately run the improve pipeline |\n| `improve \u003cid\u003e` | Web search + wiki evidence + structured critique (default) |\n| `improve \u003cid\u003e --local` | Wiki-only evidence, skip web search |\n\n\u003cdetails\u003e\n\u003csummary\u003eImprove pipeline\u003c/summary\u003e\n\n1. Extract search terms from the idea text\n2. Fragment-aware search across the wiki index (`score-idea`)\n3. Web evidence pass: search arXiv, ingest top results (unless `--local`)\n4. Dispatch `idea-improve` agent with idea text + evidence\n5. Splice structured critique (Supporting, Contradicting, Gaps, Suggestions) into the idea page\n6. Update index with merged tags, references, and fragments\n\nThe improve pipeline preserves the user's original idea text and appends the evidence review below it.\n\n\u003c/details\u003e\n\n---\n\n### /web-ingest\n\nFetch an arXiv paper by ID or URL, download the PDF, and add a stub entry.\n\n```\n/web-ingest 2312.00752\n/web-ingest https://arxiv.org/abs/2312.00752\n/web-ingest arXiv:2312.00752\n/web-ingest 2312.00752 --yes                   # Skip confirmation\n/web-ingest 2312.00752 --dry-run               # Fetch metadata only\n```\n\n| Flag | Effect |\n|---|---|\n| `--yes` / `-y` | Skip the confirmation prompt |\n| `--dry-run` | Fetch metadata only, do not download or add |\n| `--tags \"t1,t2\"` | Comma-separated tags to apply |\n| `--project \u003cname\u003e` | Project name to apply |\n\narXiv only -- no other web sources. Shows a confirmation screen with title/authors/abstract before downloading. Run `/render \u003cid\u003e` afterwards to generate the wiki page.\n\n---\n\n### /normalize-tags\n\nScan all tags, identify duplicates and synonyms, propose a canonical mapping, and apply it.\n\n```\n/normalize-tags\n/normalize-tags --dry-run\n/normalize-tags --apply /path/to/aliases.json\n/normalize-tags --promote 5                      # promote aliased tags with 5+ entries\n/normalize-tags --promote 5 --dry-run            # preview promotions\n```\n\n| Flag | Effect |\n|---|---|\n| `--dry-run` | Show proposals without applying. Prints the final aliases file path for later use with `--apply`. |\n| `--apply \u003cfile\u003e` | Apply a previously generated alias map from a JSON file (skip LLM). |\n| `--promote N` | Promote aliased tags with N+ entries to independent canonicals (removes them from the alias map). Combinable with `--dry-run` to preview. |\n\nRun after batch ingestion when tag drift accumulates. The pipeline uses a multi-stage approach: MiniLM cosine similarity clusters similar tags, then LLM calls confirm ambiguous merges (`mini` tier for clustering confirmation, `nano` tier for junk detection) and match unresolved tags to canonical forms. Singleton-to-singleton mappings are automatically blocked.\n\nThe alias map is cumulative -- each run adds to the existing map in `config.yaml`. Previously resolved tags are never re-evaluated. Aliases are loaded by `/build` during MOC grouping, so running `/normalize-tags` before `/build` ensures synonym consolidation flows into MOC pages.\n\n---\n\n### /lint\n\nAudit the wiki for structural problems and semantic inconsistencies.\n\n```\n/lint                       # Full: structural + contradiction detection + cross-link suggestions\n/lint --mechanical-only     # Structural checks only (no LLM cost)\n/lint --fix                 # Structural checks + auto-repair orphan fragments and broken paths\n/lint --concurrency 5       # Limit parallel API calls for contradiction checks\n/lint --min-frags 10        # Raise minimum fragments per tag for contradiction check (default 5)\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eStructural checks\u003c/summary\u003e\n\n- Index entries with no corresponding wiki file\n- Wiki files with no corresponding index entry\n- Orphan fragments (source entry not in index)\n- Entries with `status: rendered` but no `wiki_path`\n- Entries with `wiki_path` pointing to missing files\n- Invalid `type`, `source_type`, or `status` values\n- Tags that are not lowercase-slugified\n- MOC entries that have fragments\n- Duplicate entry IDs\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eSemantic checks (LLM-powered)\u003c/summary\u003e\n\n- **Contradiction detection**: Compares `claim` and `finding` fragment titles across entries that share a tag. Flags direct contradictions, scope conflicts, and temporal contradictions.\n- **Cross-link suggestions**: Identifies orphan entries (no inbound references) that should be linked to related entries.\n\n\u003c/details\u003e\n\n---\n\n### /lock and /unlock\n\n```\n/lock fischerDeepLearning2024       # Prevent /render from overwriting\n/unlock fischerDeepLearning2024     # Allow /render to overwrite again\n```\n\nLocking prevents `/render` from overwriting generated content. `/build` still updates `## Connections` and staleness banners. `## Notes` is always preserved regardless.\n\n---\n\n### /wiki-init\n\nInitialize or reconfigure the wiki.\n\n```\n/wiki-init\n/wiki-init --non-interactive\n```\n\nRun once during first-time setup, when adding sources, or after a fresh clone. Interactively generates `.wiki/config.yaml` (including the `llm` section with OpenRouter defaults), scaffolds directories, and creates an empty index. If `config.yaml` already exists, shows the current config and asks before overwriting. Use `--non-interactive` to create a minimal config without prompts.\n\n---\n\n### /sync-docs\n\nReconcile project documentation with current implementation.\n\n```\n/sync-docs              # Both README and design docs\n/sync-docs --readme     # README only\n/sync-docs --design     # Design docs only\n```\n\nDispatches the readme-agent (for `README.md`) and/or the sync-docs-agent (for `design/` docs and `COMMANDS.md`).\n\n---\n\n### /publish\n\nRebuild and force-push the `public` branch, stripping all personal content (wiki pages, index, sources, design docs) while preserving the framework (scripts, agents, schemas, skills, config template).\n\n```\n/publish              # Rebuild and push\n/publish --dry-run    # Show what would be removed/rewritten without pushing\n```\n\nWorks entirely in a temporary clone so the real working tree is never touched. Confirms before force-pushing.\n\n---\n\n## How To\n\n### Adding Content\n\n\u003cdetails\u003e\n\u003csummary\u003eHow do I add a paper from Zotero?\u003c/summary\u003e\n\nExport your Zotero collection as BetterBibTeX JSON into `sources/`. Then:\n\n```\n/ingest --sync --source my-library\n/render \u003ccitekey\u003e\n```\n\nOr sync all sources at once with `/ingest --sync`, then batch render with `/render --all-stubs`.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eHow do I add a paper from arXiv?\u003c/summary\u003e\n\n```\n/web-ingest 2312.00752\n/render \u003cid\u003e\n```\n\nOr use the full URL: `/web-ingest https://arxiv.org/abs/2312.00752`. The command fetches metadata, shows a confirmation screen, and downloads the PDF. You can also add tags and a project during confirmation.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eHow do I add my own writing (blog post, analysis, notes)?\u003c/summary\u003e\n\n```\n/ingest my-analysis.md\n```\n\nWhen asked, choose \"authored synthesis.\" The file is placed in `wiki/syntheses/` with a `synthesis-` prefix and is automatically locked so `/render` will not overwrite it. If your markdown has relative image links, the file is symlinked (not copied) so figure references resolve correctly.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eHow do I add an image or diagram?\u003c/summary\u003e\n\n```\n/ingest screenshot.png\n```\n\nThe image is copied to `wiki/assets/`, Claude Code's vision generates a caption and description, and you confirm before the entry is created. The resulting page at `wiki/images/img-\u003cslug\u003e.md` embeds the image with its description.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eHow do I ingest a batch of loose files?\u003c/summary\u003e\n\n```bash\nmkdir -p import\ncp ~/Downloads/*.pdf import/\n```\n\n```\n/ingest import/\n```\n\nEach file is ingested individually with full interactive flow. Successfully ingested files are removed from the directory. Failed or skipped files are left in place.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eHow do I record a research idea?\u003c/summary\u003e\n\n```\n/idea create \"My research idea title\" --tags \"tag-a,tag-b\"\n```\n\nFill in the content at `wiki/ideas/idea-\u003cslug\u003e.md`, then run `/idea improve idea-\u003cslug\u003e` to get an evidence-based critique. Or ingest an existing document: `/idea ingest plans/my-plan.md --improve`.\n\n\u003c/details\u003e\n\n### Querying \u0026 Synthesis\n\n\u003cdetails\u003e\n\u003csummary\u003eHow do I ask the wiki a question?\u003c/summary\u003e\n\n```\n/query \"What loss functions work best for imbalanced fraud detection?\"\n```\n\nThe answer draws from fragment search results. If fragment data is insufficient, full wiki pages are read and synthesized. Contradictions between sources are surfaced explicitly.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eHow do I save a query answer as a wiki page?\u003c/summary\u003e\n\n```\n/query \"My question\" --save\n```\n\nOr run `/save` after viewing the answer. The result becomes a synthesis page in `wiki/syntheses/` with its own fragments.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eHow do I discover papers I might be missing?\u003c/summary\u003e\n\n```\n/query \"My question\" --web\n```\n\nAfter the answer, a panel shows arXiv papers on the topic that are not in the wiki. Select which to ingest, and the query re-runs against the enriched index.\n\nFor fully automatic discovery and re-query:\n\n```\n/query \"My question\" --web --auto\n```\n\n\u003c/details\u003e\n\n### Maintenance \u0026 Housekeeping\n\n\u003cdetails\u003e\n\u003csummary\u003eHow do I run a full maintenance pass?\u003c/summary\u003e\n\n```\n/normalize-tags\n/build\n/lint\n```\n\n`/normalize-tags` resolves tag drift. `/build` reconciles connections, staleness, and MOC pages. `/lint` audits for structural and semantic problems.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eHow do I re-render a page with better quality?\u003c/summary\u003e\n\n```\n/render \u003cid\u003e --depth deep\n```\n\nDeep renders use the `mini` model tier and produce 6-12 fragments with detailed methods and results. Your `## Notes` section is preserved.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eHow do I protect a page I have manually edited?\u003c/summary\u003e\n\n```\n/lock \u003cid\u003e\n```\n\n`/render` will refuse to overwrite it (unless `--force` is used). `/build` still updates connections and banners.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eHow do I fix structural problems in the wiki?\u003c/summary\u003e\n\n```\n/lint --fix\n```\n\nThis auto-repairs orphan fragments and broken `wiki_path` references. Orphan wiki files (no index entry) are reported for manual review but not deleted.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eWhat does a staleness banner mean?\u003c/summary\u003e\n\nIt means source entries referenced by a synthesis or idea have been re-rendered since the synthesis/idea was last written. The information may be outdated. Run `/render \u003cid\u003e` (for syntheses) or `/idea improve \u003cid\u003e` (for ideas) to refresh. For syntheses, you can also use the `synthesis-refresh` CLI command to programmatically integrate new evidence.\n\n\u003c/details\u003e\n\n### Customization \u0026 Extension\n\n\u003cdetails\u003e\n\u003csummary\u003eHow do I add a new Zotero source?\u003c/summary\u003e\n\nRun `/wiki-init` to reconfigure, or edit `.wiki/config.yaml` directly:\n\n```yaml\nsources:\n  - name: my-new-library\n    type: zotero\n    bbt_export_path: ./sources/MyLibrary.json\n    zotero_storage: /Users/you/Zotero/storage\n```\n\nThen `/ingest --sync --source my-new-library`.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eHow do I add ML experiment logs?\u003c/summary\u003e\n\nAdd an ml-journal source to `.wiki/config.yaml`:\n\n```yaml\nsources:\n  - name: my-experiments\n    type: ml-journal\n    journal_path: /path/to/.project-log/journal.jsonl\n```\n\nThen `/ingest --sync --source my-experiments`. Experiment entries get `exp-` prefixes and route to `wiki/experiments/`.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eHow do I add a new entry type?\u003c/summary\u003e\n\n1. Add the type to `VALID_TYPES` in `wiki_index.py`\n2. Add a template in `wiki_render.py`'s `REQUIRED_BLOCKS` dict\n3. Add a corresponding agent prompt in `.wiki/agents/render-*.md`\n4. Add a JSON schema in `.wiki/schemas/render-\u003ctype\u003e.json`\n5. Add path routing in the `wiki_path()` function if the new type needs its own directory\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eHow do I add a new source type?\u003c/summary\u003e\n\n1. Add the source to `.wiki/config.yaml` under `sources`\n2. If it needs custom parsing, add a reader script in `.wiki/scripts/` with PEP 723 inline metadata\n3. Update `wiki_cli.py`'s `ingest` command to handle the new source type in its detection logic\n4. Add a `source_type` value to `VALID_SOURCE_TYPES` in `wiki_index.py`\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eHow do I change the LLM models?\u003c/summary\u003e\n\nEdit the `llm.models` section in `.wiki/config.yaml`:\n\n```yaml\nllm:\n  models:\n    nano: openai/gpt-5.4-nano\n    mini: openai/gpt-5.4-mini\n    full: openai/gpt-5.4\n```\n\nOr override a specific agent without changing the tier default:\n\n```yaml\nllm:\n  model_overrides:\n    render-deep: anthropic/claude-sonnet-4\n    query-p2: anthropic/claude-opus-4\n```\n\nAny model available on OpenRouter can be used. The `api_key_env` field controls which environment variable holds the API key.\n\n\u003c/details\u003e\n\n---\n\n## Directory Structure\n\n```\nwiki/\n  _pages/               # Paper pages (Zotero, PDF, pasted text)\n  experiments/          # ML experiment pages (from ml-journal)\n  ideas/                # Idea pages (proposals, research plans)\n  topics/               # topic-\u003ctag\u003e.md MOC pages\n  projects/             # project-\u003cname\u003e.md MOC pages\n  syntheses/            # synthesis-\u003cslug\u003e.md authored/query pages\n  images/               # img-\u003cslug\u003e.md image pages\n  sources/              # arXiv PDFs downloaded by /web-ingest\n  assets/               # Original image files (png, jpg, etc.)\n  TOPICS.md             # Tiered alphabetical index of all topic MOCs\n  MAP.md                # Thematic landscape overview (8-12 research domains)\n\n.wiki/\n  scripts/              # Python scripts (the mechanical + orchestration layers)\n    wiki_cli.py         # Click CLI orchestrator — replaces SKILL.md procedures\n    wiki_llm.py         # OpenRouter LLM client (sync + async batch), JSON extraction, schema validation\n    wiki_validate.py    # Domain-specific post-validators (tag/fragment count enforcement, prose coercion)\n    wiki_util.py        # Pure utilities (slugify, source-type detection, citation extraction, tag normalization)\n    wiki_build.py       # Build helpers (listing construction, banner generation, candidate pre-filtering)\n    wiki_idea.py        # Idea helpers (page generation, tag merging)\n    wiki_index.py       # JSONL CRUD engine, path routing, search, fragments, lint\n    wiki_render.py      # Source extraction, page assembly, batch operations\n    wiki_config.py      # YAML -\u003e JSON config bridge (PyYAML boundary)\n    wiki_embed.py       # Sentence-transformer embedding, cosine scoring, tag clustering, canonical matching\n    arxiv_fetch.py      # arXiv API client (normalize, fetch, download-pdf)\n    zotero_reader.py    # BetterBibTeX JSON parser\n    COMMANDS.md         # Full subcommand reference, organized by workflow stage\n    tests/              # Test suite (294 tests, run via uv run --group test pytest)\n  agents/               # LLM agent prompt templates (JSON output)\n    render-shallow.md   # Fast structured summary (nano tier)\n    render-deep.md      # Comprehensive extraction (mini tier)\n    build-moc.md        # MOC narrative generation (full tier)\n    build-rank.md       # Related link ranking (nano tier)\n    build-map.md        # Thematic map generation (mini tier)\n    query-p1.md         # Query narrowing pass (nano tier)\n    query-p2.md         # Query synthesis pass (full tier)\n    lint.md             # Contradiction detection + cross-link suggestions (mini tier)\n    normalize-tags.md   # Tag deduplication and synonym resolution (nano tier)\n    normalize-tags-cluster.md  # Tag cluster confirmation (mini tier)\n    normalize-tags-junk.md     # Junk tag detection (nano tier)\n    normalize-tags-assign.md   # Orphan tag assignment (mini tier)\n    idea-improve.md     # Idea critique with evidence review (mini tier)\n    search-terms.md     # Search term extraction (nano tier)\n    synthesis-refresh.md  # Synthesis refresh with new evidence (full tier)\n  schemas/              # JSON schemas for LLM output validation\n    render-paper.json\n    render-experiment.json\n    render-synthesis.json\n    build-moc.json\n    build-rank.json\n    build-map.json\n    query-p1.json\n    query-p2.json\n    lint-contradictions.json\n    lint-crosslinks.json\n    normalize-tags.json\n    normalize-tags-cluster.json\n    normalize-tags-junk.json\n    normalize-tags-assign.json\n    idea-improve.json\n    search-terms.json\n    synthesis-refresh.json\n  sources/              # Pasted text storage (git-tracked)\n  config.yaml           # Wiki configuration (includes LLM model tiers)\n  index.jsonl           # JSONL index (entries + fragments)\n\n.claude/\n  agents/               # Claude Code agent prompts (for native skills only)\n    readme-agent.md     # README generation (dispatched by /sync-docs)\n    sync-docs-agent.md  # Design doc reconciliation (dispatched by /sync-docs)\n  skills/               # Skill definitions (thin wrappers calling wiki_cli.py)\n    ingest/SKILL.md\n    render/SKILL.md\n    process/SKILL.md    # Native Claude Code skill (reads flow reference files)\n    build/SKILL.md\n    query/SKILL.md\n    save/SKILL.md\n    idea/SKILL.md\n    web-ingest/SKILL.md\n    normalize-tags/SKILL.md\n    lint/SKILL.md\n    lock/SKILL.md\n    unlock/SKILL.md\n    wiki-init/SKILL.md\n    publish/SKILL.md    # Rebuild and push public branch\n    sync-docs/SKILL.md  # Native Claude Code skill (dispatches agents)\n    howto/SKILL.md      # Native Claude Code skill (answers how-to questions)\n\nscripts/\n  publish_public.py     # Public branch builder (temp clone, strip personal content)\n\nsources/                # Zotero BetterBibTeX JSON exports (read-only input)\n\npyproject.toml          # Test dependency group and pytest config (scripts use PEP 723 inline metadata, not this)\n```\n\n---\n\n## Scripts Reference\n\nAll scripts use PEP 723 inline metadata and are run via `uv run`. The CLI layer (`wiki_cli.py`) imports `wiki_util`, `wiki_build`, `wiki_idea`, `wiki_validate`, `wiki_llm`, and `wiki_render` -- these are the only cross-script imports in the system. All other scripts communicate via stdin/stdout/files/exit codes.\n\n| Script | Purpose | Dependencies |\n|---|---|---|\n| `wiki_cli.py` | Click CLI orchestrator. Each subcommand replaces a former SKILL.md procedure. | click, httpx, jsonschema, openai |\n| `wiki_llm.py` | OpenRouter LLM client: sync calls, async batch dispatch, JSON extraction, schema validation, retry logic. | jsonschema, openai |\n| `wiki_validate.py` | Domain-specific post-validators: tag/fragment count enforcement, prose field coercion, deduplication. | stdlib only |\n| `wiki_util.py` | Pure utilities: slug generation, source-type detection, citation extraction, tag normalization. | stdlib only |\n| `wiki_build.py` | Build helpers: listing block construction, staleness banner generation, candidate pre-filtering. | stdlib only |\n| `wiki_idea.py` | Idea helpers: page generation, tag merging. | stdlib only |\n| `wiki_index.py` | JSONL CRUD, path routing, search, fragment management, connection resolution, MOC prep, staleness checks, structural lint, idea scoring | stdlib only |\n| `wiki_render.py` | Source extraction (PDF/text/markdown/ml-journal), page assembly from JSON output, batch operations, connection/banner updates, idea splicing, experiment listing | pymupdf |\n| `wiki_config.py` | Reads `config.yaml`, emits JSON to stdout. The single PyYAML boundary. Tag alias merge and persistence. | pyyaml |\n| `wiki_embed.py` | Sentence-transformer embedding, cosine similarity scoring, tag clustering, and canonical tag matching. Used by `--rank semantic`, `/idea improve` evidence ranking, and `/normalize-tags` tag deduplication. Downloads `all-MiniLM-L6-v2` (~80MB) on first run. | sentence-transformers |\n| `arxiv_fetch.py` | arXiv API client: normalize IDs, fetch metadata as index-ready JSON, download PDFs. The only script with network access (besides `wiki_llm.py`). | stdlib only |\n| `zotero_reader.py` | Parse BetterBibTeX JSON exports. Boolean citekey lookup, single-entry parse, diff against index. | stdlib only |\n\n\u003cdetails\u003e\n\u003csummary\u003ewiki_cli.py subcommands\u003c/summary\u003e\n\n| Subcommand | Description |\n|---|---|\n| `lock` | Lock an entry |\n| `unlock` | Unlock an entry |\n| `save` | File query result as synthesis page |\n| `web-ingest` | Fetch arXiv paper, download PDF, add stub |\n| `ingest` | Register source (single file, text, directory, or sync) |\n| `init` | Scaffold directories, generate config |\n| `render` | Single or batch render with LLM |\n| `query` | Two-pass research question answering |\n| `build` | Full wiki reconciliation pipeline |\n| `lint` | Structural + semantic audit |\n| `normalize-tags` | Tag drift resolution with LLM proposals |\n| `idea create` | Create empty idea page |\n| `idea ingest` | Create idea from file/text |\n| `idea improve` | Evidence-based critique pipeline |\n| `synthesis-refresh` | Refresh a stale synthesis with new evidence (full tier) |\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003ewiki_index.py subcommands\u003c/summary\u003e\n\n| Category | Subcommand | Description |\n|---|---|---|\n| **Index CRUD** | `create-entry` | Build entry JSON from CLI flags (stdout) |\n| | `add` | Add entry from stdin (exit 2 on collision) |\n| | `get` | Fetch entry by ID |\n| | `update` | Set fields on an existing entry |\n| | `delete` | Remove entry from index |\n| | `exists` | Check presence (exit 0=found, 1=not found, 2=collision) |\n| | `list` | Filter by status/type/tag/project |\n| | `add-batch` | Bulk-add from JSON array on stdin |\n| | `update-batch` | Bulk-update by filter or input file |\n| | `wiki-path` | Resolve wiki file path from ID (pure function, no I/O) |\n| **Fragments** | `add-fragments-batch` | Bulk-add fragments from stdin |\n| | `delete-fragments` | Remove all fragments for a source ID |\n| | `delete-fragments-batch` | Remove fragments by ID list from stdin |\n| **Build prep** | `resolve-connections` | Compute backlinks and related candidates |\n| | `prep-rank-batches` | Partition connection map for LLM ranking |\n| | `fast-rank-connections` | Rank candidates by shared-tag count (no LLM) |\n| | `prep-moc-batches` | Prepare per-group data for MOC narrative gen |\n| | `filter-dirty-mocs` | Filter MOC groups to only those needing rebuild |\n| | `update-moc-entries` | Upsert MOC entries from manifest (stdin) |\n| **MOC \u0026 tags** | `list-moc-groups` | Emit tag-\u003eids and project-\u003eids maps |\n| | `get-unresolved-tags` | Find tags absent from alias map (stdin) |\n| | `get-tag-counts` | Return {tag: count} for all tags in index |\n| | `get-canonical-tags` | Tags with N+ entries (non-fragment, default 10) |\n| | `normalize-tags` | Apply alias map to all entry/fragment tags |\n| | `sync-frontmatter-tags` | Sync wiki page frontmatter tags with index tags |\n| **Query \u0026 lint** | `search` | Keyword search across titles and tags |\n| | `score-idea` | Score fragments by relevance to an idea |\n| | `check-stale` | Find stale synthesis/idea entries |\n| | `lint-structural` | Run all mechanical lint checks |\n| | `get-fragments-by-type` | Filter fragments by type, grouped by tag |\n| | `get-entries-batch` | Return {id, title, tags} for IDs from stdin |\n| | `get-orphan-ids` | Extract orphan entry IDs from lint report |\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003ewiki_render.py subcommands\u003c/summary\u003e\n\n| Category | Subcommand | Description |\n|---|---|---|\n| **Source extraction** | `extract-source` | Pull text from PDF, arXiv, Zotero, ml-journal, text, markdown |\n| | `extract-zone` | Extract user zone (between frontmatter and `## Notes`) |\n| | `list-experiments` | List journal experiments as stub-ready index entries |\n| | `extract-notes` | Read preserved Notes section from an existing page |\n| **Page assembly** | `assemble` | Build a wiki page from agent output + entry JSON |\n| | `assemble-image` | Write an image wiki page directly (no agent output) |\n| | `prep-batch` | Read source text for a batch of stub entries |\n| | `finalize-batch` | Assemble pages for a completed render batch |\n| | `splice-idea` | Splice improve agent output into idea page |\n| **Build operations** | `read-journal-refs` | Collect experiment refs from journal JSONLs |\n| | `read-batch` | Read page content for a list of entries |\n| | `update-connections-batch` | Rewrite Connections sections from connection map |\n| | `update-banners-batch` | Add/remove staleness banners |\n| | `generate-listings-batch` | Write mechanical MOC listing pages (`--fast` mode) |\n| | `generate-topic-index` | Write `wiki/TOPICS.md` from moc-groups JSON |\n| | `generate-map` | Write `wiki/MAP.md` from build-agent output |\n| **Utilities** | `extract-agent-block` | Pull `===MARKER===...===END===` from stdin |\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003ewiki_embed.py subcommands\u003c/summary\u003e\n\n| Subcommand | Description |\n|---|---|\n| `score-connections` | Re-rank related candidates by MiniLM cosine similarity |\n| `score-idea` | Rank fragments by MiniLM cosine similarity against idea text from stdin |\n| `cluster-tags` | Cluster tag slugs by MiniLM cosine similarity |\n| `match-canonicals` | Match unresolved tags to nearest canonical by MiniLM cosine similarity |\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003ewiki_config.py subcommands\u003c/summary\u003e\n\n| Subcommand | Description |\n|---|---|\n| `get sources` | List all configured sources (JSON array) |\n| `get sources --name X` | Get one source by name |\n| `get sources --type X` | Filter sources by type |\n| `get wiki_dir` | Wiki content directory path |\n| `get index_path` | Path to index.jsonl |\n| `get default_render_depth` | Default render depth (shallow or deep) |\n| `get tag_aliases` | Current alias map |\n| `get llm` | LLM config (base_url, models, params, overrides) |\n| `set-aliases \u003cfile\u003e` | Overwrite tag_aliases from JSON file |\n| `merge-aliases \u003cfile\u003e` | Merge existing + new alias JSON -\u003e stdout |\n| `validate` | Check that all config paths exist |\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eExit codes with special meanings\u003c/summary\u003e\n\n| Script | Subcommand | Exit code | Meaning |\n|---|---|---|---|\n| `wiki_index.py` | `add` | 2 | ID collision (not 1) |\n| `wiki_index.py` | `exists` | 0 | Found |\n| `wiki_index.py` | `exists` | 1 | Not found |\n| `wiki_index.py` | `exists` | 2 | Source path mismatch |\n| `zotero_reader.py` | `lookup` | 0 | Found |\n| `zotero_reader.py` | `lookup` | 1 | Not found |\n\n\u003c/details\u003e\n\nFor a full subcommand reference organized by workflow stage with examples, see `.wiki/scripts/COMMANDS.md`.\n\n### Test suite\n\n295 tests live in `.wiki/scripts/tests/`. The repo includes a `pyproject.toml` that defines a `test` dependency group (pytest, plus all script dependencies needed for import-based testing). Run tests with:\n\n```bash\nuv run --group test pytest\n```\n\nScripts still use PEP 723 inline metadata for their runtime dependencies -- the `pyproject.toml` exists solely for test configuration and is not used at runtime.\n\n---\n\n## Known Gotchas\n\n- **BetterBibTeX JSON only** -- Zotero sources must use BetterBibTeX JSON export, not Better CSL JSON. The schema is different.\n- **`wiki_index.py add` exits with code 2 on ID collision**, not 1. Exit 1 means \"not found\" (for `exists`).\n- **`assemble` rejects incomplete agent output in delimiter format** -- the `===END===` marker must be present or assembly fails. `assemble` also accepts JSON directly (used by `/save` and `/render`), in which case no delimiter markers are needed.\n- **pymupdf** is resolved by `uv run` from the `# /// script` block in `wiki_render.py`. If `import fitz` fails, `uv` will install it.\n- **`## Connections` runs from the header to EOF** -- anything you add after it will be overwritten by `/build`.\n- **`\u003c!-- end-notes --\u003e` is required** -- without it, the Notes/Connections boundary is ambiguous and notes may be lost on re-render.\n- **MOC entries do not get fragments** -- they are navigational, not source material.\n- **Fragment IDs are not stable** -- they change on re-render. Reference the source entry ID, not fragment IDs.\n- **`zotero_reader.py lookup` is a boolean check** (exit 0/1). Use `parse --citekey` to retrieve entry data.\n- **Tag aliases are cumulative** -- the `tag_aliases` map in `config.yaml` grows with each `/normalize-tags` run. It is auto-managed; do not hand-edit it.\n- **`/build --fast` skips groups that already have a rendered narrative page.** This makes it safe to run without clobbering existing LLM-generated narratives. Use `--force` alongside `--fast` to override.\n- **`/render` rejects idea entries.** Ideas use `/idea improve` instead of `/render`.\n- **Idea pages have a user zone above `## Notes`** -- the original idea text lives there and is preserved by the improve pipeline. The evidence review is spliced below it.\n- **`/build` updates locked pages** -- locking only prevents `/render` from overwriting. Connections and staleness banners are always updated.\n- **`wiki_embed.py` downloads `all-MiniLM-L6-v2` on first run** (~80MB). Subsequent runs reuse the cached model.\n- **`arxiv_fetch.py` and `wiki_llm.py` have network access** -- all other scripts operate on local files only.\n- **`OPENROUTER_API_KEY` must be set** -- all LLM commands will fail without it. The env var name is configurable via `llm.api_key_env` in config.\n- **`wiki_cli.py` imports sibling modules** -- `wiki_util`, `wiki_build`, `wiki_idea`, `wiki_validate`, `wiki_llm`, and `wiki_render` are imported at runtime. This is the only cross-script coupling in the system; all other scripts remain fully independent.\n- **`/normalize-tags` has LLM cost** -- despite being a tag management command, it dispatches LLM calls at the `mini` tier (cluster confirmation) and `nano` tier (junk detection) during its multi-stage pipeline.\n- **Default topic page threshold is 10 entries** -- only tags with 10+ entries get MOC pages by default. Use `--min-entries` on `/build` to lower the threshold.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchris-santiago%2Fml-wiki","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchris-santiago%2Fml-wiki","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchris-santiago%2Fml-wiki/lists"}