{"id":35620947,"url":"https://github.com/u9401066/asset-aware-mcp","last_synced_at":"2026-05-13T05:01:50.730Z","repository":{"id":330530883,"uuid":"1123038362","full_name":"u9401066/asset-aware-mcp","owner":"u9401066","description":"Asset-Aware MCP Server — AI Agent precisely accesses tables, figures, sections from PDFs + .docx round-trip editing (DFM) with 46 tools / 13 resources, segmentation export, layout overlay, OCR preprocessing, knowledge graph (LightRAG)","archived":false,"fork":false,"pushed_at":"2026-04-09T01:23:06.000Z","size":7653,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-04-09T03:24:40.217Z","etag":null,"topics":["ai","document-processing","docx","etl","fastmcp","knowledge-graph","layout-analysis","lightrag","llm","mcp","mcp-server","medical","ocr","pdf","python","rag","segmentation"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/u9401066.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":"ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-26T04:29:14.000Z","updated_at":"2026-04-06T14:29:52.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/u9401066/asset-aware-mcp","commit_stats":null,"previous_names":["u9401066/asset-aware-mcp"],"tags_count":23,"template":false,"template_full_name":null,"purl":"pkg:github/u9401066/asset-aware-mcp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/u9401066%2Fasset-aware-mcp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/u9401066%2Fasset-aware-mcp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/u9401066%2Fasset-aware-mcp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/u9401066%2Fasset-aware-mcp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/u9401066","download_url":"https://codeload.github.com/u9401066/asset-aware-mcp/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/u9401066%2Fasset-aware-mcp/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32005837,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-18T20:23:30.271Z","status":"online","status_checked_at":"2026-04-19T02:00:07.110Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","document-processing","docx","etl","fastmcp","knowledge-graph","layout-analysis","lightrag","llm","mcp","mcp-server","medical","ocr","pdf","python","rag","segmentation"],"created_at":"2026-01-05T06:25:42.555Z","updated_at":"2026-05-13T05:01:50.723Z","avatar_url":"https://github.com/u9401066.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# asset-aware-mcp\n\n\u003e 🏥 Medical RAG with Asset-Aware MCP - Precise PDF asset retrieval (tables, figures, sections) and Knowledge Graph for AI Agents.\n\n[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n\n🌐 [繁體中文](README.zh-TW.md) · [Docs Site](https://u9401066.github.io/asset-aware-mcp/#/overview-zh) · [GitHub Wiki](https://github.com/u9401066/asset-aware-mcp/wiki)\n\n## 🎯 Why Asset-Aware MCP?\n\n**AI cannot directly read image files on your computer.** This is a common misconception.\n\n| Method | Can AI analyze image content? | Description |\n|------|:-------------------:|------|\n| ❌ Provide PNG path | No | AI cannot access the local file system |\n| ✅ **Asset-Aware MCP** | **Yes** | Retrieves Base64 via MCP, allowing AI vision to understand directly |\n\n### Real-world Effect\n\n```\n# After retrieving the image via MCP, the AI can analyze it directly:\n\nUser: What is this figure about?\n\nAI: This is the architecture diagram for Scaled Dot-Product Attention:\n    1. Inputs: Q (Query), K (Key), V (Value)\n    2. MatMul of Q and K\n    3. Scale (1/√dₖ)\n    4. Optional Mask (for decoder)\n    5. SoftMax normalization\n    6. Final MatMul with V to get the output\n```\n\n**This is the value of Asset-Aware MCP** - enabling AI Agents to truly \"see\" and understand charts and tables in your PDF literature.\n\n---\n\n## ✨ Features\n\n- 📄 **Asset-Aware ETL** - PDF → Markdown with a PyMuPDF-first parser and retained Marker code path:\n  - **PyMuPDF** (default) - Fast extraction (~50MB)\n  - **Marker** (`use_marker=True`) - High-precision structured parsing code path retained, but packaged runtime remains on security hold in v0.6.31 until upstream `marker-pdf` supports patched Pillow\n- 🧩 **Unified Segmentation Export** - Normalized `segmentation.json` merges manifest, blocks, reading order, and persisted markdown line spans for downstream tools and extensions.\n- 🖼️ **Layout Overlay Debugging** - Render page overlays from `original.pdf` to inspect bbox, segment type, and reading order visually.\n- 🔤 **On-Demand OCR Preprocessing** - Optional `ocrmypdf` preprocessing path for scanned PDFs before ETL.\n- 🧭 **Section Navigation** - Dynamic hierarchy section tree with 5 tools: browse, search, detail, content reading, and block extraction for any depth of headings.\n- 🔄 **Async Job Pipeline** - Supports asynchronous ingest, Marker-required parse, OCR, and conversion jobs with progress tracking.\n- 🗺️ **Document Manifest** - Provides a structured \"map\" of the document for precise data access by Agents.\n- 🧠 **LightRAG Integration** - Knowledge Graph + Vector Index, supporting cross-document comparison and reasoning.\n- 🧾 **Verified Citation Bundles** - `citation_bundle`, Foam evidence packs, citation health checks, table/figure evidence notes, and claim promotion export citation-ready spans with locator, quote/hash, context, CRAAP scaffold, and verification status.\n- 📝 **Docx Editing (DFM)** - Edit .docx files in Markdown via **Docx-Flavored Markdown** format. Supports legacy `.doc`, `.odt`, and `.ods` ingest via LibreOffice auto-conversion. 17 tools: ingest, read, save, list, delete, export, strict round-trip validation, DOCX→PDF/DOC/ODT, table edit planning, and Docx ↔ A2T bridges.\n- 🛡️ **DFM Integrity Checker** - Automatic validation and auto-repair at every pipeline stage (post-ingest, pre-save, post-save). Catches orphan markers, column mismatches, and format inconsistencies.\n- 📊 **A2T (Anything to Table)** - 7 operation-based tools for building professional tables from **any source** (PDF assets, Knowledge Graph, URLs, user input). Features: **Citations** (AssetRef), **Audit Trail**, **Schema Evolution**, **Templates**, **Drafting**, and **Token-efficient resumption**.\n- 🖥️ **VS Code Management Extension** - Graphical interface for monitoring server status, ingested documents, document artifacts, citation spans, and **A2T tables/drafts** with one-click Excel export.\n- 🔌 **MCP Server** - Exposes tools and resources to Copilot/Claude via FastMCP.\n- 🏥 **Medical Research Focus** - Optimized for medical literature, supporting Base64 image transmission for Vision AI analysis.\n\n## 🏗️ Architecture\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/images/architecture-overview.jpg\" alt=\"Asset-Aware MCP Architecture\" width=\"700\"\u003e\n\u003c/p\u003e\n\n```\n┌─────────────────────────────────────────────────────────┐\n│                    AI Agent (Copilot)                   │\n└─────────────────────┬───────────────────────────────────┘\n                      │ MCP Protocol (Tools \u0026 Resources)\n┌─────────────────────▼───────────────────────────────────┐\n│            MCP Server (Modular Presentation)            │\n│  ┌─────────────────────────────────────────────────┐   │\n│  │ tools/: 62 tools in 7 modules                   │   │\n│  │   document (19) │ docx (17) │ section (5)       │   │\n│  │   job (4) │ knowledge (3) │ table (7) │ profile (7) │\n│  └─────────────────────────────────────────────────┘   │\n│  ┌─────────────────────────────────────────────────┐   │\n│  │ resources/: 13 resources in 2 modules           │   │\n│  └─────────────────────────────────────────────────┘   │\n└─────────────────────┬───────────────────────────────────┘\n                      │\n┌─────────────────────▼───────────────────────────────────┐\n│                  ETL Pipeline (DDD)                     │\n│  ┌──────────┐  ┌──────────┐  ┌──────────┐              │\n│  │ PyMuPDF  │  │  Asset   │  │ LightRAG │              │\n│  │ Adapter  │→ │  Parser  │→ │  Index   │              │\n│  └──────────┘  └──────────┘  └──────────┘              │\n└─────────────────────┬───────────────────────────────────┘\n                      │\n┌─────────────────────▼───────────────────────────────────┐\n│                   Local Storage                         │\n│  ./data/                                                │\n│  ├── {doc_id}/        # PDF document artifacts          │\n│  ├── docx_{id}/       # Docx IR + DFM + Assets          │\n│  ├── tables/          # A2T Tables (JSON/MD/XLSX)       │\n│  │   └── drafts/      # Table Drafts (Persistence)      │\n│  └── lightrag_db/     # Knowledge Graph                 │\n└─────────────────────────────────────────────────────────┘\n```\n\n## 📁 Project Structure (DDD)\n\n```\nasset-aware-mcp/\n├── src/\n│   ├── domain/              # 🔵 Domain: Entities, Value Objects, Interfaces\n│   ├── application/         # 🟢 Application: Doc Service, Table Service (A2T), Asset Service\n│   ├── infrastructure/      # 🟠 Infrastructure: PyMuPDF, LightRAG, Excel Renderer\n│   └── presentation/        # 🔴 Presentation: MCP Server (FastMCP)\n├── data/                    # Document and Asset Storage\n├── docs/\n│   └── spec.md              # Technical Specification\n├── tests/                   # Unit and Integration Tests\n├── vscode-extension/        # VS Code Management Extension\n└── pyproject.toml           # uv Project Config\n```\n\n## 📐 Architecture Diagrams\n\nVisual overview for the project. All diagrams use consistent GitHub README style.\n\n| Diagram | Description |\n|---------|-------------|\n| [01 — System Architecture](docs/diagrams/01-system-architecture.jpg) | Full stack: Telegram → Gateway → MCP Adapter → 3 MCP servers → Ollama |\n| [02 — Data Layout](docs/diagrams/02-data-layout.jpg) | 62 tools organized in 7 categories with asset-aware data tree |\n| [03 — PDF Ingestion Pipeline](docs/diagrams/03-pdf-ingestion-pipeline.jpg) | 7-stage flow from PDF upload to knowledge graph |\n| [04 — DOCX Bidirectional Edit](docs/diagrams/04-docx-edit-pipeline.jpg) | DOCX ingest → TableContext edit → round-trip save workflow |\n| [05 — Knowledge Graph Search](docs/diagrams/05-knowledge-graph-search.jpg) | Cross-document search with 3 parallel query paths |\n| [06 — Installation Steps](docs/diagrams/06-installation-steps.jpg) | 7-step installation from clone to verification |\n| [07 — PDF ETL Pipeline](docs/diagrams/07-pdf-etl-pipeline.jpg) | PyMuPDF default path + Marker security-hold diagnostics |\n| [08 — KG Architecture](docs/diagrams/08-knowledge-graph-architecture.jpg) | lightrag-hku 3-layer KG architecture |\n| [09 — Agent Harness Concept](docs/diagrams/09-agent-harness-concept.jpg) | Assistant harness model for stateless agents |\n\n\u003e 💡 All generation prompts are saved in [docs/diagrams/ALL-PROMPTS.md](docs/diagrams/ALL-PROMPTS.md) for style consistency and regeneration.\n\n## 🚀 Quick Start\n\n```bash\n# Install dependencies (using uv) — default install skips Marker/torch\nuv sync\n\n# v0.6.31: Marker extra is temporarily empty because marker-pdf pins\n# Pillow\u003c11 while the secure runtime requires Pillow\u003e=12.2.0.\n# Use the default PyMuPDF backend until upstream marker-pdf supports patched Pillow.\n\n# Run MCP Server\nuv run python -m src.presentation.server\n\n# Or use the VS Code extension for graphical management\n```\n\nRuntime note:\nThe VS Code extension prefers a managed Python 3.11 runtime when launching the MCP server via `uv` or `uvx`. This avoids native package builds on end-user machines, especially macOS systems without Xcode Command Line Tools, while keeping the project itself compatible with newer Python versions.\n\nInstallation scope note:\n- The VS Code extension installs once per user (global). MCP launch env defaults `DATA_DIR` to workspace `./data` and `UV_CACHE_DIR` to `DATA_DIR/.uv-cache`; Prepare Server Runtime warms a workspace `.uv-cache`, falling back to extension global storage only when no workspace is open.\n- Runtime data stays with your repo: `.env` and `assetAwareMcp.dataDir` default to `./data`, so ingested assets and the uv cache used by the launched server remain scoped to the current workspace.\n\nMarker note:\nSince v0.6.28 the packaged Marker extra has intentionally stayed on security hold: upstream `marker-pdf` 1.10.2 requires `Pillow\u003c11`, while this release pins `Pillow\u003e=12.2.0` for patched image-processing security. Default installs use the PyMuPDF backend only. `use_marker=True` / `parse_pdf_structure` will report that Marker is unavailable until upstream Marker supports a patched Pillow range.\n\n## 🔌 MCP Tools\n\n### Document \u0026 Asset Tools\n\n| Tool | Purpose |\n|------|---------|\n| `ingest_documents` | Process PDF files with PyMuPDF; `use_marker=True` currently falls back or fails closed while Marker is on security hold |\n| `list_documents` | List all ingested documents and their asset counts |\n| `delete_document` | Delete an ingested PDF, its local artifacts, and LightRAG index entries when enabled |\n| `convert_pdf_to_docx` | Reconstruct a readable DOCX from extracted PDF content; defaults to a conversion background job |\n| `convert_pdf_to_pptx` | Rebuild editable PPTX slides from extracted PDF markdown and figures; defaults to a conversion background job |\n| `inspect_document_manifest` | Inspect document structure before fetching specific assets |\n| `fetch_document_asset` | Precisely retrieve tables (MD) / figures (B64) / sections |\n| `parse_pdf_structure` | Queue structured parsing work; Marker output remains unavailable until upstream Marker supports patched Pillow |\n| `search_source_location` | Search exact source locations with page + bbox for verification |\n| `export_document_segmentation` | Export normalized `segmentation.json` with reading order + line ranges |\n| `visualize_document_layout` | Render page overlay images for bbox / type / reading-order inspection |\n| `ocr_pdf_document` | Run OCR preprocessing and generate a cleaned PDF for later ETL |\n| `find_evidence_spans` | Search citation-ready spans with source revision, locator, hash, and CRAAP scaffold |\n| `verify_citation_ref` | Verify span AssetRefs against the current citation index and locator metadata |\n| `citation_bundle` | Export verified evidence bundles with AssetRef, quote/hash, locator, context, CRAAP scaffold, and verification status |\n| `document` | Operation-based facade over PDF ingest/list/delete/inspect/parse |\n| `document_asset` | Operation-based facade over asset fetch and section tree/detail/blocks/search |\n| `evidence` | Operation-based facade over citation span find/verify/source-location search and bundle export |\n| `convert_document` | Operation-based facade for PDF, DOCX/DFM, and Markdown conversions; conversion paths default to background jobs |\n\n### Job Management Tools\n\n| Tool | Purpose |\n|------|---------|\n| `get_job_status` | Get async ingestion/conversion job progress and final result |\n| `list_jobs` | List active or historical ETL jobs |\n| `cancel_job` | Cancel a running ETL job |\n| `job` | Operation-based facade over job get/list/cancel |\n\n### Knowledge Graph Tools\n\n| Tool | Purpose |\n|------|---------|\n| `consult_knowledge_graph` | Citation-aware knowledge graph query with `structured`, `data`, `text`, and optional verified evidence bundles |\n| `export_knowledge_graph` | Export graph summary / JSON / Mermaid for inspection |\n| `knowledge` | Operation-based facade over knowledge graph consult/export |\n\nKnowledge graph note:\n- `consult_knowledge_graph` defaults to `response_mode=\"structured\"` and can return `answer`, `references`, `metadata`, `retrieval`, `counts`, and `verified_evidence` when `verify_references=true`.\n- Use `response_mode=\"data\"` when you want retrieval payloads without final answer synthesis, or `response_mode=\"text\"` for legacy plain-text behavior.\n\n### Section Navigation Tools (Dynamic Hierarchy)\n\n| Tool | Purpose |\n|------|---------|\n| `list_section_tree` | Display complete section hierarchy tree (supports any depth) |\n| `get_section_detail` | Get detailed info for a specific section |\n| `get_section_blocks` | Extract all blocks from a section with page + bbox |\n| `search_sections` | Search section titles |\n| `get_section_content` | Read section content via asset service |\n\n### Docx Editing Tools (DFM — Docx-Flavored Markdown)\n\n\u003e Edit .docx files as Markdown. Preserves formatting, tables, media on round-trip.\n\n| Tool | Purpose |\n|------|---------|\n| `ingest_docx` | Import .docx and decompose into DFM blocks |\n| `get_docx_content` | Read DFM content of specific blocks |\n| `save_docx` | Write DFM edits back to .docx |\n| `list_docx_blocks` | List document block structure |\n| `list_docx_documents` | List all ingested DOCX/DFM documents |\n| `delete_docx` | Delete an ingested DOCX/DFM document and its local artifacts |\n| `convert_docx_to_pdf` | Export the current DOCX/DFM state to PDF in fidelity mode; defaults to a conversion background job |\n| `convert_docx_to_doc` | Export the current DOCX/DFM state to DOC in fidelity mode; defaults to a conversion background job |\n| `docx_validate_roundtrip` | 6-dimension round-trip fidelity validation + file-level comparison (SHA-256, ZIP diff) |\n| `docx_table_to_context` | Bridge: Docx table → A2T context |\n| `docx_table_from_context` | Bridge: A2T table → Docx table |\n| `docx_chart_data` | Extract chart data from Docx |\n| `docx_table_edit_plan` | Preview table cell/row/column/header changes and structural risks before write-back |\n| `export_markdown` | Export Markdown to .docx/.pdf/.doc; defaults to a conversion background job |\n| `convert_docx_to_odt` | Export the current DOCX/DFM state to ODT; defaults to a conversion background job |\n| `docx` | Operation-based facade over DOCX/DFM ingest/get/save/list/delete/blocks/validate |\n| `docx_table` | Operation-based facade over DOCX table to_context/from_context/chart_data/edit_plan |\n\n### A2T (Anything to Table) Tools — 7 Operation-Based Tools\n\n\u003e Agent-friendly design: each tool handles multiple operations via `operation` parameter.\n\u003e Tables accept **any source** — PDF assets, KG entities, external URLs, or user input.\n\n| Tool | Operations | Purpose |\n|------|-----------|----------|\n| `plan_table` | `schema` / `templates` / `from_template` | Schema planning, browse 4 built-in templates, create from template |\n| `table_manage` | `create` / `delete` / `list` / `preview` / `resume` / `render` / `add_column` / `remove_column` / `rename_column` | Table lifecycle + Schema evolution |\n| `table_data` | `add_rows` / `get_row` / `update_row` / `delete_row` / `get_cell` / `update_cell` / `clear_cell` | Row \u0026 cell CRUD |\n| `table_cite` | `add` / `get` / `remove` / `cell_history` | Citation management with AssetRef (7 source types) |\n| `table_history` | `changes` / `tokens` | Audit trail \u0026 token estimation |\n| `table_draft` | `create` / `update` / `add_rows` / `resume` / `commit` / `list` / `delete` | Draft workflow with persistence |\n| `discover_sources` | — | Cross-document source discovery (sections, tables, figures, KG) |\n\n### ETL Profile Tools\n\nDifferent journals/formats need different extraction settings. Use these tools to switch profiles.\n\n| Tool | Purpose |\n|------|---------|\n| `list_etl_profiles` | List all available profiles (default, arxiv, nature, ieee, elsevier) |\n| `get_etl_profile` | Get detailed configuration of a specific profile |\n| `get_current_etl_profile` | Show currently active profile |\n| `set_etl_profile` | Switch profile for subsequent document ingestion |\n| `load_etl_profile_from_json` | Load custom profile from JSON file |\n| `detect_etl_profile` | Detect the best built-in profile from PDF path, doc_id, or sample text |\n| `etl_profile` | Operation-based facade over profile list/get/current/set/load/detect |\n\n## 🔧 Tech Stack\n\n| Category | Technology |\n|----------|------------|\n| Language | Python 3.10+ |\n| Package Manager | **uv** (all pip/setup-python removed) |\n| ETL | **PyMuPDF** (fitz); **Marker** is temporarily on security hold |\n| RAG | LightRAG (lightrag-hku) |\n| MCP | FastMCP |\n| Storage | Local filesystem (JSON/Markdown/PNG) |\n\n## 📋 Documentation\n\nInstallation guidance:\n- Default install: `uv sync`\n- Marker backend: temporarily disabled in v0.6.31 because `marker-pdf` pins vulnerable `Pillow\u003c11`; the `marker` / `pdf` extras are compatibility placeholders until upstream supports patched Pillow.\n- VS Code extension: `assetAwareMcp.enableMarkerBackend` is retained as a setting, but the launcher will not install `marker-pdf` while the security hold is active.\n\n- [Technical Spec](docs/spec.md) - Detailed technical specification\n- [Architecture](ARCHITECTURE.md) - System architecture\n- [Constitution](CONSTITUTION.md) - Project principles\n- [Competitive Analysis](docs/competitor-analysis.md) - MCP + DOCX ecosystem landscape\n\n## 📄 License\n\n[Apache License 2.0](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fu9401066%2Fasset-aware-mcp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fu9401066%2Fasset-aware-mcp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fu9401066%2Fasset-aware-mcp/lists"}