{"id":51640138,"url":"https://github.com/sunilgentyala/marksentry","last_synced_at":"2026-07-13T18:32:47.405Z","repository":{"id":363723905,"uuid":"1264582776","full_name":"sunilgentyala/marksentry","owner":"sunilgentyala","description":"Secure, local-first document-to-Markdown conversion. Zero-trust input sanitization, multi-column PDF layout, LaTeX math, PII masking. No cloud dependencies.","archived":false,"fork":false,"pushed_at":"2026-06-10T04:04:22.000Z","size":119,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-06-10T06:08:32.653Z","etag":null,"topics":["document-conversion","docx","latex","local-first","markdown","pdf","pii-masking","rag","security","ssrf-protection"],"latest_commit_sha":null,"homepage":"https://sunilgentyala.github.io/marksentry/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sunilgentyala.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-10T02:28:42.000Z","updated_at":"2026-06-10T04:04:26.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/sunilgentyala/marksentry","commit_stats":null,"previous_names":["sunilgentyala/marksentry"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/sunilgentyala/marksentry","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunilgentyala%2Fmarksentry","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunilgentyala%2Fmarksentry/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunilgentyala%2Fmarksentry/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunilgentyala%2Fmarksentry/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sunilgentyala","download_url":"https://codeload.github.com/sunilgentyala/marksentry/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunilgentyala%2Fmarksentry/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35432816,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-13T02:00:06.543Z","response_time":119,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["document-conversion","docx","latex","local-first","markdown","pdf","pii-masking","rag","security","ssrf-protection"],"created_at":"2026-07-13T18:32:46.734Z","updated_at":"2026-07-13T18:32:47.399Z","avatar_url":"https://github.com/sunilgentyala.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MarkSentry\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/python-3.10%2B-blue?logo=python\u0026logoColor=white\" alt=\"Python 3.10+\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/license-MIT-green\" alt=\"MIT License\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/execution-100%25%20local-brightgreen\" alt=\"Local-first\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/MCP-Claude%20Code-blueviolet?logo=anthropic\" alt=\"Claude Code MCP\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/zero--trust-input%20sanitizer-red\" alt=\"Zero-trust\"\u003e\n\u003c/p\u003e\n\n\u003e **Secure, local-first document-to-Markdown conversion.**\n\u003e Zero cloud dependencies. Zero-trust input sanitization. Correct multi-column layout. Full LaTeX math support.\n\nBuilt to address the layout bugs, mathematical omissions, and critical security vulnerabilities (SSRF, path traversal, cloud-API lock-in) found in existing conversion utilities.\n\n---\n\n## Table of Contents\n\n- [Why MarkSentry?](#why-marksentry)\n- [Architecture](#architecture)\n- [Installation](#installation)\n- [Quick Start](#quick-start)\n  - [CLI](#cli)\n  - [Python SDK](#python-sdk)\n- [Claude Code MCP Integration](#claude-code-mcp-integration)\n- [Feature Deep Dives](#feature-deep-dives)\n  - [Zero-Trust Input Sanitizer](#zero-trust-input-sanitizer)\n  - [Multi-Column Layout Processor](#multi-column-layout-processor)\n  - [Table Reconstruction](#table-reconstruction)\n  - [LaTeX Math Output](#latex-math-output)\n  - [PII Masking](#pii-masking)\n- [CLI Reference](#cli-reference)\n- [Development](#development)\n- [License](#license)\n- [Author](#author-and-attribution)\n\n---\n\n## Why MarkSentry?\n\n| Capability | MarkSentry | Microsoft MarkItDown | Typical Python converters |\n|---|:---:|:---:|:---:|\n| Path traversal prevention | ✅ full jail + null-byte checks | ❌ | ❌ |\n| SSRF mitigation | ✅ embedded URI scanner, RFC-1918 block | ❌ | ❌ |\n| VBA macro stripping | ✅ OOXML rewrite before parse | ❌ | ❌ |\n| Zip bomb detection | ✅ ratio + nesting depth | ❌ | ❌ |\n| Multi-column PDF layout | ✅ gap-analysis algorithm | ❌ reads across columns | ❌ |\n| LaTeX math output | ✅ OMML + Unicode to LaTeX | ❌ omits equations | ❌ omits equations |\n| GFM table reconstruction | ✅ coordinate-aligned grid | Partial | Partial |\n| PII masking (pre-RAG) | ✅ 10 pattern categories | ❌ | ❌ |\n| 100% local execution | ✅ zero network calls | ❌ calls Azure OCR | Varies |\n| Magic-byte validation | ✅ extension + header check | ❌ | ❌ |\n\n---\n\n## Architecture\n\n```\n                  User-supplied file\n                        |\n              ┌─────────▼──────────┐\n              │  Zero-Trust        │  sanitizer.py\n              │  Input Sanitizer   │  - path traversal jail\n              │                    │  - SSRF URI scan\n              │                    │  - macro stripping\n              │                    │  - zip-bomb detection\n              └─────────┬──────────┘\n                        |\n          ┌─────────────┼─────────────┐\n          |             |             |\n   ┌──────▼──────┐ ┌────▼────┐ ┌─────▼──────┐\n   │  PDF Parser │ │  DOCX   │ │  ZIP       │\n   │             │ │  Parser │ │  Dispatcher│\n   │ pdfminer    │ │ python- │ └─────┬──────┘\n   │ + BBox grid │ │ docx    │       | recurse\n   └──────┬──────┘ └────┬────┘       |\n          |             |            |\n          └──────┬───────────────────┘\n                 |\n     ┌───────────▼──────────────┐\n     │  Multi-Column Layout     │  layout.py\n     │  Processor               │  - gap analysis\n     │                          │  - column assignment\n     │                          │  - reading-order sort\n     └───────────┬──────────────┘\n                 |\n     ┌───────────▼──────────────┐\n     │  Table + Math Core       │  pdf_parser.py / math_converter.py\n     │                          │  - GFM table reconstruction\n     │                          │  - OMML → LaTeX (DOCX)\n     │                          │  - Unicode → LaTeX (PDF)\n     └───────────┬──────────────┘\n                 |\n     ┌───────────▼──────────────┐\n     │  PII Masking Filter      │  pii_filter.py  (optional)\n     │                          │  - SSN, email, credit card\n     │                          │  - private keys, JWT, AWS keys\n     └───────────┬──────────────┘\n                 |\n          Markdown output\n```\n\n---\n\n## Installation\n\n**Requirements:** Python 3.10+\n\n```bash\n# From source\ngit clone https://github.com/sunilgentyala/marksentry\ncd marksentry\npip install -e .\n\n# From PyPI (once published)\npip install marksentry\n```\n\nOptional OCR support (for scanned PDFs):\n\n```bash\npip install marksentry[ocr]\n# Also requires: sudo apt install tesseract-ocr  (or brew install tesseract)\n```\n\n---\n\n## Quick Start\n\n### CLI\n\n```bash\n# Convert a PDF, output to file\nmarksentry convert paper.pdf --output paper.md\n\n# Convert with PII masking enabled\nmarksentry convert hr_report.pdf --mask-pii --output clean.md\n\n# Convert multiple files into an output directory\nmarksentry convert *.pdf --output-dir ./converted/\n\n# Convert a DOCX with LaTeX math, no table detection\nmarksentry convert thesis.docx --no-tables --output thesis.md\n\n# Convert a ZIP archive containing mixed documents\nmarksentry convert bundle.zip --output-dir ./extracted/\n\n# Dry-run PII audit (no output written)\nmarksentry audit-pii confidential.pdf\n\n# Show document metadata and security scan results\nmarksentry info suspicious.docx\n```\n\n### Python SDK\n\n```python\nfrom marksentry import convert\n\n# Simple conversion\nresult = convert(\"research_paper.pdf\")\nprint(result.markdown)\n\n# With all security + compliance features\nresult = convert(\n    \"internal_report.pdf\",\n    mask_pii=True,\n    pii_patterns=[\"SSN\", \"EMAIL\", \"CREDIT_CARD\"],\n    multi_column=True,\n    detect_math=True,\n    allowed_base=\"/safe/data/directory\",   # path jail\n    strip_macros=True,\n    check_ssrf=True,\n)\n\nfor warning in result.warnings:\n    print(f\"[WARNING] {warning}\")\n\nwith open(\"output.md\", \"w\") as f:\n    f.write(result.markdown)\n```\n\n---\n\n## Claude Code MCP Integration\n\nMarkSentry ships a [Model Context Protocol](https://modelcontextprotocol.io) (MCP) server so\nClaude Code can convert documents **before** sending their content to the model. This eliminates\nthe token cost of reading raw binary files: a 20-page IEEE paper that would consume 15,000+\ntokens as raw bytes arrives as ~2,000 tokens of clean Markdown.\n\n### Why it saves tokens\n\n| Without MCP | With MCP |\n|---|---|\n| Claude reads raw PDF bytes or you paste text manually | Claude calls `convert_to_markdown`, receives clean Markdown |\n| Every page consumes image or binary tokens | Only the extracted text is sent |\n| Tables and math arrive garbled or are skipped | Tables are GFM-formatted, equations are LaTeX |\n| PII may reach the model | Optional `mask_pii=True` redacts before conversion |\n\n### Setup\n\n**1. Install MarkSentry and the MCP package:**\n\n```bash\ngit clone https://github.com/sunilgentyala/marksentry\ncd marksentry\npip install -e .\npip install mcp\n```\n\n**2. Register with Claude Code:**\n\n```bash\n# macOS / Linux\nclaude mcp add --scope user marksentry python /path/to/marksentry/marksentry_mcp.py\n\n# Windows — use the full path to your Python executable to avoid shell stubs\nclaude mcp add --scope user marksentry \"C:\\Python\\python.exe\" \"C:\\path\\to\\marksentry\\marksentry_mcp.py\"\n```\n\n\u003e **Windows note:** Using just `python` can resolve to the Windows Store stub that has no\n\u003e packages installed. Run `where python` and pass the full path returned.\n\n**3. Verify:**\n\n```bash\nclaude mcp get marksentry\n# Status: Connected\n```\n\nRestart Claude Code after registration for the tools to load into the session.\n\n### MCP Tools\n\n| Tool | Purpose |\n|---|---|\n| `convert_to_markdown` | Primary tool. Converts PDF, DOCX, or ZIP to clean Markdown. Use whenever a document path is shared. |\n| `audit_pii` | Pre-flight PII scan. Reports pattern types and hit counts without producing full output. |\n| `document_info` | Metadata preflight. Returns file type, size, macro status, and any SSRF-risk URLs. |\n\n### Parameters for `convert_to_markdown`\n\n| Parameter | Default | Description |\n|---|:---:|---|\n| `path` | required | Absolute or relative path to the document |\n| `mask_pii` | `false` | Replace detected PII with `[REDACTED:TYPE]` placeholders |\n| `include_page_breaks` | `false` | Emit `---` between pages in PDF output |\n\n### Usage examples\n\nOnce the MCP server is connected, just share a file path in your prompt:\n\n```\n\"Summarise this paper: /home/user/papers/research.pdf\"\n  → Claude calls convert_to_markdown, receives Markdown, answers from text\n\n\"Does this DOCX contain any PII? /home/user/docs/report.docx\"\n  → Claude calls audit_pii first, reports findings before converting\n\n\"What type of file is this and is it safe to convert? /tmp/unknown.zip\"\n  → Claude calls document_info for a fast preflight check\n```\n\n---\n\n## Feature Deep Dives\n\n### Zero-Trust Input Sanitizer\n\nEvery file passes through the sanitizer before reaching any parser:\n\n| Check | Detail |\n|---|---|\n| **Path traversal** | Resolves symlinks, rejects `../`, null bytes, UNC paths, and `file://`/`http://` schemes |\n| **Path jail** | Pass `allowed_base` to restrict access to a directory tree; any escape raises `SanitizationError` |\n| **Magic-byte validation** | Compares file header bytes against the expected signature; rejects spoofed extensions |\n| **SSRF mitigation** | Scans XML/relationship/HTML entries in OOXML archives; blocks RFC-1918, loopback, link-local, and `file://` URLs |\n| **Macro stripping** | Detects `vbaProject.bin` and macro content-type declarations; rewrites the archive without them |\n| **Zip bomb detection** | Enforces a max uncompressed:compressed ratio (default 100x) and nesting depth (default 3) |\n| **File size ceiling** | Configurable hard limit (default 256 MiB) to prevent memory exhaustion |\n\n### Multi-Column Layout Processor\n\nAcademic papers commonly use 2- or 3-column layouts. Naive parsers read text horizontally,\nproducing garbled output:\n\n```\nIntroduction  Methodology  Results\nWe studied...  We applied...  Our findings...\n```\n\nMarkSentry uses a gap-analysis algorithm:\n\n1. Build a 1-point-resolution coverage histogram across the page width\n2. Identify contiguous uncovered regions wider than the minimum gap threshold (default 18pt)\n3. Declare each gap a column separator\n4. Assign every text block to the column it overlaps most\n5. Sort within each column top-to-bottom\n6. Interleave full-width blocks (titles, section headers, captions) at their correct vertical position\n\nThe result is correct sequential flow through each column before moving to the next.\n\n### Table Reconstruction\n\nThe `TableDetector` clusters text blocks by shared y-bands (rows) and x-positions (columns),\nidentifies grid regions where at least 3 rows each contain 2+ aligned cells, and emits\nstandard GitHub-Flavored Markdown pipe tables:\n\n```markdown\n| Method     | Precision | Recall | F1   |\n|------------|-----------|--------|------|\n| Baseline   | 0.82      | 0.79   | 0.80 |\n| MarkSentry | 0.94      | 0.91   | 0.92 |\n```\n\n### LaTeX Math Output\n\n**PDF documents:** Unicode mathematical symbols (Greek letters, operators, set notation,\nintegrals, sums) are mapped to LaTeX equivalents. Inline-length formulas use `$...$`;\ndisplay-length formulas use `$$...$$`.\n\n**DOCX documents:** Office Math Markup Language (OMML) equations are converted via a\nrecursive XML descent parser:\n\n| OMML element | LaTeX output |\n|---|---|\n| `m:f` (fraction) | `\\frac{num}{den}` |\n| `m:rad` (radical) | `\\sqrt{x}` or `\\sqrt[n]{x}` |\n| `m:sSup` (superscript) | `x^{n}` |\n| `m:sSub` (subscript) | `x_{i}` |\n| `m:nary` (integral, sum, product) | `\\int_{a}^{b}`, `\\sum_{i=0}^{n}` |\n| `m:d` (delimiters) | `\\left( \\right)`, `\\left[ \\right]` |\n| `m:m` (matrix) | `\\begin{pmatrix}...\\end{pmatrix}` |\n| `m:eqArr` (equation array) | `\\begin{aligned}...\\end{aligned}` |\n\n### PII Masking\n\nTen pattern categories, all with validation (credit cards are Luhn-checked to eliminate\nfalse positives):\n\n| Category | Example input | Masked output |\n|---|---|---|\n| `SSN` | `123-45-6789` | `[REDACTED:SSN]` |\n| `EMAIL` | `user@corp.com` | `[REDACTED:EMAIL]` |\n| `CREDIT_CARD` | `4111 1111 1111 1111` | `[REDACTED:CREDIT_CARD]` |\n| `PHONE` | `(555) 867-5309` | `[REDACTED:PHONE]` |\n| `PRIVATE_KEY` | `-----BEGIN RSA PRIVATE KEY-----...` | `[REDACTED:PRIVATE_KEY]` |\n| `AWS_ACCESS_KEY` | `AKIAIOSFODNN7EXAMPLE` | `[REDACTED:AWS_ACCESS_KEY]` |\n| `JWT_TOKEN` | `eyJhbGciOiJIUzI1NiIs...` | `[REDACTED:JWT_TOKEN]` |\n| `IPV4` | `192.168.1.100` | `[REDACTED:IPV4]` |\n| `PASSWORD_ASSIGNMENT` | `password=s3cr3t!` | `[REDACTED:PASSWORD_ASSIGNMENT]` |\n| `HIGH_ENTROPY_HEX` | `a3f2c1d4e5b6a7f8...` | `[REDACTED:HIGH_ENTROPY_HEX]` |\n\nUse `marksentry audit-pii` for a dry run that reports findings without modifying any output.\n\n---\n\n## CLI Reference\n\n```\nUsage: marksentry [OPTIONS] COMMAND [ARGS]...\n\nCommands:\n  convert    Convert one or more documents to Markdown.\n  audit-pii  Scan for PII without modifying output (dry run).\n  info       Display document metadata and security scan summary.\n\nmarksentry convert [OPTIONS] FILES...\n\nOptions:\n  -o, --output PATH          Output file (single input only)\n  --output-dir DIR           Output directory for batch conversion\n  --mask-pii                 Enable PII masking\n  --pii-patterns TEXT        Comma-separated pattern names (default: all)\n  --no-tables                Disable table reconstruction\n  --no-math                  Disable LaTeX math conversion\n  --no-multi-column          Disable column layout reconstruction\n  --column-gap FLOAT         Column gap threshold in points (default: 18.0)\n  --page-breaks              Emit --- between PDF pages\n  --no-strip-macros          Skip macro stripping\n  --no-ssrf-check            Skip embedded URL scanning\n  --allowed-base DIR         Restrict file access to this directory\n  -v, --verbose              Enable debug logging\n```\n\n---\n\n## Development\n\n```bash\n# Install with dev dependencies\npip install -e \".[dev]\"\n\n# Run tests with coverage\npytest\n\n# Type checking\nmypy marksentry/\n\n# Linting\nruff check marksentry/\n```\n\n---\n\n## License\n\nMIT License. See `LICENSE` for details.\n\n---\n\n## Author and Attribution\n\n**MarkSentry** was designed and built by **Sunil Gentyala**.\n\nSunil Gentyala is a principal software architect and cybersecurity researcher at HCL America\nInc., with expertise in secure systems design, document intelligence pipelines, and adversarial\nrobustness. MarkSentry was created to solve real security gaps in existing document conversion\ntooling -- in particular the complete absence of input sanitization, SSRF defenses, and correct\nmulti-column layout handling in tools like Microsoft MarkItDown -- and to provide a\nproduction-grade, offline-first foundation for RAG data pipelines that must handle sensitive\ndocuments safely.\n\nContact: sunil.gentyala@ieee.org\nGitHub: [github.com/sunilgentyala](https://github.com/sunilgentyala)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsunilgentyala%2Fmarksentry","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsunilgentyala%2Fmarksentry","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsunilgentyala%2Fmarksentry/lists"}