{"id":50472360,"url":"https://github.com/semcod/mdflow","last_synced_at":"2026-06-01T11:03:12.996Z","repository":{"id":355368604,"uuid":"1227819593","full_name":"semcod/mdflow","owner":"semcod","description":null,"archived":false,"fork":false,"pushed_at":"2026-05-03T08:52:54.000Z","size":1070,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-03T10:19:39.744Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/semcod.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-03T07:53:01.000Z","updated_at":"2026-05-03T08:52:56.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/semcod/mdflow","commit_stats":null,"previous_names":["semcod/mdflow"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/semcod/mdflow","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semcod%2Fmdflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semcod%2Fmdflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semcod%2Fmdflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semcod%2Fmdflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/semcod","download_url":"https://codeload.github.com/semcod/mdflow/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semcod%2Fmdflow/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33771630,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-01T02:00:06.963Z","response_time":115,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-01T11:03:12.891Z","updated_at":"2026-06-01T11:03:12.989Z","avatar_url":"https://github.com/semcod.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# mdflow\n\n## AI Cost Tracking\n\n![PyPI](https://img.shields.io/badge/pypi-costs-blue) ![Version](https://img.shields.io/badge/version-0.1.6-blue) ![Python](https://img.shields.io/badge/python-3.9+-blue) ![License](https://img.shields.io/badge/license-Apache--2.0-green)\n![AI Cost](https://img.shields.io/badge/AI%20Cost-$0.75-orange) ![Human Time](https://img.shields.io/badge/Human%20Time-2.0h-blue) ![Model](https://img.shields.io/badge/Model-openrouter%2Fqwen%2Fqwen3--coder--next-lightgrey)\n\n- 🤖 **LLM usage:** $0.7500 (5 commits)\n- 👤 **Human dev:** ~$200 (2.0h @ $100/h, 30min dedup)\n\nGenerated on 2026-05-03 using [openrouter/qwen/qwen3-coder-next](https://openrouter.ai/qwen/qwen3-coder-next)\n\n---\n\n**Markdown dependency analyzer — extract all dependencies, generate diagrams and charts.**\n\n`mdflow` parses Markdown files and extracts every possible structural element:\nheadings, links, fenced code blocks (including `markpact:*` embedded file references),\nlist items, TOON/YAML quality sections, and document metadata.\nIt then generates Mermaid diagrams, HTML reports, and Markdown summaries.\n\n---\n\n## What it extracts\n\n| Element | Details |\n|---|---|\n| **Headings** | Full H1–H6 hierarchy, anchor slugs |\n| **Links** | `[text](href)` — classified as internal / external / anchor / image |\n| **Code blocks** | Language, content, line range, `markpact:type path=...` metadata |\n| **List items** | Depth, parent heading, clean text |\n| **TOON sections** | ALERTS, REFACTOR, HOTSPOTS, HEALTH, NEXT, RISKS, PIPELINES… |\n| **Document metadata** | `## Metadata` key/value lists |\n| **Cross-doc dependencies** | Links between files, `markpact` embedded file paths |\n\n---\n\n## Generated outputs\n\n| Output | Description |\n|---|---|\n| `{stem}_report.html` | Self-contained HTML report with all diagrams (Mermaid.js) |\n| `{stem}_report.md` | Markdown summary with inline Mermaid |\n| `{stem}_heading_mindmap.mermaid` | Mindmap of heading hierarchy |\n| `{stem}_section_flow.mermaid` | Section flowchart with code/link annotations |\n| `{stem}_code_pie.mermaid` | Pie chart of code blocks by language |\n| `{stem}_markpact_graph.mermaid` | Graph of embedded file references |\n| `{stem}_alerts_graph.mermaid` | TOON alerts \u0026 refactor tasks flowchart |\n| `{stem}_workflow.mermaid` | DOQL workflow steps diagram |\n| `dependency_graph.html` | Cross-document dependency graph (directory scan) |\n\n---\n\n## Installation\n\n```bash\n# Clone or copy the mdflow/ directory, then:\npip install -e .\n# No mandatory dependencies — pure stdlib.\n```\n\n---\n\n## Usage\n\n### Python API\n\n```python\nfrom mdflow import MdFlow\n\nflow = MdFlow()\n\n# ── Single file ───────────────────────────────────────────────\ndoc = flow.parse(\"SUMR.md\")\n\nprint(doc.title)                        # \"Ze źródeł\"\nprint(len(doc.headings))               # 24\nprint([ts.name for ts in doc.toon_sections])  # ['HEALTH', 'REFACTOR', ...]\nprint(doc.metadata)                    # {'name': 'redsl', 'version': '1.2.45', ...}\n\n# Access markpact embedded file references\nfor cb in doc.markpact_blocks:\n    print(f\"markpact:{cb.markpact_type}  path={cb.markpact_path}\")\n\n# Get TOON quality metrics\nmetrics = flow.toon_metrics(doc)\nprint(metrics[\"health\"])               # {'cc_mean': 20.0, 'critical': 7}\nprint(metrics[\"refactors\"][:3])        # list of refactor tasks\n\n# Get all Mermaid diagrams as strings (no files written)\ndiagrams = flow.diagrams(doc)\nprint(diagrams[\"section_flow\"])        # flowchart TD ...\n\n# Generate reports to disk\nflow.report(doc, \"output/\")            # writes HTML + MD + .mermaid files\n\n# ── Directory scan ────────────────────────────────────────────\ndocs, graph = flow.scan(\"docs/\", \"output/\")\nprint(f\"{len(docs)} files, {len(graph.edges)} dependency edges\")\n```\n\n### CLI\n\n```bash\n# Analyze a single file\nmdflow analyze SUMR.md --output output/\n\n# Select formats\nmdflow analyze SUMR.md --format html,md\n\n# Scan a directory\nmdflow scan docs/ --output output/\n\n# Print a specific Mermaid diagram to stdout\nmdflow diagram SUMR.md --diagram section_flow\nmdflow diagram SUMR.md --diagram list        # list available diagrams\n\n# Write diagram to file\nmdflow diagram SUMR.md --diagram alerts_graph -o alerts.mermaid\n```\n\n## Mermaid validation\n\nEvery generated `.mermaid` file is automatically validated before writing.\nDetected issues are printed inline and written as tickets to `TODO.md`:\n\n```\n[mdflow] ⚠ 1 error(s) output/SUMR_section_flow.mermaid\n  ✗ [BACKTICK_IN_LABEL] Backtick inside node label (line 5): ...\n[mdflow] → 1 validation ticket(s) written to TODO.md\n```\n\nValidation checks: `EMPTY_DIAGRAM`, `NO_DIAGRAM_TYPE`, `BACKTICK_IN_LABEL`,\n`DUPLICATE_NODE_ID`, `MINDMAP_ILLEGAL_CHARS`.\n\n---\n\n## Quality tooling\n\nmdflow uses [`prefact`](https://github.com/semcod/prefact) and\n[`pyqual`](https://github.com/semcod/pyqual) for automated code quality gates.\n\n```bash\n# Run full quality loop (prefact scan → ruff → pytest → LLM fix on fail)\ntask quality          # alias: pyqual run\n\n# Scan for code issues (duplicate imports, wildcard imports, …)\ntask prefact          # alias: prefact scan -p .\n\n# Auto-fix detected issues\ntask prefact-fix      # alias: prefact fix -p .\n```\n\nA **git pre-commit hook** (`.git/hooks/pre-commit`) runs all checks automatically\nbefore every commit and blocks on failures, writing tickets to `TODO.md`.\n\n---\n\n## Testing\n\n### Unit tests\n\n```bash\npytest tests/ -v\n```\n\n### E2E / CLI tests (TestQL)\n\n142 scenarios covering CLI commands, output file validation, and integration\nwith real semcod workspace projects:\n\n```bash\n# All scenarios\ntask testql-run\n\n# Smoke only (help, subcommands)\ntask testql-smoke\n\n# Full E2E (analyze, scan, diagram, semcod projects, mermaid validation)\ntask testql-e2e\n\n# Single scenario\ntestql run testql-scenarios/02_cli_analyze_e2e.testql.toon.yaml\n```\n\nScenarios in `testql-scenarios/`:\n\n| File | Tests | Scope |\n|---|---|---|\n| `01_cli_help_version` | 16 | help, subcommand help |\n| `02_cli_analyze_e2e` | 35 | analyze: HTML/MD/mermaid output |\n| `03_cli_scan_e2e` | 13 | scan: per_file output, dependency graph |\n| `04_cli_diagram_e2e` | 23 | diagram: list, stdout, file, unknown name |\n| `05_e2e_semcod_projects` | 30 | prefact, pyqual, planfile, goal SUMD.md |\n| `06_e2e_mermaid_validation` | 22 | backtick-free labels, pie title format |\n\n---\n\n## Architecture\n\n```\nmdflow/\n├── __init__.py         ← MdFlow façade (high-level API)\n├── models.py           ← Data classes: MdDocument, DependencyGraph, …\n├── parser.py           ← Core Markdown parser (stdlib only)\n├── validators.py       ← Mermaid diagram validator + TODO.md ticket writer\n├── analyzers/\n│   └── __init__.py     ← DependencyAnalyzer, StructureAnalyzer,\n│                          CodeInventoryAnalyzer, ToonAnalyzer\n├── generators/\n│   ├── __init__.py\n│   ├── mermaid.py      ← All Mermaid diagram generators\n│   ├── html.py         ← Self-contained HTML report (split into helpers)\n│   └── markdown.py     ← Markdown summary report (split into helpers)\n└── cli.py              ← argparse CLI entry point\n```\n\n---\n\n## Examples\n\n### Basic\n\n- **`examples/basic/01_parse_single_file.py`** — Parse and inspect a single document\n- **`examples/basic/02_generate_reports.py`** — Generate HTML, Markdown, and Mermaid reports\n- **`examples/basic/03_diagrams_as_strings.py`** — Get diagrams as strings (no file I/O)\n- **`examples/basic/04_cli_basics.sh`** — CLI: `analyze`, `scan`, `diagram`\n\n### Advanced\n\n- **`examples/advanced/01_directory_scan.py`** — Scan a directory, build dependency graphs\n- **`examples/advanced/02_toon_analysis.py`** — Extract TOON quality metrics\n- **`examples/advanced/03_custom_diagram_pipeline.py`** — Custom HTML with selected diagrams\n\n### API / Extensibility\n\n- **`examples/api/01_low_level_parser.py`** — Use `MdParser` directly\n- **`examples/api/02_custom_analyzer.py`** — Build your own analyzer\n\n### semcod workspace\n\n- **`examples/semcod/analyze_prefact.py`** — Parse `prefact/SUMD.md`, extract TOON metrics\n- **`examples/semcod/scan_semcod_workspace.py`** — Scan 6 semcod projects, cross-project TOON summary\n- **`examples/semcod/toon_comparison.py`** — CC/alerts/refactors comparison table across projects\n- **`examples/semcod/04_cli_semcod.sh`** — CLI shell examples for the semcod workspace\n\n```bash\npython examples/semcod/toon_comparison.py\npython examples/semcod/scan_semcod_workspace.py\n```\n\n---\n\n## Supported TOON sections\n\n`mdflow` recognises these TOON section names inside `toon` / `yaml` code blocks\nand in blocks tagged `markpact:analysis`:\n\n`ALERTS` · `REFACTOR` · `HOTSPOTS` · `HEALTH` · `NEXT` · `RISKS` · `PIPELINES`\n· `DUPLICATES` · `WARNINGS` · `MODULES` · `EVOLUTION` · `COUPLING`\n\n---\n\n## Extension points\n\n- **Custom extractor**: subclass or monkey-patch `MdParser`\n- **Custom diagram**: call `flow.diagrams(doc)` and extend the `mermaid` module\n- **Graphviz output**: install `graphviz` Python package and use\n  `DependencyGraph` data directly\n\n---\n\n## License\n\nLicensed under Apache-2.0.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsemcod%2Fmdflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsemcod%2Fmdflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsemcod%2Fmdflow/lists"}