{"id":42341533,"url":"https://github.com/stefapi/markdown_aggregator","last_synced_at":"2026-01-27T15:03:37.044Z","repository":{"id":329350576,"uuid":"1086439465","full_name":"stefapi/markdown_aggregator","owner":"stefapi","description":"Aggregate markdown trees or manifests into a single document with a generated table of contents.","archived":false,"fork":false,"pushed_at":"2025-12-30T12:11:14.000Z","size":25,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-01-03T03:25:05.778Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stefapi.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-30T12:24:50.000Z","updated_at":"2025-12-30T12:11:18.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/stefapi/markdown_aggregator","commit_stats":null,"previous_names":["stefapi/markdown_aggregator"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/stefapi/markdown_aggregator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefapi%2Fmarkdown_aggregator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefapi%2Fmarkdown_aggregator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefapi%2Fmarkdown_aggregator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefapi%2Fmarkdown_aggregator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stefapi","download_url":"https://codeload.github.com/stefapi/markdown_aggregator/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefapi%2Fmarkdown_aggregator/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28815385,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-27T12:25:15.069Z","status":"ssl_error","status_checked_at":"2026-01-27T12:25:05.297Z","response_time":168,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-27T15:03:23.428Z","updated_at":"2026-01-27T15:03:37.039Z","avatar_url":"https://github.com/stefapi.png","language":"Python","readme":"# markdownaggregator\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://img.shields.io/pypi/v/markdownaggregator.svg?style=for-the-badge\" alt=\"PyPI\"\u003e\n  \u003cimg src=\"https://img.shields.io/pypi/pyversions/markdownaggregator.svg?style=for-the-badge\" alt=\"Python versions\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/type%20hints-100%25-4B8BBE?style=for-the-badge\" alt=\"Typing\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/CI-GitHub%20Actions-2088FF?style=for-the-badge\u0026logo=githubactions\" alt=\"CI\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/License-MIT-2EA043?style=for-the-badge\" alt=\"License\"\u003e\n\u003c/div\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cem\u003eAssemble Markdown forests into polished, navigable documents — CLI or Python, batteries included.\u003c/em\u003e\n\u003c/p\u003e\n\n---\n\n- **Status**: Beta – ready for everyday use, API kept stable for the 0.x series\n- **Docs**: [`docs/usage.md`](docs/usage.md)\n\n---\n\n## 🌟 Why Markdown Aggregation Should Be Effortless\n\n| Challenge | markdownaggregator Solution | Modes |\n| --------- | -------------------------- | ----- |\n| Maintaining a knowledge base scattered across folders | Deterministic traversal with manifest or auto-discovery | CLI / Library |\n| Producing customer-ready guides from technical notes | Automatic TOC, heading normalization, source breadcrumbs | CLI / Library |\n| Reusing documentation snippets via `@include` | Recursive include resolution with cycle detection | CLI / Library |\n| Mixing curated order and “everything else” | Hybrid mode merges manifest priority with discovery | CLI |\n| Cleaning YAML front-matter for publication | One flag to strip it before merge | CLI / Library |\n| Automating build pipelines | Pure Python function `aggregate_markdown` + zero I/O side effects | Library |\n\n## 🧭 User Journeys\n\n### 1. Documentation Lead (CLI)\n\n1. Curate `manifest.txt` describing the expected order.\n2. Run `markdownaggregator docs --manifest manifest.txt --process-includes`.\n3. Commit the generated `dist/documentation.md` and ship the release notes.\n\n### 2. Platform Engineer (Python Library)\n\n1. Import `aggregate_markdown` in a build step.\n2. Feed it the docs root, manifest and ignore patterns.\n3. Pipe the returned string into your static site generator.\n\n---\n\n## ⚡ Quickstart\n\n### CLI\n\n```bash\npip install markdownaggregator\n\n# Aggregate docs into dist/docs.md\nmarkdownaggregator docs/ \\\n  --manifest manifest.txt \\\n  --process-includes \\\n  --toc \\\n  --strip-frontmatter \\\n  --output dist/docs.md\n```\n\n**Pro tips**\n\n- Repeat `--ignore` for glob patterns (e.g. `drafts/*`).\n- Use `--hybrid-mode` to append undiscovered files after the manifest order.\n- Disable separators with `--no-separator` or customize via `--separator`.\n\n### Python API\n\n```python\nfrom pathlib import Path\nfrom markdownaggregator import aggregate_markdown\n\nmerged = aggregate_markdown(\n    root=Path(\"docs\"),\n    manifest=Path(\"manifest.txt\"),\n    ignore=[\"drafts/*\", \"archive/**\"],\n    separator=\"---\",\n    strip_frontmatter_from_files=True,\n    hybrid_mode=True,\n    process_includes_flag=True,\n    include_toc=True,\n    auto_file_title=True,\n    output=Path(\"dist/guide.md\"),  # optional\n)\n\nprint(merged[:200])  # preview\n```\n\n- When `output` is provided, the file is written in addition to returning the string.\n- All parameters are keyword-only for clarity and forward compatibility.\n\n---\n\n## 🏗️ Architecture at a Glance\n\n```\nmarkdownaggregator/\n├── aggregator.py    # core domain logic (discovery, merge, TOC, includes)\n├── cli.py           # argparse-powered interface \u0026 logging\n├── __main__.py      # enables `python -m markdownaggregator`\n└── __init__.py      # re-exports aggregate_markdown \u0026 package metadata\n```\n\nSupporting assets:\n\n- `docs/usage.md` — extended scenarios and tips  \n- `tests/test_aggregator.py` — pytest coverage for discovery, manifest, includes, CLI  \n- `.github/workflows/release.yml` — PyPI publish pipeline (GitHub release driven)\n\n---\n\n## 📚 Documentation Index\n\n- [Usage guide](docs/usage.md) – scenarios, ignores, includes, logging\n- [CHANGELOG](CHANGELOG.md) – Keep a Changelog format\n- [RELEASING](RELEASING.md) – how to tag, build, verify and publish on PyPI\n- [CONTRIBUTING](CONTRIBUTING.md) – workflow, tooling, standards\n- [CODE_OF_CONDUCT](CODE_OF_CONDUCT.md) – Contributor Covenant v2.1\n\n---\n\n## 🛠️ Tech Stack \u0026 Tooling\n\n| Area | Tooling |\n| ---- | ------- |\n| Language | Python ≥ 3.10 (typed, `mypy --strict`) |\n| CLI | `argparse`, rich logging, exit codes |\n| Packaging | `setuptools`, `MANIFEST.in`, console script `markdownaggregator` |\n| Quality | `pytest`, `pytest-cov`, `ruff` (lint + format), `mypy`, `pre-commit` |\n| CI/CD | GitHub Actions – unit tests + PyPI publishing workflow |\n\n---\n\n## 🗺️ Roadmap\n\n- [ ] HTML/PDF export helpers\n- [ ] Incremental rebuild mode (skip unchanged files)\n- [ ] Optional YAML-driven configuration file\n- [ ] Pluggable include resolvers (HTTP, git URLs)\n- [ ] Rich logging formatter (colors / verbosity presets)\n\nContribute to the roadmap by opening GitHub issues with the `enhancement` label.\n\n---\n\n## 💻 Development Workflow\n\n```bash\ngit clone https://github.com/stefapi/markdownaggregator\ncd markdownaggregator\n\npip install -e \".[dev]\"\npre-commit install\n\nruff check .\nruff format .\nmypy .\npytest\n```\n\nRun the CLI locally:\n\n```bash\npython -m markdownaggregator docs --output dist/out.md\n```\n\n---\n\n## 🚀 Release \u0026 PyPI Publishing\n\n1. Update `pyproject.toml` version + `CHANGELOG.md`.\n2. Run local validations: `pre-commit run --all-files`, `pytest`, `mypy .`.\n3. Build artifacts: `python -m build` (creates `dist/*.whl` and `*.tar.gz`).\n4. Option A — **Manual**: `twine upload dist/*`.\n5. Option B — **Automated**: push tag `vX.Y.Z`, publish GitHub release, let the workflow `Publish to PyPI` deploy using `PYPI_API_TOKEN`.\n\nFull checklist in [RELEASING.md](RELEASING.md).\n\n---\n\n## 🤝 Community, Contributing \u0026 Support\n\nWe welcome feature ideas, bug reports, docs tweaks and new tests.\n\n- Read [CONTRIBUTING.md](CONTRIBUTING.md) for branch naming, tooling and review expectations.\n- Respect the [Code of Conduct](CODE_OF_CONDUCT.md) — contact `stephane_nospam@apiou.org` for incident reports.\n- Discussions happen through GitHub issues and pull requests.\n\n---\n\n## 📄 License\n\nMIT © 2025 Stefapi — see [LICENSE](LICENSE) for full text.\n\nIf markdownaggregator streamlines your doc workflows, mention it in your release notes or give it a ⭐️ on GitHub. Merci !\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstefapi%2Fmarkdown_aggregator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstefapi%2Fmarkdown_aggregator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstefapi%2Fmarkdown_aggregator/lists"}