https://github.com/makukha/docsub
Embed text and data into Markdown files
https://github.com/makukha/docsub
docs documentation template-engine
Last synced: 4 months ago
JSON representation
Embed text and data into Markdown files
- Host: GitHub
- URL: https://github.com/makukha/docsub
- Owner: makukha
- License: mit
- Created: 2024-12-28T10:36:50.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-02-21T10:05:51.000Z (over 1 year ago)
- Last Synced: 2025-12-01T11:34:31.586Z (7 months ago)
- Topics: docs, documentation, template-engine
- Language: Python
- Homepage:
- Size: 269 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 9
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# docsub
> Substitute dynamically generated content in Markdown files
[](https://github.com/makukha/docsub/blob/main/LICENSE)
[](https://pypi.python.org/pypi/docsub)
[](https://pypi.org/project/docsub)
[](https://github.com/makukha/docsub)
[](https://github.com/makukha/docsub)
[](https://github.com/makukha/multipython)
[](https://github.com/makukha/docsub)
[](http://mypy.readthedocs.io)
[](https://github.com/astral-sh/ruff)
[](https://github.com/astral-sh/ruff)
> [!WARNING]
> * With `docsub`, every documentation file may become executable.
> * Never use `docsub` to process files from untrusted sources.
> * This project is in experimental state, syntax and functionality may change significantly.
> * If still want to try it, use pinned package version `docsub==0.9.0`
# Features
* Embed **static** files
* Embed **command execution** results
* **Idempotent** substitutions
* **Invisible** non-intrusive markup using comment blocks
* **Plays nicely** with other markups
* **Extensible** with project-local commands
* **Configurable** with config files and env vars
# Use cases
* Manage partially duplicate docs for multiple destinations
* Manage docs for monorepositories
* Embed CLI reference in docs
* Embed dynamically generated content:
* Project metadata
* Test reports
* Models evaluation results
> [!NOTE]
> This file uses docsub itself. Dig into raw markup if interested.
## Docsub is not...
* ...a documentation engine like [Sphinx](https://www.sphinx-doc.org) or [MkDocs](https://www.mkdocs.org)
* ...a full-featured static website generator like [Pelican](https://getpelican.com)
* ...a templating engine like [Jinja](https://jinja.palletsprojects.com)
* ...a replacement for [Bump My Version](https://callowayproject.github.io/bump-my-version)
# Usage
```shell
$ uv run docsub sync -i README.md
```
## From separate files...
### README.md
````markdown
# Title
...
## Table
| Col 1 | Col 2 |
|-------|-------|
...
## Code
```python
...
```
````
### info.md
````markdown
> Long description.
````
### features.md
````markdown
* Feature 1
* Feature 2
* Feature 3
````
### data.md
````markdown
| Key 1 | value 1 |
| Key 2 | value 2 |
| Key 3 | value 3 |
````
### func.py
````python
def func():
pass
````
## Get merged document
***and keep it updated!***
````markdown
# Title
> Long description.
* Feature 1
* Feature 2
* Feature 3
## Table
| Col 1 | Col 2 |
|-------|-------|
| Key 1 | value 1 |
| Key 2 | value 2 |
| Key 3 | value 3 |
## Code
```python
def func():
pass
```
````
# Installation
## Development dependency
Recommended. The most flexible installation option, allowing [project-local commands](#project-local-commands) to utilize project codebase.
```toml
# pyproject.toml
[dependency-groups]
dev = [
"docsub==0.9.0",
]
```
## Global installation
Works for simple cases.
```shell
uv tool install docsub==0.9.0
```
# Syntax
The syntax is purposefully verbose. This is fine, you are not supposed to edit it often. But it's searchable and sticks in eye when scrolling down large documents.
Docsub uses line-based substitution syntax based on *directives* and *substitution blocks*.
## Markdown
### Directive
*Markdown directive* is one-line comment:
```text
```
There are multiple [directive types](#directives).
### Substitution block
*Markdown substitution block* is a sequence of lines, starting with `begin` directive and ending with `end` directive.
```markdown
Inner text will be replaced.
This text will be replaced too.
```
One or many other directives must come at the top of the block, otherwise they are treated as plain text. Blocks without *producing directives* are not allowed. Block's inner text will be replaced upon substitution, unless modifier directives are used, e.g. `lines`.
If docsub substitution block lies inside markdown fenced code block, it is not substituted *(example: fenced code blocks above and below this paragraph, see the raw markup)*. To put dynamic content into a fenced code block, place `begin` and `end` around it and use `lines after N upto -M` *(example: [Usage](#usage) section)*.
For nested blocks, only top level substitution is performed. Use block `#identifier` to distinguish between nesting levels.
```markdown
```
When substitution block is indented, the indentation is preserved:
```markdown
* List item
* Sub-item 1
* Sub-item 2
* Sub-item 3
```
# Directives
* *Block delimiters*: `begin`, `end`
* *Producing commands*: `exec`, `help`, `include`, `x`
* *Modifying commands*: `lines`, `strip`
## `begin`
```text
begin [#identifier]
```
Open substitution target block. To distinguish between nesting levels, use block `#identifier`, starting with `#`.
## `end`
```text
end [#identifier]
```
Close substitution target block.
## `exec`
```text
exec
```
Execute `` with `sh -c` and substitute stdout. Allows pipes and other shell functionality. If possible, avoid using this directive.
* `cmd.exec.work_dir` — shell working directory, default `'.'`
* `cmd.exec.env_vars` — dict of additional environment variables, default `{}`
## `help`
```text
help [subcommand...]
help python -m [subcommand...]
```
Display help for CLI utility or Python module. Use this command to document CLI instead of `exec`. Runs `command [subcommand...] --help` or `python -m command [subcommand...] --help` respectively. *Directive args* must be a space-separated sequence of characters `[-._a-zA-Z0-9]`.
* `cmd.help.env_vars` — dict of additional environment variables, default `{}`
## `include`
```text
include path/to/file
```
Literally include file specified by path relative to `base_dir` config option.
* `cmd.include.base_dir` — base directory for relative paths
## `lines`
```text
lines [after N] [upto -M]
```
Upon substitution, keep original target block lines: first `N` and/or last `M`. Only one `lines` command is allowed inside the block.
## `strip`
```text
strip
```
Strip whitespace in substitution result:
* initial and trailing blank lines
* trailing whitespace on every line
## `x`
```text
x [args and --options]
```
Execute [project-local](#project-local-commands) command declared in `docsubfile.py` in project root. The naming is inspired by `X-` HTTP headers and `x-` convention for reusable YAML sections.
* `cmd.x.docsubfile` — path to file with project-local commands, absolute or relative to project root (default: `docsubfile.py`)
# Project-local commands
When project root contains file `docsubfile.py` with commands defined as in example below, they can be used in `docsub: x ` directive. Project commands must be defined as [click](https://click.palletsprojects.com) command and gathered under `x` group. There is no need to install `click` separately as docsub depends on it.
If docsub is installed globally and called as `uvx docsub`, project commands in `docsubfile.py` have access to docsub dependencies only: `click`, `loguru`, `rich` (see docsub's pyproject.toml for details).
If docsub is installed as project dev dependency and called as `uv run docsub`, user commands also have access to project modules and dev dependencies. This allows more flexible scenarios.
Project command author can get access to docsub `Environment` object (including command configs) from click context object (see example below). The docsub `Environment` object has some useful methods *(not documented yet)*.
## Example
```shell
$ uv run docsub sync -i sample.md
```
### sample.md
```markdown
Hi there, Alice!
Hi there, Bob!
```
### docsubfile.py
```python
from docsub import Environment, click, pass_env
@click.group()
def x() -> None:
pass
@x.command()
@click.argument('users', nargs=-1)
def say_hello(users: tuple[str, ...]) -> None:
for user in users:
click.echo(f'Hi there, {user}!')
@x.command()
@click.argument('users', nargs=-1)
@pass_env
def log_hello(env: Environment, users: tuple[str, ...]) -> None:
base = env.get_temp_dir('log_hello')
(base / 'hello.log').write_text(f'said hello to {users}')
```
## Calling project-local commands
Docsub exposes `x` as CLI command, letting project commands to be executed with project settings:
```shell
$ uv run docsub x say-hello Alice Bob
Hi there, Alice!
Hi there, Bob!
```
# Configuration
Configuration resolution order
* command line options *(to be documented)*
* environment variables *(to be documented)*
* `docsub.toml` config file in current working directory
* `pyproject.toml`, section `[tool.docsub]` *(to be implemented)*
* default config values
## Root settings
* `local_dir` — internal working directory at the project root (default: `.docsub`)
## Command settings
See [Commands](#commands).
## Environment variables
*(to be documented)*
## Command line options
*(to be documented)*
## Complete config example
All config keys are optional.
```toml
local_dir = ".docsub" # default
[logging]
#level = "DEBUG" # default: missing, logging disabled
[cmd.exec]
env_vars = {} # default
work_dir = "." # default
[cmd.help.env_vars]
COLUMNS = "60" # more compact
[cmd.include]
base_dir = "." # default
[cmd.x]
docsubfile = "docsubfile.py" # default
```
> [!WARNING]
> In future releases config keys will be moved under `[tool.docsub]` root for both `pyproject.toml` and `docsub.toml`, this will be a breaking change.
# Logging
Docsub uses [loguru](https://loguru.readthedocs.io) for logging. Logging is disabled by default. To enable logging, set config option `level` to one of [logging levels](https://loguru.readthedocs.io/en/stable/api/logger.html#levels) supported by loguru.
*(logging is rudimentary at the moment)*
# CLI Reference
```shell
$ docsub --help
Usage: python -m docsub [OPTIONS] COMMAND [ARGS]...
╭─ Options ──────────────────────────────────────────────────────────╮
│ --config-file -c PATH │
│ --local-dir -l PATH │
│ --cmd-exec-work-dir PATH │
│ --cmd-exec-env-vars TEXT │
│ --cmd-help-env-vars TEXT │
│ --cmd-include-base-dir PATH │
│ --cmd-x-docsubfile -x PATH │
│ --version Show the version and exit. │
│ --help Show this message and exit. │
╰────────────────────────────────────────────────────────────────────╯
╭─ Commands ─────────────────────────────────────────────────────────╮
│ sync Update Markdown files with embedded content. │
│ x Project-local commands. │
╰────────────────────────────────────────────────────────────────────╯
```
## `docsub sync`
```shell
$ docsub sync --help
Usage: python -m docsub sync [OPTIONS] FILES...
Update Markdown files with embedded content.
Read FILES and perform substitutions one by one. If one file depends
on another, place it after that file.
╭─ Options ──────────────────────────────────────────────────────────╮
│ --in-place -i Process files in-place │
│ --help Show this message and exit. │
╰────────────────────────────────────────────────────────────────────╯
```
## `docsub x`
```shell
$ docsub x --help
Usage: python -m docsub x [OPTIONS] COMMAND [ARGS]...
Project-local commands.
╭─ Options ──────────────────────────────────────────────────────────╮
│ --help Show this message and exit. │
╰────────────────────────────────────────────────────────────────────╯
```
# History
This project appeared to maintain docs for [multipython](https://github.com/makukha/multipython) project. You may check it up for usage examples.
# Authors
* [Michael Makukha](https://github.com/makukha)
# License
[MIT License](https://github.com/makukha/docsub/blob/main/LICENSE)
# Changelog
[CHANGELOG.md](https://github.com/makukha/docsub/blob/main/CHANGELOG.md)