{"id":13537146,"url":"https://github.com/bashup/mdsh","last_synced_at":"2025-05-07T00:00:40.897Z","repository":{"id":37390676,"uuid":"99276039","full_name":"bashup/mdsh","owner":"bashup","description":"Multi-lingual, Markdown-based Literate Programming... in run-anywhere bash","archived":false,"fork":false,"pushed_at":"2022-07-01T11:01:36.000Z","size":151,"stargazers_count":177,"open_issues_count":0,"forks_count":15,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-10-09T13:14:00.179Z","etag":null,"topics":["bash","literate-programming","markdown","scripting-language","shell-scripting"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bashup.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-08-03T21:20:54.000Z","updated_at":"2024-10-09T08:29:53.000Z","dependencies_parsed_at":"2022-07-07T23:09:30.111Z","dependency_job_id":null,"html_url":"https://github.com/bashup/mdsh","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bashup%2Fmdsh","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bashup%2Fmdsh/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bashup%2Fmdsh/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bashup%2Fmdsh/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bashup","download_url":"https://codeload.github.com/bashup/mdsh/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252788514,"owners_count":21804284,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bash","literate-programming","markdown","scripting-language","shell-scripting"],"created_at":"2024-08-01T09:00:55.643Z","updated_at":"2025-05-07T00:00:40.839Z","avatar_url":"https://github.com/bashup.png","language":"Shell","readme":"## Multi-Lingual Literate Programming with `mdsh`\n\n`mdsh` is a bash script compiler and interpreter for markdown files.  It can be used in a `#!` line to make markdown files executable, or it can be used as a standalone tool to generate dependency-free, distributable bash scripts from markdown files.\n\nBy default, `mdsh` only considers `shell` code blocks to be bash code, but you can also use `@mdsh` blocks to define handlers for other languages.  For example, this script will run `python`-tagged code blocks by piping them to the `python` command:\n\n~~~markdown\n#!/usr/bin/env mdsh\n\n# Hello World in Python\n\nThe following code block is executed at compile time (due to the `@mdsh`).\n(The first word on the opening line could be `shell` or `sh` or anything\nelse, as long as the second word is `@mdsh`.)\n\n```bash @mdsh\nmdsh-lang-python() { python; }\n```\n\nNow that we've defined a language handler for `python`, this next code\nblock is translated to shell code that runs python with the block's\ncontents on stdin:\n\n```python\nprint(\"hello world!\")\n```\n~~~\n\nRunning the above markdown file produces the same results as this equivalent bash script:\n\n~~~bash\n#!/usr/bin/env bash\n{ python; } \u003c\u003c'```'\nprint(\"hello world!\")\n```\n~~~\n\n`mdsh` supports processing blocks of any language that you can write a bash code snippet for, and even lets you write \"compile-time\" code to transform blocks containing metadata or DSL snippets into bash code.  The results can be either executed on the fly for development, or deployed/distributed via `mdsh --compile`.  Compiled scripts do not include any `@mdsh` code, nor do they have any hidden runtime dependencies: everything `mdsh --compile` outputs is code or data you gave it, or generated by bash code you gave it!\n\n**Contents**\n\n\u003c!-- toc --\u003e\n\n- [Installation](#installation)\n- [Usage](#usage)\n  * [Data Blocks](#data-blocks)\n  * [Processing Non-`shell` Languages](#processing-non-shell-languages)\n  * [Advanced Block Compilation Techniques](#advanced-block-compilation-techniques)\n    + [Compile-Time Variables](#compile-time-variables)\n    + [Programmatic Block Generation](#programmatic-block-generation)\n  * [Command Blocks and Arguments](#command-blocks-and-arguments)\n- [Tips and Techniques](#tips-and-techniques)\n  * [Literate Testing](#literate-testing)\n  * [Excluding Blocks From The Generated Script](#excluding-blocks-from-the-generated-script)\n  * [Making Executable (and Editable) Markdown Files](#making-executable-and-editable-markdown-files)\n  * [Making Sourceable Scripts (and handling $0)](#making-sourceable-scripts-and-handling-0)\n  * [Syntax Highlighting and Language Aliasing](#syntax-highlighting-and-language-aliasing)\n- [Metaprogramming and Code Generation](#metaprogramming-and-code-generation)\n  * [\"Static Linking\" for Distribution](#static-linking-for-distribution)\n- [Extending `mdsh` or Reusing its Functions](#extending-mdsh-or-reusing-its-functions)\n  * [Adding File Headers or Footers](#adding-file-headers-or-footers)\n  * [Altering Existing Functions](#altering-existing-functions)\n  * [Available Functions](#available-functions)\n\n\u003c!-- tocstop --\u003e\n\n## Installation\n\n`mdsh` can be installed in any of the following ways:\n\n* Using [basher](https://github.com/basherpm/basher), via `basher install bashup/mdsh`\n* Using [composer](https://getcomposer.org/), via `composer require bashup/mdsh:dev-master` (to add it to your project) or `composer global require bashup/mdsh:dev-master` (to install it globally)\n* Using git, by cloning this repo and copying or linking the `bin/mdsh` file to a directory on your `PATH`, or\n* Just [downloading the script directly](https://github.com/bashup/mdsh/raw/master/bin/mdsh) to a directory on your `PATH`, then running `chmod +x` on it)\n\n## Usage\n\nRunning `mdsh` *markdownfile args...* will read and translate unindented, triple-backquote fenced code blocks from *markdownfile* into bash code, based on the language listed on the block and any translation rules you've defined.  The resulting translated script is then run, passing in *args* as positional arguments to the script.\n\nBlocks tagged as `shell` are interpreted as bash code, and directly copied to the translated script.  So arguments passed to `mdsh` after the path to the markdown file are available as `$1`, `$2`, etc. within the top-level code of `shell` blocks, just like in a regular bash script.\n\n(Typically, you won't run `mdsh` directly, but will put `#!/usr/bin/env mdsh` on the first line of your markdown file instead, and make it executable with `chmod +x`.  That way, users of your script won't need to do anything special to run it.)\n\nYou can also use `mdsh --compile` *file1 file2...* to translate one or more markdown files to bash code, sending the result to stdout.  (A filename of `-` means \"read from standard input\".)  This can be useful for debugging, or to make a distributable version of your script that does not require its users to have `mdsh`.\n\n(There is also an `mdsh --eval` *filename* option, which is similar to `--compile`, but only takes one, non-stdin file, and emits special code at the end to support markdown files being sourced; see the section below on [Making Sourceable Scripts](#making-sourceable-scripts-and-handling-0) for more details.)\n\nBoth `--eval` and `--compile` can be preceded with `--out` *filename*, in which case *filename*'s contents will be replaced with `mdsh`'s output, if and only if the compilation or run succeeds without any errors.  (The output is buffered in-memory, then output all at once upon successful completion.   If the file already existed, its permissions will remain unchanged.)\n\n### Data Blocks\n\nThe contents of unindented, triple-backquoted blocks that are *not* tagged `shell` or `shell @mdsh` are treated as *data* by default: their contents are added to bash arrays named according to the language on the block, e.g.:\n\n~~~markdown\n# Data Arrays Example\nBlocks without a defined language processor get translated to a variable\nassignment like `mdsh_raw_json+=(\\#\\ block\\ 0)` at that point in the\ngenerated script:\n\n```json\n{ \"hello\": \"world\" }\n```\n```shell\necho \"${mdsh_raw_json[0]}\"   # prints '{ \"hello\": \"world\" }'\n```\n```json\n{ \"this is\": \"great\" }\n```\n```shell\necho \"${mdsh_raw_json[0]}\"   # prints '{ \"hello\": \"world\" }'\necho \"${mdsh_raw_json[1]}\"   # prints '{ \"this is\": \"great\" }'\n```\n\n## Naming Rules\nLanguage names are *case sensitive*, and non-identifier\ncharacters in language names become `_` in variable names:\n\n```C++\n// hey\n```\n```shell\necho \"${mdsh_raw_C__[0]}\"   # prints '// hey'\n```\n~~~\n\nOf course, it would be even better if you could automate the processing of these blocks, so you don't have to follow every block with another `shell` block to process it!  Which is why the next section is on...\n\n### Processing Non-`shell` Languages\n\nTo automate the handling of non-`shell` language blocks, you can define one or more `@mdsh` blocks, containing \"hook functions\".   `@mdsh` blocks are a bit like a Makefile, in that they define *rules* for how to *build* parts of your script, based on the language used.\n\nThese build rules are specified by defining specially-named bash functions.  Unlike functions in `shell` blocks, these functions are *not* part of your script and therefore can't be called directly.  Instead, `mdsh` itself invokes them (or copies out their source code), whenever a subsequent block's language matches one of the functions' names.\n\nThe language of a markdown code block is normally just one word after its opening backquotes.  But if more than one word appears, then mdsh considers the language to be either the second word (if it begins with `@`), or the entire line flattened into a single variable name.  Some example translations:\n\n| Block Opening             | Effective Language |                                               Function Names |\n| ------------------------- | ------------------ | -----------------------------------------------------------: |\n| `` ```C++ ``              | `C++`              |                      `mdsh-lang-C++`\u003cbr /\u003e`mdsh-compile-C++` |\n| `` ```C++ example``       | `C___example`      |      `mdsh-lang-C___example`\u003cbr /\u003e`mdsh-compile-C___example` |\n| `` ```foo bar.baz spam``  | `foo_bar_baz_spam` | `mdsh-lang-foo_bar_baz_spam`\u003cbr /\u003e`mdsh-compile-foo_bar_baz_spam` |\n| `` ```foo @bar.baz spam`` | `bar.baz`          |              `mdsh-lang-bar.baz`\u003cbr /\u003e`mdsh-compile-bar.baz` |\n| `` ```shell script ``     | `shell_script`     |    `mdsh-lang-shell_script`\u003cbr /\u003e`mdsh-compile-shell_script` |\n| `` ```shell @mdsh``       | `mdsh`             |                    `mdsh-lang-mdsh`\u003cbr /\u003e`mdsh-compile-mdsh` |\n\nFunction names are interpreted as follows:\n\n* An `mdsh-lang-X` function is a template for code to be run when a block of language `X` is encountered.  Its function body is copied to the translated script as a bash compound statment (i.e. in curly braces`{...}`) , that will execute with the block contents as on its standard input.  (Its standard output is the same as the overall script's.)\n* An `mdsh-compile-X` function is invoked *at compile time* with the block contents as `$1`, and must output a bash source code translation of the block on its stdout.  The block's full original language tag is in `$2`, and the code block's starting line number is in `$3`.\n* If neither an `mdsh-lang-X` nor `mdsh-compile-X` function exists, `mdsh-misc` is invoked *at compile time* with the raw language tag as `$1` and the block contents as `$2`.  The output of `mdsh-misc` will be added to the compiled script.  (The default implementation of `mdsh-misc` outputs code to save the block contents in a variable, as described above in the [Data Blocks](#data-blocks) section, above.)\n* An `mdsh-after-X` function is a template for code to be run *after* a block of language `X` is encountered.  Its function body is copied to the translated script as a block just after the `mdsh-lang-X` body, `mdsh-compile-X` output, or `mdsh_raw_X+=(contents)` statement.  It does *not* receive the block source, so its standard input and output are those of the script itself.\n\nIf both an `mdsh-lang-X` and `mdsh-compile-X` function exist, `mdsh-lang-X` takes precedence.  Defining either one also disables the `$mdsh_raw_X` functionality: only untranslatable \"data\" blocks are added to the arrays.\n\nIf there is no `mdsh-lang-X` or `mdsh-compile-X` however, the `mdsh-after-X` function can read the most recent block's contents from `${mdsh_raw_VARNAME[-1]}` (unless you've replaced the default `mdsh-misc` implementation).  If you don't unset the array, it will keep growing as more blocks of that language are encountered.\n\nNote: these function names are **case sensitive**, so a block tagged with an uppercase `C` will not trigger the same functions as a block tagged with a lowercase `c`, or vice-versa.  Also, note that because `mdsh` blocks are executed at compile time, they do **not** have access to the script's arguments or I/O: all you can do in them is define hook functions.\n\nFinally, please remember that you usually shouldn't put any code in an `@mdsh` block aside from hook functions, unless you're intentionally doing [metaprogramming or code generation](#metaprogramming-and-code-generation).  That's because `@mdsh` blocks are *not* part of the translated script, they are part of the *translation process*.  So any functions you define in them won't be around when the script actually runs, and any changes you make to variables won't be still around when the actual script execution happens.\n\n### Advanced Block Compilation Techniques\n\nOnce you've gotten used to doing some `mdsh-lang-X` functions, why not try your hand at some `mdsh-compile` ones?\n\nFor example, in the [`jqmd` project](https://github.com/bashup/jqmd), I originally had some code that looked like this:\n\n```bash\nYAML() { JSON \"$(echo \"$1\" | yaml2json -)\"; }\n\nmdsh-lang-yaml() { YAML \"$(cat)\"; }\n```\n\nWhich works pretty well, except, since the YAML is a constant value, why not convert it to JSON during compilation?  That way, we could eliminate the runtime overhead (if we save and rerun the compiled script):\n\n```bash\nmdsh-compile-yaml() { printf 'JSON %q\\n' \"$(echo \"$1\" | yaml2json)\"; }\n```\n\nNotice the difference between the two functions: the `lang` function is a code *template*.  `mdsh` copies its body into your script source, resulting in code that looks like:\n\n```bash\n{\n    YAML \"$(cat)\"\n} \u003c\u003c'```'\n... yaml data here ...\n​```\n```\n\nBut the `compile` function simply runs `yaml2json` immediately, and then writes out the translated data, like so:\n\n```bash\nJSON ...shell-quoted json here...\n```\n\nNotice the use of `printf` with `%q` -- this results in the data being properly escaped to work as a command line argument.  (Take care when you do direct code generation to escape such values properly.  When you need to insert variable data into generated code, always use `printf` with a constant string format, with `%q` placeholders for any standalone arguments.)\n\nNotice too, by the way, that `compile` functions get access to the actual block text, which means that you can do any sort of code generation you like.  For example, I could have taken the output of `yaml2json`, and run `jq` over it, then looped over the output and written bash code to set variables based on the result, or generated code for subcommands based on the specification, or maybe even generated an argument parser from it.  There are all sorts of interesting possibilities for these kinds of code generation techniques!\n\n#### Compile-Time Variables\n\nIn addition to their positional arguments, compile-time hooks such as  `mdsh-misc` and  `mdsh-compile-X` also receive a few variables that can be helpful for parsing special block headers or generating error messages:\n\n* `${tag_words[@]}` is an array of the whitespace-separated words from the original block opening line.  For example, if a block opened with `` ```foo @bar.baz spam ``, then `tag_words=([0]=\"foo\" [1]=\"@bar.baz\" [2]=\"spam\")`.  (`${#tag_words[@]}` is the number of words.)\n* `$mdsh_lang` is the language of the block as viewed by mdsh -- i.e., the `X` in `mdsh-lang-X`.  (So it's either `${tag_words[0]}`, `${tag_words[1]#@}`, or the entire line contents with non-identifier characters replaced by `_`.)\n* If the source being compiled is a file, `$MDSH_SOURCE` is the source filename.\n* `$block_start` is the starting line number of the block in the original source.\n* `$mdsh_block` contains the text of the block\n* `$mdsh_tag` contains the original block opening line (i.e., the unsplit form of tag_words)\n\n(These variables are also usable by compile-time command blocks, as described in the next section.)\n\n#### Programmatic Block Generation\n\nThe `mdsh-block` function allows you to programmatically generate a code block of a designated language.  This can be useful for e.g. conditional blocks.  For example, this `if-env` function can be used in a command block to generate code that will check the value of `$WP_ENV` at runtime and conditionally execute the block's contents:\n\n~~~markdown\n```shell @mdsh\nif-env() {\n   printf -v REPLY '|%q' \"$@\"\n   echo \"case \\$WP_ENV in ${REPLY#|})\"\n   mdsh-block \"$mdsh_lang\" \"$mdsh_block\" \"$block_start\"\n   echo\n   echo \"esac\"\n}\n```\n\n```css !if-env dev staging\n/* This CSS is only used in dev and staging */\n```\n~~~\n\nThe `mdsh-block` function takes up to four arguments: a language, a block body, a starting line number, and a \"raw\" language tag (which defaults to the language if not given).  The first three arguments are also optional, defaulting to `$mdsh_lang`, `$mdsh_block`, and `$block_start` if omitted.  (Which means the above code could have just called `mdsh-block` with no arguments!)\n\n`mdsh-block` follows the standard language lookup logic, looking first for `mdsh-lang-X`, then `mdsh-compile-X`, and then falling back to `mdsh-misc`, cloning `mdsh-after-X` as well if applicable.  It does not support command blocks or language aliases, so no `@` ,`+`, `!`, or `|` expressions can be used.  It's intended for use in compile-time code only, i.e. `!` command blocks, `@mdsh` blocks, and handlers like `mdsh-misc` and `mdsh-compile-X` functions.\n\n### Command Blocks and Arguments\n\nSometimes you have only one block that needs to be processed in a particular way, or each block of a particular language needs unique arguments to compile or execute.  For these scenarios, you can define \"command blocks\".\n\nA command block is a code block whose language tag's *second word* begins with a `|`, `+`, or `!`:\n\n* If it's a `|`, the remainder of the language tag is executed at runtime with the block's contents on standard input (just like an `mdsh-lang-X` function body), and the shell variable `mdsh_lang` set to the first word of the language tag.\n* If it's a `+`, the remainder of the language tag is executed at runtime with the block's contents as an extra command line argument, and the shell variable `mdsh_lang` set to the first word of the language tag.\n* If it's a `!`, the remainder of the language tag is executed at **compile** time with the block's contents in `$1`, and must output compiled code to standard output (just like an `mdsh-compile-X` function).  The full language tag is in `$2`, and the code block's starting line number is in `$3`.  All of the standard [compile-time variables](#compile-time-variables) are available, including `mdsh_lang`, `tag_words`, `block_start`, and possibly `MDSH_SOURCE`.\n\nIn all of the above cases, `$mdsh_lang` is set to the *first* word of the language tag, but is not otherwise included in the command line executed.  (It's assumed to be a syntax highlighting hint, but can also be used as a parameter if your code references `$mdsh_lang`.)\n\nCommand blocks override normal language function lookups, so no  `mdsh-after-X` , `mdsh-lang-X`, or `mdsh-compile-X` functions are looked up or executed for command blocks.  Thus, this code as input to mdsh:\n\n~~~markdown\n```json !printf \"echo %q\\n\" \"# line $3, $mdsh_lang block:\"  \"def example: $1;\"\n{\"foo\": \"bar\"}\n```\n\n```html +echo \"The $mdsh_lang is:\"\n\u003chtml /\u003e\n```\n\n```python |python - \"a $mdsh_lang block\"\nimport sys; print \"hello, world from \"+sys.argv[1]\n```\n~~~\n\nwould compile to the following shell script:\n\n~~~bash\necho \\#line\\ 49, json block:\necho $'def example: {\"foo\": \"bar\"}\\n;'\necho \"The html is:\" $'\u003chtml /\u003e\\n'\npython - \"a python block\" \u003c\u003c\u003c'```'\nimport sys; print \"hello, world from \"+sys.argv[1]\n```\n~~~\n\nNotice that both forms of command block tags can contain virtually arbitrary bash code, including pipes, substitutions, etc., but **must not** contain backquote characters, as the [Commonmark specification](http://spec.commonmark.org/0.28/#fenced-code-blocks) calls for treating such lines as regular text with inline code, rather than the start of a fenced code block.  mdsh and other Commonmark-conforming tools will therefore not even recognize the line as beginning a code block, and parsing of the rest of the input file will be thrown off, with code being interpreted as text and vice versa.\n\n## Tips and Techniques\n\n### Literate Testing\n\nDocuments created with mdsh can include example shell sessions that are tested using the [cram](https://bitheap.org/cram/) functional testing tool.  Just set cram's indent level to 4 and use 4-space indented blocks for your cram-tested examples, optionally wrapped in `~~~` fenced code blocks like this:\n\n~~~~shell\n    $ echo \"hello world!\"\n    hello world!\n~~~~\n\nCram looks for `$` or `\u003e` and a space, indented to a certain level, then runs the command(s) and verifies the output.  So cram needs to know what indentation you're using.\n\nmdsh ignores 4-space indented and `~~~` fenced blocks, so it won't be confused by your examples.  You can get github to syntax-highlight your examples by including a language tag like `~~~bash` or `~~~shell`.\n\nExplaining all the ins and outs of using cram is beyond the scope of this guide, but in the simplest case, using `cram --indent 4 mydocument.md` will run any 4-space indented examples in `mydocument.md`.\n\n(Note that `cram` does not actually understand markdown, so it will try to run anything that begins with `\"$ \"` or `\"\u003e \"` at the specified indent.  Non-example code that starts that way can typically be outdented slightly or indented further so that cram will ignore it.  Until [this PR](https://github.com/brodie/cram/pull/29) is merged, you'll want to use [this fork of cram](https://github.com/pjeby/cram/tree/indent-fix), which is patched to ignore indented lines that aren't directly part of an example.  If you use [.devkit](https://github.com/bashup/.devkit)'s cram module, your tests will automatically run with the correct version.)\n\n### Excluding Blocks From The Generated Script\n\nIf your script has a lot of documentation examples that contain fenced code blocks, you may want to exclude these from being processed or copied to bash variables.  There are two main ways you can do this.\n\nFirst, you can change the way you indicate certain code blocks.  All of these are currently ignored by `mdsh` and do not generate any code:\n\n* Code blocks indented with four spaces, instead of fenced\n\n* Code blocks fenced with `~~~X` instead of `` ```X ``\n\n* Code blocks fenced with more than three backquotes, or which are indented\n\n* Code blocks with no language given\n\n* Immediate command blocks whose command is empty or a no-op.  (That is, blocks whose language is a single word followed by a space and a `!` character, optionally followed by a shell comment or no-op shell command.)  For example both of these code blocks would be omitted from a compiled script:\n\n  ~~~markdown\n  ​```python ! # mdsh won't do anything with this block\n  raise RuntimeError(\"This won't actually run!\")\n  ​```\n  ​```shell !\n  echo \"No comment is actually needed.  This block is ignored, too.\"\n  ​```\n  ~~~\n\nAlternately, you can define empty `mdsh-compile-X` functions in an mdsh block, for each language you want to exclude from the compilation, or define an `mdsh-misc` function that does nothing.  (Which will disable data blocks entirely; see the section on [Metaprogramming and Code Generation](#metaprogramming-and-code-generation) below for more info on `mdsh-misc`.)\n\n### Making Executable (and Editable) Markdown Files\n\nIf you want to make a markdown file directly executable, you can `chmod +x` it and give it a shebang line such as `#!/usr/bin/env mdsh`.  This will then let you run `somescript.md args..` without needing to type `mdsh` in front of it.\n\nIf you want to get rid of the `.md` extension on your script, you'll probably want to also add a line to tell Github, your editor, etc. that the file is still markdown, e.g.:\n\n```sh\n#!/usr/bin/env mdsh\n\u003c!-- ex: set syntax=markdown : --\u003e\n\u003c!-- -*- mode: markdown -*- --\u003e\n```\n\nThis will tell Github, atom (with the `vim-modeline` package), vim, emacs, and other editor/display tools that the file is actually Markdown.\n\n(Alternately, you can keep the `.md` on the file for editing, but use an extensionless symlink to run it without needing to type the `.md`.)\n\n### Making Sourceable Scripts (and handling $0)\n\nIt's often good practice to write scripts in such a way that the functions they define can be used by other scripts, usually by `source`-ing them.  This leads to the common pattern of writing code like this at the end of bash scripts, to only run the script's main function if the script was *not* sourced:\n\n```bash\nif [[ $0 == $BASH_SOURCE ]]; then\n    # Not `source`d: run as script\n    my-main \"$@\"\n    exit $?\nfi\n```\n\nThis practice is no different with `mdsh`: whether you execute your script with an `mdsh` `#!` line, or compile it and run it, the variables `$0` and `$BASH_SOURCE` will only be equal if your program was run as a script.\n\nUnfortunately, when `mdsh` is run via the `#!` line, the only way to ensure these values are equal is if they're both *empty*.  That means `$0` will be an empty string, which may interfere with common practices like using it in \"usage\" messages to denote the program name.\n\nAs a workaround, `mdsh` defines an `$MDSH_ZERO` argument for you, if and only if your script was invoked directly.  It contains the value that `$0` and `$BASH_SOURCE`*would* have had, if your script had been compiled.  You can thus use `${0:-$MDSH_ZERO}` to portably retrieve the invoking script name, regardless of whether your script was compiled or interpreted.\n\nBut that still doesn't make your script *sourceable*.  It's markdown, not bash, after all.  So if another script tries to `source` it, it'll get all sorts of syntax and other errors.\n\nTo fix that, we need a *shelldown* header: two lines of code that execute in bash, but are hidden in most markdown renderings, and still tell our editors (and Github) to ignore the `#!` line and treat the file as markdown, not bash:\n\n~~~markdown\n#!/usr/bin/env bash\n: '\n\u003c!-- ex: set ft=markdown : '; eval \"$(mdsh --eval \"$BASH_SOURCE\")\" # --\u003e\n\n# My Awesome Script\n\n...code goes here...\n~~~\n\nNow, our script is in `bash` syntax, for just long enough to compile the document and run the result.  Compiling the script with  `mdsh --eval` makes a version that's safe to `eval` and `source`, by adding an extra line to the end of the code that does a `return $?` or `exit $?`, depending on whether the script was sourced.  (This ensures that bash stops processing the file during the `eval`, *before* it can get confused by the markdown content that follows.)\n\nAs a result, this file can be either run or `source`-d without any issues.  What's more, you can `mdsh --compile` it to create a plain bash script (that's still runnable and `source`-able), without needing any code changes.\n\nThat's because when you directly run or source the script, you're really executing the *same* code that would be compiled, in the same environment, with `$BASH_SOURCE` pointing to the actual file.  (And `$0` matching it, unless it's being sourced.)\n\nSo why not use this method all the time?  Well, you certainly *can*.  And if you don't mind copying and pasting it into all your new scripts, then by all means go ahead!  However, an `mdsh` `#!` line is *definitely* the easier choice for one-off scripts that aren't being sourced, where you aren't using the value of `$0`, and you don't care about editor support.\n\n### Syntax Highlighting and Language Aliasing\n\nBecause `mdsh` is not a widely-recognized language, Github and other markdown editing/processing tools generally don't know how to highlight them properly.  As a workaround, you can give a block tag of `shell @mdsh` , which most tools will then interpret as a shell block for highlighting purposes.  e.g.:\n\n~~~markdown\n```shell @mdsh\necho 'echo \"Most tools will highlight this block as shell script\"'\n```\n~~~\n\n## Metaprogramming and Code Generation\n\nAny output from a `mdsh`-tagged block becomes part of the generated bash script at the point where the block occurs.  This means that you can simply `cat` other bash files to include them (or use `mdsh-embed`; see the next section), or do anything else you like to generate code there.  This can be a useful alternative to using `source` to load functions, as it means that the resulting script can be `--compile`d to a single file that doesn't need the other modules present.\n\nIf you want to programmatically process individual blocks in some fashion (for example, to extract filenames from their language tags), you can define an `mdsh-misc` function.  For each block without an `mdsh-lang-X ` or `mdsh-compile-X` function, `mdsh-misc` is called with the language tag and block contents as arguments, and its output is appended to the compiled script at that point in the file.\n\nSo for example, when run, this script outputs the contents of the text block to `file1.txt`:\n\n~~~markdown\n​```mdsh\nmdsh-misc() {\n    if [[ $1 == *'\u003e'* ]]; then echo -n \"$2\" \u003e\"${1#*\u003e}\"; fi\n}\n​```\n\n​```text \u003efile1.txt\nSome text goes here!\n​```\n~~~\n\nOf course, the possible applications of `mdsh-misc` are considerably more varied than just writing blocks to files.  You could, for example:\n\n* Emulate the \"tangling\" features of other literate programming tools (by having `mdsh-misc` save the contents of blocks to different variables based on their tag information, and then ending your program with an `mdsh` block that outputs the saved blocks in the desired order)\n* Interpret tags as arguments for how a block should be processed\n* Treat blocks tags starting with `|` as a pipeline to preprocess the block with\n\n...and just about anything else you can imagine.\n\n### \"Static Linking\" for Distribution\n\nYou can use the `mdsh-embed` function to embed the source of other bash modules into your script.  Calling `mdsh-embed` *modulename* inside an `mdsh` block will search `PATH` for *modulename* (unless *modulename* contains a `/`), and then output its source code, wrapped in a heredoc and `source` command.  (This ensures that the embedded module will know it was sourced, and not run via the command line, even if the embedding script *was* run from the command line.)\n\nThe net result is that by using `mdsh-embed` in your `mdsh` block(s) to load your modules (instead of `source` inside your `shell` blocks), you gain the ability to `--compile` your script to a \"statically-linked executable\".  That is, you can create a single file that contains all the modules it needs, so your users don't have to install all your dependencies themselves, and don't need a specific package manager to install your script.\n\n## Extending `mdsh` or Reusing its Functions\n\nSourcing `mdsh` from a bash script will define all its functions, but not actually run a program.  This allows you to change how command line arguments are processed, or predefine additional language hooks, teardown hooks, etc.   (You can also just do it to make use of the included markdown processing functions.)\n\n(Note that sourcing `mdsh` will set bash to  [\"unofficial strict mode\"](http://redsymbol.net/articles/unofficial-bash-strict-mode/) , i.e. `-euo pipefail`.  `mdsh` is written with the assumption that these settings are in effect, so changing them may have undesirable results.)\n\nIf what you're writing is just \"mdsh with more languages\", you can do so like this:\n\n```bash\n#!/usr/bin/env bash\nsource \"$(command -v mdsh)\"\n\nmdsh-compile-somelang() {\n    # etc.\n}\n\n# ...\n\n[[ $0 == $BASH_SOURCE ]] \u0026\u0026 mdsh-main \"$@\"\n\n```\n\nThat is, just source mdsh and define your additional language handlers, then run `mdsh-main \"$@\"`: your script will then have the same command-line interface as `mdsh`, but all its help messages will refer to the name of your script, instead of `mdsh`.\n\n### Adding File Headers or Footers\n\nIf your extended version of mdsh needs to add headers or footers to generated files, you can define functions named `mdsh:file-header` and/or `mdsh:file-footer`.  The `--compile` option will call these functions once at the start and end of the compilation process, wrapping the entire output.  `--eval` will works similarly, except that the `--eval` footer will appear *after* `mdsh:file-footer`.\n\n### Altering Existing Functions\n\nIn some cases, you may wish to also alter parts of mdsh's behavior, by replacing some of its functions.  You can, however, avoid the need to copy those function into your code by using `mdsh-rewrite`.  `mdsh-rewrite` is a function normally used in the mdsh compiler to rewrite `mdsh-lang-X` and `mdsh-after-X` function bodies, but you can adapt it to do AOP-like editing of bash functions.\n\nFor example, [jqmd](https://github.com/bashup/jqmd) adds a header and footer to files compiled with `jqmd --compile` and `jqmd --eval`, by adding a function call to the start and end of the `mdsh.--compile` function:\n\n```bash\neval \"mdsh.--compile() $(mdsh-rewrite mdsh.--compile '{ jqmd-header;' 'jqmd-footer; }')\"\n```\n\n(`mdsh-rewrite` takes a function name and two optional strings that will replace the opening and closing brace lines of the function body.  The result is output to stdout, where it becomes the body of the new function.)\n\n### Available Functions\n\nThe following functions are available for your use or alteration in scripts sourcing `mdsh`:\n\n* `mdsh-run` *mdfile [cache-key [args...]]* -- `source` the compiled version of the specified markdown file with *args* as its positional arguments (`$1`,  `$2`, etc.).  The directory in `$MDSH_CACHE` (if any) is used to cache the compiled version in a file whose name is generated using *cache-key*.  If *cache-key* is missing or empty, *mdfile* is used as the cache key.\n\n  To set a specific cache directory or disable caching, set it using `MDSH_CACHE=` on the same line as `mdsh-run`, e.g. `MDSH_CACHE= mdsh-run somefile` to run `somefile` without caching.  Sourcing or running `mdsh` sets the default `MDSH_CACHE` to `$XDG_CACHE_HOME/mdsh` or `$HOME/.cache/mdsh`, as appropriate for your OS.\n\n* `mdsh-use-cache` *[cachedir]* -- sets `MDSH_CACHE` to *cachedir*.  If no arguments are given, `MDSH_CACHE` is reset to the default of  `$XDG_CACHE_HOME/mdsh` or `$HOME/.cache/mdsh`, as appropriate.\n\n* `run-markdown` *mdfile args...* -- `source` the compiled version of *mdfile* with *args* as its positional arguments (`$1`,  `$2`, etc.), without using a cache.\n\n* `mdsh-error` *format args...* --`printf` *format args* to stderr and terminate the process with errorlevel 64 ([EX_USAGE](https://www.freebsd.org/cgi/man.cgi?query=sysexits\u0026sektion=3#DESCRIPTION)) .  (A linefeed is added to the format string automatically.)\n\n* `mdsh-compile` -- accepts markdown on stdin and outputs bash code on stdout.  The compilation takes place in a subshell, so hook functions defined in the code being compiled do **not** affect the caller's environment.  Hook functions *already* defined in the caller's environment, however, will be used to translate blocks of the relevant languages.\n\n* `mdsh-source` -- like `mdsh-compile`, but not run in subshell, so any hook functions or compile-time variable changes will affect the caller, as if the included source were part of the including document.\n\n* `mdsh-embed` *modulename* -- look for *modulename* on `PATH` (unless it contains a `/`), and dump its contents wrapped in a `source` command and heredoc.  Returns failure if *modulename* isn't found or can't be read.  (Note: unlike Bash's `source` command, this function does **not** fall back to looking for the module in the current directory.  If you want a file in the current directory, use `./modulename`).\n\n* `mdsh-rewrite` *function* *before* *after* -- output the body block of *function* on stdout, optionally replacing the opening and closing braces with *before* and *after*.  (If you're using this to \"edit\" a function, remember that the replacements must include the opening and closing braces, and the closing brace must be preceded by either a newline or a semicolon or space.)\n\n* `mdsh-make` *sourcefile destfile [cmd args...]* -- compile *sourcefile* to *destfile* if *destfile* doesn't exist or has a different timestamp than *sourcefile*, running *cmd args...* before starting compilation.  Compilation (and *cmd args...*) are run in a subshell.  On successful compilation, *destfile* is touched so its timestamp is the same as *sourcefile*.\n\n* `mdsh-cache` *cachedir sourcefile [key [cmd args...]]* -- like `mdsh-make`, except the destination file is automatically generated as a filename within *cachedir* and returned in `$REPLY`.  If a non-empty *key* is specified, it is used instead of *sourcefile* to generate the destination filename.  The *cachedir* is automatically created if it does not exist.  Generated filenames are an escaped version of the *key* or *sourcefile*, such that they are always a file directly under *cachedir*, even if the source filename has slashes in it.\n\n* `exit` *[code [message [args...]]]* -- exit with status *code* (defaulting to `$?` if not given), after displaying *message* on stderr.  If *args* are given, they're formatted using *message* as a `printf` format string (with an automatically-added newline).\n\n","funding_links":[],"categories":["Shell","bash","Libraries"],"sub_categories":["Testing"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbashup%2Fmdsh","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbashup%2Fmdsh","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbashup%2Fmdsh/lists"}