{"id":15629691,"url":"https://github.com/anko/txm","last_synced_at":"2025-04-14T21:52:38.800Z","repository":{"id":31128433,"uuid":"34688055","full_name":"anko/txm","owner":"anko","description":"Markdown code example tester; language-agnostic","archived":false,"fork":false,"pushed_at":"2023-07-18T21:17:32.000Z","size":504,"stargazers_count":46,"open_issues_count":5,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-10T11:57:27.974Z","etag":null,"topics":["documentation","language-agnostic","markdown","shell","tap","testing"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"isc","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/anko.png","metadata":{"files":{"readme":"readme.markdown","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-04-27T19:59:20.000Z","updated_at":"2025-01-03T04:46:30.000Z","dependencies_parsed_at":"2024-06-19T01:30:48.991Z","dependency_job_id":"bcf96d6f-6c1d-4797-9d1b-ccb5233a05a5","html_url":"https://github.com/anko/txm","commit_stats":{"total_commits":251,"total_committers":2,"mean_commits":125.5,"dds":"0.20318725099601598","last_synced_commit":"4360a14a20461e8c862b14cecab82eff9fc72da4"},"previous_names":["anko/tests-ex-markdown"],"tags_count":31,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anko%2Ftxm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anko%2Ftxm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anko%2Ftxm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anko%2Ftxm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/anko","download_url":"https://codeload.github.com/anko/txm/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248968759,"owners_count":21191158,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["documentation","language-agnostic","markdown","shell","tap","testing"],"created_at":"2024-10-03T10:28:10.738Z","updated_at":"2025-04-14T21:52:38.782Z","avatar_url":"https://github.com/anko.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# txm [![](https://img.shields.io/npm/v/txm.svg)][1] [![](https://img.shields.io/github/actions/workflow/status/anko/txm/ci.yml?branch=master)][2] [![](https://img.shields.io/coveralls/github/anko/txm)][coveralls]\n\n\u003cimg align=\"right\" width=\"40%\" alt=\"example output for a test failure\" src=\"https://user-images.githubusercontent.com/5231746/78293904-a7f23a00-7529-11ea-9632-799402a0219b.png\"\u003e\u003c/img\u003e\n\nCommand-line tool that checks correctness of your [Markdown][markdown]\ndocumentation's code examples.  Parses `\u003c!-- !test command --\u003e` annotations\npreceding code blocks, runs them, and checks that the outputs match.\n\n - __Only uses HTML comments__\n   \u003cbr\u003e\u003csup\u003eThe annotations aren't rendered.  You retain formatting\n   control.\u003c/sup\u003e\n - __Works with any programming language__\n   \u003cbr\u003e\u003csup\u003eYou choose the shell command(s).  Many languages in the same doc\n   are OK.\u003c/sup\u003e\n - __Helpful failure diagnostics__\n   \u003cbr\u003e\u003csup\u003eColours (optional), diffs, line numbers, exit code, stderr,\n   invisible characters, etc.\u003c/sup\u003e\n - __Parallel tests on multi-core machines__\n   \u003cbr\u003e\u003csup\u003eConfigurable.  Result output ordering remains constant.\u003c/sup\u003e\n - __[TAP][tap-spec] format output__\n   \u003cbr\u003e\u003csup\u003eThe standard supported by many testing tools.\u003c/sup\u003e\n\n# Example\n\n\u003c!-- !test program node src/cli.js --\u003e\n\n\u003c!-- !test in example --\u003e\n\n1. Write a `README.md`, with comment annotations:\n\n   ```markdown\n   # console.log\n\n   The [console.log][1] function in [Node.js][2] stringifies the given arguments\n   and writes them to `stdout`, followed by a newline.  For example:\n\n   \u003c!-- !test program node --\u003e\n\n   \u003c!-- !test in simple example --\u003e\n\n       console.log('a')\n       console.log(42)\n       console.log([1, 2, 3])\n\n   The output is:\n\n   \u003c!-- !test out simple example --\u003e\n\n       a\n       42\n       [ 1, 2, 3 ]\n\n   [1]: https://nodejs.org/api/console.html#console_console_log_data_args\n   [2]: https://nodejs.org/\n   ```\n\n   See [§ *Use*](#use) for more detail on how annotations work. Fenced code\n   blocks delimited by `` ``` `` work too.  Language tags also.\n\n2. Run:\n\n   ```bash\n   $ txm README.md\n   ```\n\n3. See output:\n\n   \u003c!-- !test out example --\u003e\n\n   \u003cimg align=\"right\" alt=\"example output\" src=\"https://user-images.githubusercontent.com/5231746/143256158-d4e8236c-720e-4b3b-be4c-e05dc679c526.png\"\u003e\u003c/img\u003e\n\n   \u003e ```tap\n   \u003e TAP version 13\n   \u003e 1..1\n   \u003e ok 1 simple example\n   \u003e\n   \u003e # 1/1 passed\n   \u003e # OK\n   \u003e ```\n\n- - -\n\nExamples of other use-cases:\n\n\u003cdetails\u003e\u003csummary\u003eTesting Node.js code with ESM imports\u003c/summary\u003e\n\nRunning code from `node`'s stdin as an ES module requires\n`--input-type=module`.\n\n\u003c!-- !test in node ESM example --\u003e\n\n```markdown\nDemonstrating that the root directory is a directory:\n\n\u003c!-- !test program node --input-type=module --\u003e\n\n\u003c!-- !test in example --\u003e\n\n    import { stat } from 'fs/promises'\n    console.log((await stat('/')).isDirectory())\n\n\u003c!-- !test out example --\u003e\n\n    true\n\n```\n\n\u003c!-- !test out node ESM example --\u003e\n\n\u003e ```\n\u003e TAP version 13\n\u003e 1..1\n\u003e ok 1 example\n\u003e\n\u003e # 1/1 passed\n\u003e # OK\n\u003e ```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eTesting C code with \u003ccode\u003egcc\u003c/code\u003e\u003c/summary\u003e\n\n\u003c!-- !test in C example --\u003e\n\nAny sequence of shell commands is a valid `!test program`, so you can e.g. cat\nthe test input into a file, then compile and run it:\n\n```markdown\n\u003c!-- !test program\ncat \u003e /tmp/program.c\ngcc /tmp/program.c -o /tmp/test-program \u0026\u0026 /tmp/test-program --\u003e\n\nHere is a simple example C program that computes the answer to life, the\nuniverse, and everything:\n\n\u003c!-- !test in printf --\u003e\n\n    #include \u003cstdio.h\u003e\n    int main () {\n        printf(\"%d\\n\", 6 * 7);\n    }\n\n\u003c!-- !test out printf --\u003e\n\n    42\n```\n\n\u003c!-- !test out C example --\u003e\n\n\u003e ```\n\u003e TAP version 13\n\u003e 1..1\n\u003e ok 1 printf\n\u003e\n\u003e # 1/1 passed\n\u003e # OK\n\u003e ```\n\nIn practice you might want to invoke `mktemp` in the `!test program` to avoid\nmultiple parallel tests overwrting each other's files.  Or pass `--jobs 1` to\nrun tests serially.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eRedirecting \u003ccode\u003estderr\u003c/code\u003e→\u003ccode\u003estdout\u003c/code\u003e, to test both in the same\nblock\u003c/summary\u003e\n\nPrepending `2\u003e\u00261` to a shell command [redirects][shell-redirection-q] `stderr`\nto `stdout`.  This can be handy if you don't want to write separate `!test out`\nand `!test err` blocks.\n\n\u003c!-- !test in redirect stderr --\u003e\n\n```markdown\n\u003c!-- !test program 2\u003e\u00261 node --\u003e\n\n\u003c!-- !test in print to both stdout and stderr --\u003e\n\n    console.error(\"This goes to stderr!\")\n    console.log(\"This goes to stdout!\")\n\n\u003c!-- !test out print to both stdout and stderr --\u003e\n\n    This goes to stderr!\n    This goes to stdout!\n```\n\n\u003c!-- !test out redirect stderr --\u003e\n\n\u003e ```\n\u003e TAP version 13\n\u003e 1..1\n\u003e ok 1 print to both stdout and stderr\n\u003e\n\u003e # 1/1 passed\n\u003e # OK\n\u003e ```\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eIgnoring the tested program's exit code\u003c/summary\u003e\n\nNormally, you'd use `!test exit nonzero` (or a specific exit code) to tell txm\nthat a test is expected to fail.  But since you need to write that before every\nfailing run, it can get pointlessly repetitive if e.g. it's obvious only from\nthe output of your program when it failed.\n\nIn such cases, just put `|| true` after the program command to make the shell\nswallow the exit code and pretend to `txm` that it was `0`.  Remember that the\nprogram tests are run with can be a whole script.\n\n\u003c!-- !test in don't fail on non-zero --\u003e\n\n```markdown\n\u003c!-- !test program node || true --\u003e\n\n\u003c!-- !test in don't fail --\u003e\n\n    console.log(\"Hi before throw!\")\n    throw new Error(\"AAAAAA!\")\n\n\u003c!-- !test out don't fail --\u003e\n\n    Hi before throw!\n```\n\n\u003c!-- !test out don't fail on non-zero --\u003e\n\n\u003e ```\n\u003e TAP version 13\n\u003e 1..1\n\u003e ok 1 don't fail\n\u003e\n\u003e # 1/1 passed\n\u003e # OK\n\u003e ```\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eTesting examples that call \u003ccode\u003eassert\u003c/code\u003e\u003c/summary\u003e\n\nIf your example code calls `assert` or such (which throw an error and exit\nnonzero when the assert fails), then you don't really need an output block,\nbecause the example already documents its assumptions.\n\nIn such cases you can use use a `!test check` annotation.  This simply runs the\ncode, ignoring its output.\n\n\u003c!-- !test in asserting test --\u003e\n\n```markdown\n\u003c!-- !test program node --\u003e\n\n\u003c!-- !test check laws of mathematics --\u003e\n\n    const assert = require('assert')\n    assert(1 + 1 == 2)\n\n```\n\n\u003c!-- !test out asserting test --\u003e\n\n\u003e ```\n\u003e TAP version 13\n\u003e 1..1\n\u003e ok 1 laws of mathematics\n\u003e\n\u003e # 1/1 passed\n\u003e # OK\n\u003e ```\n\nIf you are using an assert library that can output ANSI colour codes, it should\ndetect that it is running without a TTY (as tests do), and not output colour.\nBut if txm itself is run in coloured mode, the `TXM_HAS_COLOUR` environment\nvariable will be set to `1`, and it's safe to force colour output on; they will\nbe included in txm's error output.\n\n\u003c/details\u003e\n\nAs you may be suspecting, this readme is itself tested with txm.  All of the\nabove examples run as part of the automatic tests, locally and [on the CI\nserver](https://github.com/anko/txm/actions/workflows/ci.yml?query=is%3Asuccess).\nIf you want to see the comment annotations, [see the readme\nsource](https://github.com/anko/txm/blob/master/readme.markdown?plain=1).\n(It's a little trippy, because txm is recursively running itself.)\n\n# Install\n\nTo install for current directory's project: `npm install txm`\n\u003cbr\u003eTo install globally: `npm install -g txm`\n\nRequires [Node.js][nodejs] (minimum version tested is _current LTS_).\n\n# Use\n\n## Command line\n\n### `txm [--jobs \u003cn\u003e] [filename]`\n\n - `filename`: Input file (default: read from `stdin`)\n - `--jobs`: How many tests may run in parallel. (default: `os.cpus().length`)\n\n   When a test finishes, txm will only print its output after all\n   earlier-defined tests have printed their outputs, so that results appear in\n   the same order tests were defined.  Further tests continue to run in the\n   background, regardless of how many results are pending print.\n\n - `--version`\n - `--help`\n\n## Annotations\n\nHTML comments that start with `!test` are read specially.  Use a separate\ncomment for each annotation.\n\n - #### `!test program \u003cprogram\u003e`\n\n   The `\u003cprogram\u003e` is run as a shell command for each following matching\n   input/output pair.  It gets the input on `stdin`, and is expected to produce\n   the output on `stdout`.  The program may be as many lines as you like; a\n   full shell script if you wish.\n\n   The declared program is used for all tests after here, until a new program\n   is declared.\n\n - #### `!test in \u003cname\u003e` / `!test out \u003cname\u003e` / `!test err \u003cname\u003e`\n\n   The next code block is read as the input to give to a program for the test\n   `\u003cname\u003e`, or expected stdout or stderr of the test `\u003cname\u003e`.  These are\n   matched by `\u003cname\u003e`, and may be anywhere in relation to each other.\n\n   Errors are raised if a test has no input (`in`) or no output (`out` nor\n   `err`), or if it has duplicates of any.\n\n - #### `!test check \u003cname\u003e`\n\n   The next code block is read as a check test.  The program gets this as\n   input, but its output is ignored.  The test will pass if the program exits\n   successfully.  (With exit code `0`, or that specified in a `!test exit`\n   command prior.)\n\n   Use this for code examples that check their own correctness, for example by\n   calling an `assert` function.\n\n - #### `!test exit \u003ccode\u003e`\n\n   The _next test_ which is fully read is expected to fail, and to exit with\n   the given `code`, instead of the default `0`.\n\n   You can use `!test exit nonzero` to accept any non-0 exit code.\n\n - #### `!test only`\n\n   If any test has this command in front of it, all tests without it are\n   skipped.  (They don't run, and their output is suppressed.)\n\n   This is intended for developer convenience:  When you have lots of tests of\n   which only a few are failing, you can use this command to focus on them, so\n   other tests don't waste time running or clutter your screen.\n\n## Behaviour details\n\n### Exit code\n\n`txm` exits `0` if and only if all tests pass.\n\n### Invisible characters\n\nIn diff additions and deletions, [C0 Control Characters][control-chars] (such\nas Null, Line Feed, or Space), which are ordinarily invisible, are shown as the\ncorresponding [Unicode Control Picture][control-picture]. These take the form\nof small diagonally arranged letters, so Null becomes ␀, Line Feed becomes ␊,\nand Space becomes ␠. This is the standard way to show this set of invisible\ncharacters.\n\nWhenever such characters are used, an index will be present in the accompanying\ntest data, listing what original character each picture corresponds to, with\nits name, C escape, and Unicode code point. This is intended to give as much\ninformation as possible, because bugs relating to invisible characters are\nawkward to debug.\n\nIf an invisible character is not part of the diff, it is shown normally\n(without a Control Picture replacement.)\n\nTo maintain line breaks, the Line Feed character is kept as-is, with its\nControl Picture (␊) added at the end of the line for clarity.\n\nInvisible characters that aren't part of the C0 set are shown as-is. Examples\ninclude the zero-width space, or right-to-left text flow marker.\n\n\n[control-chars]: https://en.wikipedia.org/wiki/C0_and_C1_control_codes#C0_controls\n[control-picture]: https://en.wikipedia.org/wiki/Unicode_control_characters#Control_pictures\n\n### Colour \u003csub\u003e(color, for Americans grepping)\u003c/sub\u003e\n\nColoured output is automatically enabled when outputting directly to a\ncolour-capable console interface, and disabled otherwise.  It can be forced on\nor off with the environment variables `NO_COLOR=1` or `FORCE_COLOR=1`, or with\nthe options `--no-color` or `--color`.\n\nStripping colour codes from coloured output does not change its logical\nmeaning, and indeed the same text is emitted regardless of whether colour is\nenabled.  The colours do not themselves carry meaning; they're just hints to\nguide the eye.\n\n### HTML comment character restrictions\n\nThe [HTML spec regarding comments][html-comments-spec] has a few restrictions\non what comments may contain:\n\n\u003e the text must not start with the string `\u003e`, nor start with the string `-\u003e`,\n\u003e nor contain the strings `\u003c!--`, `--\u003e`, or `--!\u003e`, nor end with the string\n\u003e `\u003c!-`.\n\nSome of those are valid constructs in some programming languages, which can be\nrestrictive if you're writing a `!test program` command in one of those\nlanguages.\n\nLuckily all of them involve hyphens (`-`), so to work around \"forbidden\"\ncharacter sequences, txm lets you optionally escape hyphens inside HTML\ncomments: `#-` is automatically replaced by `-`.  So for example, `\u003c!-- !test\nin -#-\u003e --\u003e` is legal HTML, and will be parsed by txm as the command `!test in\n--\u003e`.\n\nTo write literally `#-`, write `##-` instead, and so on.  `#` acts normally\neverywhere else, and doesn't need to be escaped.\n\n### Environment variables\n\nFor advanced use, your test program sees the same environment variables that\ntxm sees, plus these introduced by txm:\n\n- `TXM_INDEX` (1-based number of test)\n- `TXM_NAME` (name of test)\n- `TXM_INDEX_FIRST`, `TXM_INDEX_LAST` (indexes of first and last tests that\n  will be run)\n- `TXM_INPUT_LANG` (the [language identifier][gh-markdown-lang] of the\n  input/check markdown code block, if any)\n- `TXM_HAS_COLOUR`, `TXM_HAS_COLOR` (both set to `1` if outputting with colours\n  enabled, or to `0` if disabled; they are logically equivalent, just alternate\n  spellings)\n\nYou can use these for example to descriptively name log files, or to easily\ndetect languages and test them differently.\n\nYou can also tell with the colour variables whether txm is doing coloured\noutput or not, and have your program emit debug output with ANSI colour codes\nby your method of choice.  This is probably only reasonable to do for the\noutput of `check` tests, which output is shown unmodified.  Don't do this for\n`in`/`out` tests unless you _really_ know what you're doing; the colour codes\nused by txm's automatic diffing will interfere, and you'll get garbage.\n\n# Trivia\n\nThe name txm stands for \"tests ex markdown\" as in \"deus ex machina\", or\n*temptamentum ex Markdown* I guess if you're feeling extra Latin.\n\n# License\n\n[ISC](LICENSE)\n\n[1]: https://www.npmjs.com/package/txm\n[2]: https://github.com/anko/txm/actions/workflows/ci.yml?query=branch%3Amaster\n[coveralls]: https://coveralls.io/github/anko/txm\n[gh-markdown-lang]: https://docs.github.com/en/github/writing-on-github/working-with-advanced-formatting/creating-and-highlighting-code-blocks#syntax-highlighting\n[html-comments-spec]: https://html.spec.whatwg.org/multipage/syntax.html#comments\n[markdown]: http://daringfireball.net/projects/markdown/syntax\n[nodejs]: https://nodejs.org/\n[shell-redirection-q]: https://superuser.com/questions/1179844/what-does-dev-null-21-true-mean-in-linux\n[tap-spec]: https://testanything.org/tap-version-13-specification.html\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanko%2Ftxm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanko%2Ftxm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanko%2Ftxm/lists"}