{"id":46650008,"url":"https://github.com/tmarktaylor/phmutest","last_synced_at":"2026-03-08T06:04:40.646Z","repository":{"id":188132362,"uuid":"678152333","full_name":"tmarktaylor/phmutest","owner":"tmarktaylor","description":"Detect and troubleshoot broken Python examples in Markdown","archived":false,"fork":false,"pushed_at":"2025-03-11T13:39:40.000Z","size":495,"stargazers_count":8,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-26T17:14:20.143Z","etag":null,"topics":["doctest","markdown","python","testing-tool"],"latest_commit_sha":null,"homepage":"https://tmarktaylor.github.io/phmutest/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tmarktaylor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"code_of_conduct.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-13T21:16:56.000Z","updated_at":"2025-07-10T11:15:23.000Z","dependencies_parsed_at":"2023-09-24T16:05:37.073Z","dependency_job_id":null,"html_url":"https://github.com/tmarktaylor/phmutest","commit_stats":null,"previous_names":["tmarktaylor/phmutest"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/tmarktaylor/phmutest","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmarktaylor%2Fphmutest","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmarktaylor%2Fphmutest/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmarktaylor%2Fphmutest/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmarktaylor%2Fphmutest/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tmarktaylor","download_url":"https://codeload.github.com/tmarktaylor/phmutest/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmarktaylor%2Fphmutest/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30246882,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-08T05:41:50.788Z","status":"ssl_error","status_checked_at":"2026-03-08T05:41:39.075Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["doctest","markdown","python","testing-tool"],"created_at":"2026-03-08T06:04:40.446Z","updated_at":"2026-03-08T06:04:40.634Z","avatar_url":"https://github.com/tmarktaylor.png","language":"Python","readme":"# phmutest 1.0.1\n\n## Detect and troubleshoot broken Python examples in Markdown\n\n- Hybrid Python library / console program checks Python syntax highlighted examples.\n- Python tools to get fenced code block contents from Markdown. | [Here](docs/api.md)\n\nTreats each Markdown file as a single long example, which continues\nacross multiple Markdown [fenced code blocks][3] (FCBs or blocks).\n\n[Skip example and jump down to Features](#features)\n\n## A broken Example\n\nWhen FCBs break we show what caused the error to help you quickly find the root cause.\nLet's start with FCBs that show how to use the library answerlib.\n| [answerlib.py](docs/answerlib_py.md).  The ask method returns the answer\nto the question. | [phmutest output](#phmutest-console-output)\n\n```python\nfrom docs.answerlib import RightAnswer, WrongAnswer, RaiserBot\n```\n\nCreate a RightAnswer instance and ask a question.\nphmutest\nassigns a pass/failed/error/skip status to each Python FCB.\nThis FCB and the FCB above are given 'pass' status.\nNote how the example continues across multiple FCBs.\nIt continues for the entire Markdown file.\n\n```python\npass_bot = RightAnswer()\npass_bot.ask(question=\"What floats?\")\n```\n\nNow we are going to cause the answerlib to raise an\nexception by calling the method inquire() which does not exist.\nThis raises an AttributeError in the library which propagates\nup and out of the first line of the FCB below.\nThis FCB is given 'error' status.\n\n```python\npass_bot.inquire(query=\"What floats?\")\n```\n\nThe test runner keeps going even after an exception. To stop\non first failure use the \"-f\" option.\nCause another exception within answerlib to see the FCB line\nwhere the exception propagates out of the FCB in the log.\nThis FCB is also given 'error' status. See the results in the\nlog below.\n\n```python\nraiser_bot = RaiserBot()\n_ = raiser_bot.ask(question=\"What floats?\")\n```\n\nAdd an FCB that immediately follows a Python code block that has no info string\nor the info string `expected-output`. Captured stdout is compared to the block.\nIn the log a \"o\" after the filename indicates expected output was checked.\n\n```python\nprint(\"Incorrect expected output.\")\n```\n\n```expected-output\nHello World!\n```\n\nTo test a value add an assert statement to the FCB. This FCB fails.\n\n```python\nfail_bot = WrongAnswer()\nanswer = fail_bot.ask(question=\"What floats?\")\nassert answer == \"apples\", f\"got= {repr(answer)}\"\n```\n\n### phmutest command line\n\n```shell\nphmutest README.md --log --quiet\n```\n\n### phmutest console output\n\nThere are two parts:\n\n- unittest printing to sys.stderr\n- phmutest printing to sys.stdout\n\n#### phmutest stdout\n\nThis shows the --log output.\nBelow the log table are the broken FCB Markdown source file lines.\n\n- The location is the file and line number of the opening fence of the FCB.\n- The \"\u003e\" indicates the line that raised the exception.\n\n```txt\nlog:\nargs.files: 'README.md'\nargs.log: 'True'\n\nlocation|label  result  reason\n--------------  ------  ---------------------------------------------------------------\nREADME.md:20..  pass\nREADME.md:31..  pass\nREADME.md:42..  error   AttributeError: 'RightAnswer' object has no attribute 'inquire'\nREADME.md:53..  error   ValueError: What was the question?\nREADME.md:62 o  failed\nREADME.md:72..  failed  AssertionError: got= 'very small rocks'\n--------------  ------  ---------------------------------------------------------------\n\nREADME.md:42\n\u003e   43  pass_bot.inquire(query=\"What floats?\")\n        AttributeError: 'RightAnswer' object has no attribute 'inquire'\n\nREADME.md:53\n    54  raiser_bot = RaiserBot()\n\u003e   55  _ = raiser_bot.ask(question=\"What floats?\")\n        ValueError: What was the question?\n\nREADME.md:62\n    63  print(\"Incorrect expected output.\")\nAssertionError: 'Hello World!\\n' != 'Incorrect expected output.\\n'\n- Hello World!\n+ Incorrect expected output.\n\nREADME.md:72\n    73  fail_bot = WrongAnswer()\n    74  answer = fail_bot.ask(question=\"What floats?\")\n\u003e   75  assert answer == \"apples\", f\"got= {repr(answer)}\"\n        AssertionError: got= 'very small rocks'\n```\n\nOn GitHub, to see Markdown line numbers, view this file and choose\nCode button. (Code is between Preview and Blame).\n\n##### traceback\n\nWhen phmutest is installed with the `[traceback]` extra,\na [stackprinter][21] formatted\ntraceback prints after each broken FCB. [Here](docs/traceback.md)\nis an example traceback.\n\n#### unittest stderr\n\nHere is the unittest output printed to sys.stderr.\nIt starts with captured stdout/stderr from the 'error' FCBs.\nMarkdown Python FCBs are copied to a temporary 'testfile' that is\nrun by the unittest test runner. The test runner prints to stderr before\nthe phmutest stdout printing. The test runner output provides tracebacks\nfor the assertions and exceptions.\nThe testfile line numbers will mostly be different than the Markdown\nline numbers. Look for the Markdown line numbers in the log. (Python 3.11)\n\n```txt\n=== README.md:53 stdout ===\nThis is RaiserBot.ask() on stdout answering 'What floats?'.\n=== end ===\n=== README.md:53 stderr ===\nThis is RaiserBot.ask() on stderr: Uh oh!\n=== end ===\n======================================================================\nERROR: tests (_phm1.Test001.tests) [README.md:42]\n----------------------------------------------------------------------\nTraceback (most recent call last):\n  File \"C:\\Users\\XXX\\AppData\\Local\\Temp\\YYY\\_phm1.py\", line 34, in tests\n    pass_bot.inquire(query=\"What floats?\")\n    ^^^^^^^^^^^^^^^^\nAttributeError: 'RightAnswer' object has no attribute 'inquire'\n\n======================================================================\nERROR: tests (_phm1.Test001.tests) [README.md:53]\n----------------------------------------------------------------------\nTraceback (most recent call last):\n  File \"C:\\Users\\XXX\\AppData\\Local\\Temp\\YYY\\_phm1.py\", line 40, in tests\n    _ = raiser_bot.ask(question=\"What floats?\")\n        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\XXX\\Documents\\u0\\docs\\answerlib.py\", line 32, in ask\n    raise ValueError(\"What was the question?\")\nValueError: What was the question?\n\n======================================================================\nFAIL: tests (_phm1.Test001.tests) [README.md:62]\n----------------------------------------------------------------------\nTraceback (most recent call last):\n  File \"C:\\Users\\XXX\\AppData\\Local\\Temp\\YYY\\_phm1.py\", line 51, in tests\n    _phm_testcase.assertEqual(_phm_expected_str, _phm_printer.stdout())\nAssertionError: 'Hello World!\\n' != 'Incorrect expected output.\\n'\n- Hello World!\n+ Incorrect expected output.\n\n\n======================================================================\nFAIL: tests (_phm1.Test001.tests) [README.md:72]\n----------------------------------------------------------------------\nTraceback (most recent call last):\n  File \"C:\\Users\\XXX\\AppData\\Local\\Temp\\YYY\\_phm1.py\", line 58, in tests\n    assert answer == \"apples\", f\"got= {repr(answer)}\"\n           ^^^^^^^^^^^^^^^^^^\nAssertionError: got= 'very small rocks'\n\n----------------------------------------------------------------------\nRan 1 test in 0.027s\n\nFAILED (failures=2, errors=2)\n```\n\n## Features\n\n- Checks either Python code examples **or** \"\u003e\u003e\u003e\" REPL examples\n  | [doctest][5].\n- Reports pass/failed/error/skip status and Markdown file line number for each block.\n- Shows block source indicating the line where the exception propagated.\n- Support for setup and cleanup. Acquire and release resources, change context,\n  Pass objects as global variables to the examples. Cleans up even when fail-fast.\n  [Suite initialization and cleanup](#suite-initialization-and-cleanup)\n- Call from Python, call from pytest.\n- Write a pytest testfile into an existing pytest or unittest test suite.\n- Runs files in user specified order.\n- TOML configuration available.\n- An example can continue **across** files.\n- Show stdout printed by examples. --stdout\n- Colors pass/failed/error/skip status. --color\n- Syntax highlights broken FCBs in the log. --style\n- Check expected output of code examples. Markdown edits are required.\n- Designated and stable **patch points** for Python standard library\n  **unittest.mock.patch()** patches. | [Here](#patch-points)\n\n## Advanced features\n\nThese features require adding tool specific HTML comment **directives**\nto the Markdown. Because directives are HTML comments they are not visible in\nrendered Markdown. View directives on GitHub\nby pressing the `Code` button in the banner at the top of the file.\n| [Advanced feature details](docs/advanced.md).\n\n- Assign test group names to blocks. Command line options select or\n  deselect test groups by name.\n- Skip blocks or skip checking printed output.\n- Label any fenced code block for later retrieval.\n- Accepts [phmdoctest][17] directives except share-names and clear-names.\n- Specify blocks as setup and teardown code for the file or setup across files.\n\n## main branch status\n\n[![license](https://img.shields.io/pypi/l/phmutest.svg)](https://github.com/tmarktaylor/phmutest/blob/main/LICENSE)\n[![pypi](https://img.shields.io/pypi/v/phmutest.svg)](https://pypi.python.org/pypi/phmutest)\n[![python](https://img.shields.io/pypi/pyversions/phmutest.svg)](https://pypi.python.org/pypi/phmutest)\n\n[![CI](https://github.com/tmarktaylor/phmutest/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/tmarktaylor/phmutest/actions/workflows/ci.yml)\n[![Build status](https://ci.appveyor.com/api/projects/status/nbu1xlraoii8x377?svg=true)](https://ci.appveyor.com/project/tmarktaylor/phmutest)\n[![readthedocs](https://readthedocs.org/projects/phmutest/badge/?version=latest)](https://phmutest.readthedocs.io/en/latest/?badge=latest)\n[![codecov](https://codecov.io/gh/tmarktaylor/phmutest/coverage.svg?branch=main)](https://codecov.io/gh/tmarktaylor/phmutest?branch=main)\n\n[Docs RTD](https://phmutest.readthedocs.io/en/latest/) |\n[Docs auto-theme](https://tmarktaylor.github.io/phmutest) |\n[Repos](https://github.com/tmarktaylor/phmutest) |\n[pytest][13] |\n[Codecov](https://codecov.io/gh/tmarktaylor/phmutest?branch=main) |\n[License](https://github.com/tmarktaylor/phmutest/blob/main/LICENSE)\n\n[Features](#features) |\n[Advanced features](#advanced-features) |\n[main branch status](#main-branch-status) |\n[Installation](#installation) |\n[Usage](#usage) |\n[FILE](#file) |\n[REPL mode](#repl-mode) |\n[Suite initialization and cleanup](#suite-initialization-and-cleanup) |\n[--color](#color-option) |\n[--style](#style-option) |\n[Extend an example across files](#extend-an-example-across-files) |\n[Skip blocks from the command line](#skip-blocks-from-the-command-line) |\n[--summary](#summary-option) |\n[TOML configuration](#toml-configuration) |\n[Run as a Python module](#run-as-a-python-module) |\n[Call from Python](#call-from-python) |\n[Call from pytest](#call-from-pytest) |\n[Patch points](#patch-points) |\n[Hints](#hints) |\n[Related projects](#related-projects) |\n[Differences between phmutest and phmdoctest](#differences-between-phmutest-and-phmdoctest)\n\n[Sections](docs/demos.md#sections) |\n[Demos](docs/demos.md#demos) |\n[Changelog](CHANGELOG.md) |\n[Contributions](CONTRIBUTING.md)\n\nSee [list of demos](docs/demos.md)\nSee [How it works](docs/howitworks.md)\n\n## Installation\n\n```shell\npython -m pip install phmutest\n```\n\n- No required dependencies since Python 3.11. Depends on tomli before Python 3.11.\n- Pure Python.\n- It is advisable to install in a virtual environment.\n\n### install extras\n\nThe extra 'color' enables the --color and\n--style options.\n\n```shell\npython -m pip install \"phmutest[color]\"  # Windows\npython -m pip install 'phmutest[color]'  # Unix/macOS\n```\n\nThe extra 'pytest' installs pytest and the plugin\npytest-subtests.\npytest-subtests continues running subtests after\nthe first subtest failure. [pytest][20] prints a very\nhelpful traceback when FCBs break.\n\n```shell\npython -m pip install \"phmutest[pytest]\"  # Windows\npython -m pip install 'phmutest[pytest]'  # Unix/macOS\n```\n\nThe extra 'traceback' enables [stackprinter][21] traceback\nprinting for each broken FCB. The traceback is\nslightly different than pytest's.\n\n```shell\npython -m pip install \"phmutest[traceback]\"  # Windows\npython -m pip install 'phmutest[traceback]'  # Unix/macOS\n```\n\nInstall with the extra 'dev' to install locally the same tools used by\nthe continuous integration scripts.\n\n```shell\npython -m pip install \"phmutest[dev]\"  # Windows\npython -m pip install 'phmutest[dev]'  # Unix/macOS\n```\n\nInstall with all the extras.\n\n```shell\npython -m pip install \"phmutest[color, traceback, dev]\"  # Windows\npython -m pip install 'phmutest[color, traceback, dev]'  # Unix/macOS\n```\n\n## Usage\n\n`phmutest --help`\n\n```txt\nusage: phmutest [-h] [--version] [--skip [TEXT ...]] [--fixture DOTTED_PATH.FUNCTION]\n                [--share-across-files [FILE ...]]\n                [--setup-across-files [FILE ...]] [--select [GROUP ...] | --deselect\n                [GROUP ...]] [--config TOMLFILE] [--replmode] [--color]\n                [--style STYLE] [-g OUTFILE]\n                [--progress] [--sharing [FILE ...]] [--log] [--summary] [--stdout] [--report]\n                [FILE ...]\n\nDetect and troubleshoot broken Python examples in Markdown. Accepts relevant unittest options.\n\npositional arguments:\n  FILE                  Markdown input file.\n\noptions:\n  -h, --help            show this help message and exit\n  --version             show program's version number and exit\n  --skip [TEXT ...]     Any block that contains the substring TEXT is not tested.\n  --fixture DOTTED_PATH.FUNCTION\n                        Function run before testing.\n  --share-across-files [FILE ...]\n                        Shares names from Markdown file to later positional files.\n  --setup-across-files [FILE ...]\n                        Apply Markdown file setup blocks to files.\n  --select [GROUP ...]  Select all blocks with phmutest-group GROUP directive for testing.\n  --deselect [GROUP ...]\n                        Exclude all blocks with phmutest-group GROUP directive from testing.\n  --config TOMLFILE     .toml configuration file.\n  --replmode            Test Python interactive sessions.\n  --color, -c           Enable --log pass/failed/error/skip result colors.\n  --style STYLE         Specify a Pygments style name as STYLE to enable syntax highlighting.\n  -g OUTFILE, --generate OUTFILE\n                        Write generated Python or docstring to output file or stdout.\n  --progress            Print block by block test progress. File by file in --replmode.\n  --sharing [FILE ...]  For these files print name sharing. . means all files.\n  --log                 Print log items when done.\n  --summary             Print test count and skipped tests.\n  --stdout              Print output printed by blocks.\n  --report              Print fenced code block configuration, deselected blocks.\n```\n\n- The **-f** option indicates fail fast.\n\n## FILE\n\nThe Markdown files are processed in the same order they are present as positional\narguments on the command line.\nShell wildcards can be used. Be aware that the shell expansion and operating system\nwill determine the order.\n\n## REPL mode\n\nWhen --replmode is specified Python interactive sessions are tested and\nPython code and expected output blocks are not tested. REPL mode tests are\nimplemented using [doctest][5].\nThe option --setup-across-files and the setup and teardown directives\nhave no effect in REPL mode.\n--progress has file by file granularity.\nSee the [Broken REPL example](docs/repl/REPLexample.md).\n\n## Suite initialization and cleanup\n\nFor background refer to definitions at the top of [unittest][18].\nUse --fixture to specify a Python initialization function that runs before the tests.\nIt works with or without --replmode, but there are differences.\nIn both modes, the fixture function may create objects (globs) that are visible\nas globals to the FCBs under test.\n\nIn the event of test errors orderly cleanup/release of resources is assured.\nFor Python code blocks the fixture may register cleanup functions by\ncalling **unittest.addModuleCleanup()**.\nIn REPL mode the fixture function optionally returns a cleanup function.\n\n- The fixture can acquire and release resources or change context.\n- The fixture can make entries to the log displayed by --log.\n- The fixture can install patches to the code under test.\n\nSpecify the --fixture function as\na relative **dotted path** where `/` is replaced with `.`.\nFor example, the function **my_init()** in the file **tests/myfixture.py**\nwould be specified:\n\n--fixture **tests.myfixture.my_init**\n\nThe function is passed keyword only arguments and **optionally**\nreturns a Fixture instance.\nThe keyword arguments and return type are described by\n[src/phmutest/fixture.py](docs/fixture_py.md).\nThe fixture file should be in the project directory tree. Fixture demos:\n\n- [fixture change workdir](docs/fix/code/chdir.md)\n- [fixture set globals](docs/fix/code/globdemo.md)\n- [fixture cleanup REPL Mode](docs/fix/repl/drink.md)\n\nThe test case test_doctest_optionflags_patch() shows an\nexample with a fixture that applies a patch to\ndoctest optionflags in --replmode.\n\n### Dotted path details\n\nThe fixture function must be at the top level of a .py file.\n\n- The dotted_path has components separated by \".\".\n- The last component is the function name.\n- The next to last component is the python file name without the .py suffix.\n- The preceding components identify parent folders. Folders should be\n  relative to the current working directory which is typically the\n  project root.\n\n## color option\n\nThe --color -c option colors the --log pass/failed/error/skip status.\n\n## style option\n\nThe --style option enables the PYPI project [Pygments][19] syntax\nhighlighting style used in the --log output.\nThe style option requires the `[color]` installation extra.\n\n```txt\n--style \u003cpygments-style-name\u003e\n```\n\n## Extend an example across files\n\nNames assigned by all the blocks in a file can be shared, as global variables,\nto files specified later in the command line.\nAdd a markdown file path to the --share-across-files command line option.\nThe 'shared' file(s) must also be specified as a FILE positional command line argument.\n\n- [share demo](docs/share/share_demo.md) |\n  [how it works](docs/codemode.md#share-across-files)\n- [--replmode share demo](docs/repl/replshare_demo.md) |\n  [how it works](docs/sessionmode.md#share-across-files)\n\n## Skip blocks from the command line\n\nThe skip `--skip TEXT` command line option\nprevents testing of any Python code or REPL block that contains the substring TEXT.\nThe block is logged as skip with `--skip TEXT` as the reason.\n\n## summary option\n\nThe example  [here](docs/share/share_demo.md) shows --summary output.\n\n## TOML configuration\n\nCommand line options can be augmented with values from a `[tool.phmutest]` section in\na .toml configuration file. It can be in a new file or added to an existing\n.toml file like pyproject.toml.\nThe configuration file is specified by the `--config FILE` command line option.\n\nZero or more of these TOML keys may be present in the `[tool.phmutest]` section.\n\n| TOML key           | Usage option        | TOML value - double quoted strings\n| :------------------| :-----------------: | :---------:\n| include-globs      | positional arg FILE | list of filename glob to select files\n| exclude-globs      | positional arg FILE | list of filename glob to deselect files\n| share-across-files | --share-across-files  | list of path\n| setup-across-files | --setup-across-files  | list of path\n| fixture            | --fixture           | dotted path\n| select             | --select            | list of group directive name\n| deselect           | --deselect          | list of group directive name\n| color              | --color             | Use unquoted true to set\n| style              | --style             | set Pygments syntax highlighting style\n\nOnly one of select and deselect can have strings.\n\n- globs are described by Python standard library **pathlib.Path.glob()**.\n- Any FILEs on the command line extend the files selected by include-globs and\n  exclude-globs.\n- Command line options supersede the keys in the config file.\n- See the example **tests/toml/project.toml**.\n\n## Run as a Python module\n\nTo run phmutest as a Python module:\n\n```bash\npython -m phmutest README.md --log\n```\n\n## Call from Python\n\nCall **phmutest.main.command()** with a string that looks like a\ncommand line less the phmutest, like this:\n`\"tests/md/project.md --replmode\"`\n\n- A `phmutest.summary.PhmResult` instance is returned.\n- When calling from Python there is no shell wildcard expansion.\n- **phmutest.main.main()** takes a list of strings like this:\n  `[\"tests/md/project.md\", \"--replmode\"]` and returns `phmutest.summary.PhmResult`.\n\n[Example](docs/callfrompython.md) | [Limitation](docs/callfrompython.md#limitation)\n\n## Call from pytest\n\nIn some of the tests the --fixture function is in the same pytest file as the\nphmutest library call.  This is not recommended because the Python file is\nimported again by fixture_function_importer() to a new module object.\nThe Python file's module level code will\nbe run a second time. If there are side-effects they will be repeated, likely\nwith un-desirable and hard to troubleshoot behavior.\n\n## Patch points\n\nFeel free to **unittest.mock.patch()** at these places in the code and not worry about\nbreakage in future versions. Look for examples in tests/test_patching.py.\n\n### List of patch points\n\n|       patched function              | purpose\n| :--------------------------------:  | :----------:\n| phmutest.direct.directive_finders() | Add directive aliases\n| phmutest.fenced.python_matcher()    | Add detect Python from FCB info string\n| phmutest.select.OUTPUT_INFO_STRINGS | Change detect expected output FCB info string\n| phmutest.session.modify_docstring() | Inspect/modify REPL text before testing\n| phmutest.reader.post()              | Inspect/modify DocNode detected in Markdown\n\n## Hints\n\n- Since phmutest generates code, the input files should be from a trusted\n  source.\n- The phmutest Markdown parser finds fenced code blocks enclosed by\n  html `\u003cdetails\u003e` and `\u003c/details\u003e` tags.\n  The tags may require a preceding and trailing blank line\n  to render correctly. See example at the bottom tests/md/readerfcb.md.\n- Markdown indented code blocks ([Spec][4] section 4.4) are ignored.\n- A malformed HTML comment ending is bad. Make sure\n  it ends with both dashes like `--\u003e`.\n- A misspelled directive will be missing from the --report output.\n- If the generated testfile has a compile error phmutest might raise an\n  ImportError when importing it.\n- Blocks skipped with --skip and the phmutest-skip directive\n  are not rendered. This is useful to avoid above import error.\n- In repl mode **no** skipped blocks are rendered.\n- \"--quiet\" is passed to the unittest test runner.\n- The unittest \"--locals\" provides more information in traces.\n- Try redirecting `--generate -` standard output into PYPI Pygments to\n  colorize the generated test file.\n- In code mode patches made by a fixture function are placed\n  when the testfile is run.\n- In code mode printing a class (not an instance) and then checking it in an\n  expected-output FCB is not feasible because Python prints the\n  `__qualname__`. See the file tests/md/qualname.md for an explanation.\n- phmutest is implemented with non-thread-safe context managers.\n\n## Related projects\n\n- phmdoctest\n- rundoc\n- byexample\n- sphinx.ext.doctest\n- sybil\n- doxec\n- egtest\n- pytest-phmdoctest\n- pytest-codeblocks\n\n## Differences between phmutest and phmdoctest\n\n- phmutest treats each Markdown file as a single long example. phmdoctest\n  tests each FCB in isolation. Adding a share-names directive is necessary to\n  extend an example across FCBs within a file.\n- Only phmutest can extend an example across files.\n- phmutest uses Python standard library unittest and doctest as test runners.\n  phmdoctest writes a pytest testfile for each Markdown file\n  which requires a separate step to run. The testfiles then need to be discarded.\n- phmdoctest offers two pytest fixtures that can be used in a pytest test case\n  to generate and run a testfile in one step.\n- phmutest generates tests for multiple Markdown files in one step\n  and runs them internally so there are no leftover test files.\n- The --fixture test suite initialization and cleanup is only available on phmutest.\n  phmdoctest offers some initialization behavior using an FCB with a setup\n  directive and its --setup-doctest option and it only works with sessions.\n  See phmdoctest documentation \"Execution Context\"\n  section for an explanation.\n- phmutest does not support inline annotations.\n\n[3]: https://github.github.com/gfm/#fenced-code-blocks\n[4]: https://spec.commonmark.org\n[5]: https://docs.python.org/3/library/doctest.html\n[13]: https://ci.appveyor.com/project/tmarktaylor/phmutest\n[17]: https://pypi.python.org/pypi/phmdoctest\n[18]: https://docs.python.org/3/library/unittest.html\n[19]: https://pypi.python.org/pypi/pygments\n[20]: https://docs.pytest.org\n[21]: https://github.com/cknd/stackprinter/blob/master/README.md\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftmarktaylor%2Fphmutest","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftmarktaylor%2Fphmutest","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftmarktaylor%2Fphmutest/lists"}