{"id":46650003,"url":"https://github.com/tmarktaylor/phmdoctest","last_synced_at":"2026-03-08T06:04:42.582Z","repository":{"id":43033078,"uuid":"248006245","full_name":"tmarktaylor/phmdoctest","owner":"tmarktaylor","description":"Python syntax highlighted Markdown doctest.","archived":false,"fork":false,"pushed_at":"2022-06-11T13:20:20.000Z","size":552,"stargazers_count":21,"open_issues_count":1,"forks_count":4,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-08-16T02:33:15.384Z","etag":null,"topics":["doctest","markdown","pytest","python"],"latest_commit_sha":null,"homepage":"https://tmarktaylor.github.io/phmdoctest/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tmarktaylor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-03-17T15:31:54.000Z","updated_at":"2025-03-05T19:27:23.000Z","dependencies_parsed_at":"2022-08-04T23:00:42.762Z","dependency_job_id":null,"html_url":"https://github.com/tmarktaylor/phmdoctest","commit_stats":null,"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"purl":"pkg:github/tmarktaylor/phmdoctest","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmarktaylor%2Fphmdoctest","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmarktaylor%2Fphmdoctest/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmarktaylor%2Fphmdoctest/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmarktaylor%2Fphmdoctest/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tmarktaylor","download_url":"https://codeload.github.com/tmarktaylor/phmdoctest/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmarktaylor%2Fphmdoctest/sbom","scorecard":{"id":889649,"data":{"date":"2025-08-11","repo":{"name":"github.com/tmarktaylor/phmdoctest","commit":"bbb8a4fb4678a7147d16bbbd87edb3c638fc1d71"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.3,"checks":[{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Code-Review","score":0,"reason":"Found 0/2 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/ci.yml:1","Warn: no topLevel permission defined: .github/workflows/demo.yml:1","Warn: no topLevel permission defined: .github/workflows/deploy.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:21: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/phmdoctest/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:23: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/phmdoctest/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:56: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/phmdoctest/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:58: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/phmdoctest/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:128: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/phmdoctest/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:131: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/phmdoctest/ci.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:154: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/phmdoctest/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:166: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/phmdoctest/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:168: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/phmdoctest/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:205: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/phmdoctest/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:207: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/phmdoctest/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:221: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/phmdoctest/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/demo.yml:12: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/phmdoctest/demo.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/demo.yml:14: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/phmdoctest/demo.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/deploy.yml:10: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/phmdoctest/deploy.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/deploy.yml:15: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/phmdoctest/deploy.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/deploy.yml:25: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/phmdoctest/deploy.yml/master?enable=pin","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:40","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:41","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:42","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:43","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:63","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:64","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:65","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:66","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:136","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:137","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:138","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:139","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:173","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:174","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:175","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:176","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:177","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:213","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:214","Warn: pipCommand not pinned by hash: .github/workflows/demo.yml:19","Warn: pipCommand not pinned by hash: .github/workflows/demo.yml:20","Warn: pipCommand not pinned by hash: .github/workflows/demo.yml:21","Warn: pipCommand not pinned by hash: .github/workflows/deploy.yml:20","Warn: pipCommand not pinned by hash: .github/workflows/deploy.yml:21","Info:   0 out of  15 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   2 third-party GitHubAction dependencies pinned","Info:   0 out of  24 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE.txt:0","Info: FSF or OSI recognized license: MIT License: LICENSE.txt:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":9,"reason":"1 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: PYSEC-2024-48 / GHSA-fj7x-q9j7-g6q6"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 30 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-24T11:17:58.315Z","repository_id":43033078,"created_at":"2025-08-24T11:17:58.315Z","updated_at":"2025-08-24T11:17:58.315Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30246882,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-08T05:41:50.788Z","status":"ssl_error","status_checked_at":"2026-03-08T05:41:39.075Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["doctest","markdown","pytest","python"],"created_at":"2026-03-08T06:04:38.994Z","updated_at":"2026-03-08T06:04:42.564Z","avatar_url":"https://github.com/tmarktaylor.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# phmdoctest 1.4.0\n\n## Introduction\n\nPython syntax highlighted Markdown doctest\n\nCommand line program and Python library to test Python syntax\nhighlighted code examples in Markdown.\n\n- Creates a [pytest][15] Python module that tests Python examples in\n  README and other Markdown files.\n- Reads these from Markdown fenced code blocks:\n  - Python interactive sessions described by [doctest][4].\n  - Python source code and expected terminal output.\n- The test cases get run later by running pytest.\n- Simple use case is possible with no Markdown edits at all.\n- More features selected by adding HTML comment **directives**\n  to the Markdown.\n  - Set test case name.\n  - Add a pytest custom marker.\n  - Add a pytest.mark.skip decorator.\n  - Promote names defined in a test case to module level globals.\n  - Label any fenced code block for later retrieval (API).\n- Configurable. Discover and process many Markdown files in a single command.\n- Add inline annotations to comment out sections of code.\n- Get code coverage by running pytest with [coverage][6].\n- Select Python source code blocks as setup and teardown code.\n- Setup applies to code blocks and optionally to session blocks.\n- An included Python library: [Latest Development tools API][10].\n  - Python function returns test file in a string. *(testfile() in main.py)*\n  - Two pytest fixtures. *(tester.py)*\n    1. **testfile_creator** runs *testfile()*. Use with testfile_tester.\n    2. **testfile_tester** runs a pytest file with pytest's pytester\n       in its isolated environment.\n  - Runs phmdoctest and can run pytest too. *(simulator.py)*\n  - Functions to read fenced code blocks from Markdown. *(tool.py)*\n  - Test Markdown for Python examples. *(tool.py)*\n  - Prepare directory for generated test files. *(tool.py)*\n  - Extract testsuite tree and list of failing trees from JUnit XML. *(tool.py)*\n- Available as the pytest plugin [pytest-phmdoctest][16].\n\n\n### default branch status\n[![](https://img.shields.io/pypi/l/phmdoctest.svg)](https://github.com/tmarktaylor/phmdoctest/blob/master/LICENSE.txt)\n[![](https://img.shields.io/pypi/v/phmdoctest.svg)](https://pypi.python.org/pypi/phmdoctest)\n[![](https://img.shields.io/pypi/pyversions/phmdoctest.svg)](https://pypi.python.org/pypi/phmdoctest)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n\n[![CI Test](https://github.com/tmarktaylor/phmdoctest/actions/workflows/ci.yml/badge.svg)](https://github.com/tmarktaylor/phmdoctest/actions/workflows/ci.yml)\n[![Build status](https://ci.appveyor.com/api/projects/status/5pp3swc1q1fgbcd6/branch/master?svg=true)](https://ci.appveyor.com/project/tmarktaylor/phmdoctest/branch/master)\n[![readthedocs](https://readthedocs.org/projects/phmdoctest/badge/?version=latest)](https://phmdoctest.readthedocs.io/en/latest/?badge=latest)\n[![codecov](https://codecov.io/gh/tmarktaylor/phmdoctest/coverage.svg?branch=master)](https://codecov.io/gh/tmarktaylor/phmdoctest?branch=master)\n\n[Website](https://tmarktaylor.github.io/phmdoctest) |\n[Docs](https://phmdoctest.readthedocs.io/en/latest/) |\n[Repos](https://github.com/tmarktaylor/phmdoctest) |\n[pytest][13] |\n[Codecov](https://codecov.io/gh/tmarktaylor/phmdoctest?branch=master) |\n[License](https://github.com/tmarktaylor/phmdoctest/blob/master/LICENSE.txt)\n\n\n[Introduction](#introduction) |\n[Installation](#installation) |\n[Sample usage](#sample-usage) |\n[Sample Usage with HTML comment directives](#sample-usage-with-html-comment-directives) |\n[CI usage](#ci-usage) |\n[--report](#report-option) |\n[Identifying blocks](#identifying-blocks) |\n[Directives](#directives) |\n[skip](#skip) |\n[label on code and sessions](#label-on-code-and-sessions) |\n[label on any fenced code block](#label-on-any-fenced-code-block) |\n[pytest skip](#pytest-skip) |\n[pytest skipif](#pytest-skipif) |\n[setup](#setup) |\n[teardown](#teardown) |\n[share-names](#share-names) |\n[clear-names](#clear-names) |\n[pytest mark decorator](#pytest-mark-decorator) |\n[label skip and mark example](#label-skip-and-mark-example) |\n[setup and teardown example](#setup-and-teardown-example) |\n[share-names clear-names example](#share-names-clear-names-example) |\n[Configuration](#configuration) |\n[Inline annotations](#inline-annotations) |\n[skipping blocks with --skip](#skipping-blocks-with-skip-option) |\n[--skip](#skip-option) |\n[short form of --skip](#short-form-of-skip-option) |\n[--fail-nocode](#fail-nocode-option) |\n[--setup](#setup-option) |\n[--teardown](#teardown-option) |\n[Setup example](#setup-example) |\n[Setup for sessions](#setup-for-sessions) |\n[Execution context](#execution-context) |\n[Send outfile to stdout](#send-outfile-to-stdout) |\n[Usage](#usage) |\n[Run as a Python module](#run-as-a-python-module) |\n[Python API](#python-api) |\n[pytest fixtures](#pytest-fixtures) |\n[Simulate command line](#simulate-command-line) |\n[Hints](#hints) |\n[Directive hints](#directive-hints) |\n[Related projects](#related-projects)\n\n\n\n[Changes](doc/recent_changes.md) |\n[Contributions](CONTRIBUTING.md) |\n[About](doc/about.md)\n\n\n## Installation\nIt is advisable to install in a virtual environment.\n\n    python -m pip install phmdoctest\n\n## Sample usage\n\nGiven the Markdown file [example1.md](doc/example1.md)\nshown in raw form here...\n\n\u003c!--phmdoctest-label example1-raw--\u003e\n~~~\n# This is Markdown file example1.md\n\n## Interactive Python session (doctest)\n\n```py\n\u003e\u003e\u003e print(\"Hello World!\")\nHello World!\n```\n\n## Source Code and terminal output\n\nCode:\n```python\nfrom enum import Enum\n\nclass Floats(Enum):\n    APPLES = 1\n    CIDER = 2\n    CHERRIES = 3\n    ADUCK = 4\n\nfor floater in Floats:\n    print(floater)\n```\n\nsample output:\n```\nFloats.APPLES\nFloats.CIDER\nFloats.CHERRIES\nFloats.ADUCK\n```\n~~~\n\nthe command...\n\u003c!--phmdoctest-label example1-command--\u003e\n```\nphmdoctest doc/example1.md --outfile test_example1.py\n```\n\ncreates the python source code file `test_example1.py` shown here...\n\n\u003c!--phmdoctest-label example1-outfile--\u003e\n```python\n\"\"\"pytest file built from doc/example1.md\"\"\"\nfrom phmdoctest.functions import _phm_compare_exact\n\n\ndef session_00001_line_6():\n    r\"\"\"\n    \u003e\u003e\u003e print(\"Hello World!\")\n    Hello World!\n    \"\"\"\n\n\ndef test_code_14_output_28(capsys):\n    from enum import Enum\n\n    class Floats(Enum):\n        APPLES = 1\n        CIDER = 2\n        CHERRIES = 3\n        ADUCK = 4\n\n    for floater in Floats:\n        print(floater)\n\n    _phm_expected_str = \"\"\"\\\nFloats.APPLES\nFloats.CIDER\nFloats.CHERRIES\nFloats.ADUCK\n\"\"\"\n    _phm_compare_exact(a=_phm_expected_str, b=capsys.readouterr().out)\n```\n\nThen run a pytest command something like this in your terminal\nto test the Markdown session, code, and expected output blocks.\n\n    pytest --doctest-modules\n\nOr these two commands:\n\n    pytest\n    python -m doctest test_example1.py\n\nThe `line_6` in the function name `session_00001_line_6` is the\nline number in [example1.md](doc/example1.md) of the first line\nof the interactive session. `00001` is a sequence number to\norder the doctests.\n\nThe `14` in the function name `test_code_14_output_28` is the\nline number of the first line\nof python code. `28` shows the line number of the expected\nterminal output.\n\nOne test case function gets generated for each:\n\n- Markdown fenced code block interactive session\n- Python-code/expected-output Markdown fenced code block pair\n\nThe `--report` option below shows the blocks discovered and\nhow they are tested.\n\n## Sample Usage with HTML comment directives\n\nGiven the Markdown file shown in raw form here...\n\u003c!--phmdoctest-label directive-example-raw--\u003e\n~~~\n\u003c!--phmdoctest-mark.skip--\u003e\n\u003c!--phmdoctest-label test_example--\u003e\n```python\nprint(\"Hello World!\")\n```\n```\nincorrect expected output\n```\n~~~\n\nthe command...\n\u003c!--phmdoctest-label directive-example-command--\u003e\n```\nphmdoctest tests/one_mark_skip.md --outfile test_one_mark_skip.py\n```\n\ncreates the python source code file shown here...\n\u003c!--phmdoctest-label directive-example-outfile--\u003e\n```python\n\"\"\"pytest file built from tests/one_mark_skip.md\"\"\"\nimport pytest\n\nfrom phmdoctest.functions import _phm_compare_exact\n\n\n@pytest.mark.skip()\ndef test_example(capsys):\n    print(\"Hello World!\")\n\n    _phm_expected_str = \"\"\"\\\nincorrect expected output\n\"\"\"\n    _phm_compare_exact(a=_phm_expected_str, b=capsys.readouterr().out)\n```\n\nRun the --outfile with pytest...\n```\n$ pytest -vv test_one_mark_skip.py\n\ntest_one_mark_skip.py::test_example SKIPPED\n```\n\n- The HTML comments in the Markdown are phmdoctest **directives**.\n- The **mark.skip** directive adds the @pytest.mark.skip() line.\n- The label directive names the test case function.\n- List of  [Directives](#directives)\n- Directives are optional.\n- Markdown edits are optional.\n\n## CI usage\n\nTest Python examples in README.md in Continuous Integration scripts.\nIn this snippet for Linux the pytest test suite is in the\n**tests** folder.\n\n\u003c!--phmdoctest-label ci-example--\u003e\n```bash\nmkdir tests/tmp\nphmdoctest README.md --report --outfile tests/tmp/test_readme.py\npytest --doctest-modules -vv tests\n```\n\nThis console shows testing Python examples in project.md.\nLook for the tmp tests at the bottom. [Windows Usage on Appveyor][13].\n\nSee this excerpt from ci.yml [Actions usage example](doc/actions_usage.md).\nIt runs on Windows, Linux, and macOS. Please find the phmdoctest command\nat the bottom.\n\nNo changes to README.md are needed [here, look in the last job log][14].\n\n## report option\n\nTo see the [GFM fenced code blocks][3] in the MARKDOWN_FILE use the\n`--report` option like this:\n\n\u003c!--phmdoctest-label report-command--\u003e\n```\nphmdoctest doc/example2.md --report\n```\n\nwhich lists the fenced code blocks it found in\nthe file [example2.md](doc/example2.md).\nThe `test role` column shows how each fenced code block gets tested.\n\n\u003c!--phmdoctest-label example2-report--\u003e\n```\n         doc/example2.md fenced blocks\n------------------------------------------------\nblock     line  test     TEXT or directive\ntype    number  role     quoted and one per line\n------------------------------------------------\npython       9  code\n            14  output\npython      20  code\n            26  output\n            31  --\npython      37  code\npython      44  code\n            51  output\nyaml        59  --\ntext        67  --\npy          75  session\npython      87  code\n            94  output\npy         102  session\n------------------------------------------------\n7 test cases.\n1 code blocks with no output block.\n```\n\n## Identifying blocks\n\nThe PYPI [commonmark][7] project provides code to extract fenced code\nblocks from Markdown. Specification [CommonMark Spec][8] and website [CommonMark][9].\n\nPython code, expected output, and Python interactive sessions get extracted.\n\nOnly [GFM fenced code blocks][3] are considered.\n\nA block is a session block if the info_string starts with ``py``\nand the first line of the block starts with the\nsession prompt: `'\u003e\u003e\u003e '`.\n\nTo be treated as Python code the opening fence should start\nwith one of these:\n\n    ```python\n    ```python3\n    ```py3\n\nplus the block contents can't start with `'\u003e\u003e\u003e '`.\n\nThe examples use the info_strings `python` for code and `py` for sessions\nsince they render with coloring on GitHub, readthedocs, GitHub Pages,\nand Python package index.\n\n[project.md](project.md) has more examples of code and session blocks.\n\nIt is ok if the [info string][11]\nis laden with additional text, it will be ignored.  The\nentire info string will be shown in the block type column of the\nreport.\n\nAn output block is a fenced code block that immediately follows a\nPython block and starts with an opening fence like this which\nhas an empty info string.\n\n    ```\n\nA Python code block has no output\nif it is followed by any of:\n\n- Python code block\n- Python session block\n- a fenced code block with a non-empty info string\n\nTest code gets generated for it, but there will be no\nassertion statement.\n\n## Directives\n\nDirectives are HTML comments containing test generation commands.\nThey are edited into the Markdown file immediately before a fenced\ncode block. It is OK if other HTML comments are present.\nSee the `\u003c!--phmdoctest-skip--\u003e` directive in the\nraw Markdown below.\nWith the skip directive no test code will be\ngenerated from the fenced code block.\n\n\u003c!--phmdoctest-label intro-to-directives--\u003e\n~~~\n\u003c!--phmdoctest-skip--\u003e\n\u003c!--Another HTML comment--\u003e\n```python\nprint(\"Hello World!\")\n```\nExpected Output\n```\nHello World!\n```\n~~~\n\nList of Directives\n```\n       Directive HTML comment      |    Use on blocks\n---------------------------------- | ---------------------\n\u003c!--phmdoctest-skip--\u003e             | code, session, output\n\u003c!--phmdoctest-label IDENTIFIER--\u003e | code, session\n\u003c!--phmdoctest-label TEXT--\u003e       | any\n\u003c!--phmdoctest-mark.skip--\u003e        | code\n\u003c!--phmdoctest-mark.skipif\u003c3.N--\u003e  | code\n\u003c!--phmdoctest-setup--\u003e            | code\n\u003c!--phmdoctest-teardown--\u003e         | code\n\u003c!--phmdoctest-share-names--\u003e      | code\n\u003c!--phmdoctest-clear-names--\u003e      | code\n\u003c!--phmdoctest-mark.ATTRIBUTE--\u003e   | code\n```\n\n[Directive hints](#directive-hints)\n\n## skip\nThe skip directive or `--skip TEXT` command line option\nprevents code generation for the code or session block.\nThe skip directive can be placed on an expected output block.\nThere it prevents checking expected against actual output.\n[Example.](#label-skip-and-mark-example)\n\n## label on code and sessions\nWhen used on a Python code block or session the label directive\nchanges the name of the generated test function.\n[Example.](#label-skip-and-mark-example)\nTwo generated tests, the first without a label,\nshown in pytest -v terminal output:\n\n```\ntest_readme.py::test_code_93 FAILED\ntest_readme.py::test_beta_feature FAILED\n```\n\n## label on any fenced code block\nOn any fenced code block, the label directive identifies the block\nfor later retrieval by the class `phmdoctest.tool.FCBChooser()`.\nThe `FCBChooser` is used separately from phmdoctest in\na different pytest file. This allows the test developer to write\nadditional test cases for fenced code blocks that are not handled by\nphmdoctest. The directive value can be any string.\n\n\u003c!--phmdoctest-label my-markdown-file--\u003e\n~~~\n# This is file doc/my_markdown_file.md\n\n\u003c!--phmdoctest-label my-fenced-code-block--\u003e\n```\nThe label directive can be placed on any fenced code block.\n```\n~~~\nHere is Python code to fetch it:\n\n\u003c!--phmdoctest-label fetch-it--\u003e\n```python\nimport phmdoctest.tool\n\nchooser = phmdoctest.tool.FCBChooser(\"doc/my_markdown_file.md\")\ncontents = chooser.contents(label=\"my-fenced-code-block\")\nprint(contents)\n```\nOutput:\n\n\u003c!--phmdoctest-label fetched-contents--\u003e\n```\nThe label directive can be placed on any fenced code block.\n```\n\n## pytest skip\nThe `\u003c!--phmdoctest-mark.skip--\u003e`  directive generates a test\ncase with a `@pytest.mark.skip()` decorator.\n[Example.](#label-skip-and-mark-example)\n\n\n## pytest skipif\nThe `\u003c!--phmdoctest-mark.skipif\u003c3.N--\u003e`  directive generates\na test case with the pytest decorator\n`@pytest.mark.skipif(sys.version_info \u003c (3, N), reason=\"requires \u003e=py3.N\")`.\nN is a Python minor version number.\n[Example.](#label-skip-and-mark-example)\n\n## setup\nA single Python code block can assign names visible to\nother code blocks by adding a setup directive or\nusing the [--setup](#setup-option) command line option.\n\nNames assigned by the setup block\nget copied to the test module's global namespace after\nthe setup block runs.\n\nHere is an example setup block from\n[setup.md](doc/setup.md):\n\u003c!--phmdoctest-label setup-md-first-block--\u003e\n```python\nimport math\n\nmylist = [1, 2, 3]\na, b = 10, 11\n\ndef doubler(x):\n    return x * 2\n```\n\nUsing setup modifies the execution context of the\nPython code blocks in the Markdown file.\nThe names `math`, `mylist`, `a`, `b`, and `doubler` are visible\nto the other Python code blocks. The objects can be modified.\n[Example.](#setup-and-teardown-example)\n\n## teardown\nSelects a single Python code block that runs\nat test module teardown time.\nA teardown block can also be designated\nusing the [--teardown](#teardown-option) command line option.\n[Example.](#setup-and-teardown-example)\n\n## share-names\nNames assigned by the Python code block get copied to\nthe test module as globals after the test code runs. This happens at run\ntime. These names are now visible to subsequent\ntest cases generated for Python code blocks in the Markdown file.\nshare-names modifies the execution context as described for\nthe setup directive above.\nThe share-names directive can be used on more than one\ncode block.\n[Example.](#share-names-clear-names-example)\n\nThis directive effectively joins its Python code block to the\nfollowing Python code blocks in the Markdown file.\n\n## clear-names\nAfter the test case generated for the Python code block\nwith the clear-names directive runs, all names that were\ncreated by one or more preceding share-names directives\nget deleted. The names that were shared are no longer visible.\nThis directive also deletes the names assigned by setup.\n[Example.](#share-names-clear-names-example)\n\n## pytest mark decorator\nThe `\u003c!--phmdoctest-mark.ATTRIBUTE--\u003e` directive adds\na @pytest.mark.ATTRIBUTE decorator to the\ngenerated test function. ATTRIBUTE is a valid Python attribute\nidentifier. This defines a marker to pytest that is used to\nselect and deselect tests. See the pytest documentation section\n\"Working with custom markers\".\nThe file [mark_example.md](doc/mark_example_raw.md) contains\nexample usage of the user defined marker \"slow\". It generates\n[test_mark_example.py](doc/test_mark_example_py.md)\n\n\n## label skip and mark example\nThe file [directive1.md](doc/directive1_raw.md) contains\nexample usage of label, skip, and mark directives.\nThe command below generates\n[test_directive1.py](doc/test_directive1_py.md).\n`phmdoctest doc/directive1.md --report`\nproduces this\n[report](doc/directive1_report_txt.md).\n\n\u003c!--phmdoctest-label directive-1-outfile--\u003e\n```\nphmdoctest doc/directive1.md --outfile test_directive1.py\n```\n\n\n## setup and teardown example\nThe file [directive2.md](doc/directive2_raw.md) contains\nexample usage of label, skip, and mark directives.\nThe command below generates\n[test_directive2.py](doc/test_directive2_py.md).\n`phmdoctest doc/directive2.md --report`\nproduces this\n[report](doc/directive2_report_txt.md).\n\n\u003c!--phmdoctest-label directive-2-outfile--\u003e\n```\nphmdoctest doc/directive2.md --outfile test_directive2.py\n```\n\n## share-names clear-names example\nThe file [directive3.md](doc/directive3_raw.md) contains\nexample usage of share-names and clear-names directives.\nThe command below generates\n[test_directive3.py](doc/test_directive3_py.md).\n`phmdoctest doc/directive3.md --report`\nproduces this\n[report](doc/directive3_report_txt.md).\n\u003c!--phmdoctest-label directive-3-outfile--\u003e\n```\nphmdoctest doc/directive3.md --outfile test_directive3.py\n```\n\n## Configuration\n\nSupply a .ini, .cfg, or .toml configuration file in place of the Markdown file.\nConfiguration features:\n- Choose Markdown files for test file generation. (glob wildcards).\n- Exclude Markdown files from test file generation. (glob wildcards).\n- Name the output directory.\n- Removes stale test files from output directory.\n- Enable printing.\n\nPlace a `[tool.phmdoctest]` section in the configuration file.\n[How to configure.](doc/configuring.md)\n\n## Inline annotations\n\nInline annotations comment out sections of code.\nThey can be added to the end of lines in Python code blocks.\nThey should be in a comment.\n\n- `phmdoctest:omit` comments out a section of code.  The line it is on,\n  plus following lines at greater indent get commented out.\n- `phmdoctest:pass` comments out one line of code and prepends the pass statement.\n\nHere is a snippet showing how to place `phmdoctest:pass` in the code.\nThe second block shows the code that is generated. Note there is no `#`\nimmediately before `phmdoctest:pass`. It is not required.\n\u003c!--phmdoctest-label pass-code--\u003e\n```python\nimport time\ndef takes_too_long():\n    time.sleep(100)    # delay for awhile. phmdoctest:pass\ntakes_too_long()\n```\n\n\u003c!--phmdoctest-label pass-result--\u003e\n```python\nimport time\ndef takes_too_long():\n    pass  # time.sleep(100)    # delay for awhile. phmdoctest:pass\ntakes_too_long()\n```\n\nUse `phmdoctest:omit` on single or multi-line statements. Note the two\ncommented out time.sleep(99). They follow and are indented more\nthat the `if condition:`line with `phmdoctest:omit`.\n\n\u003c!--phmdoctest-label omit-code--\u003e\n```python\nimport time                      # phmdoctest:omit\n\ncondition = True\nif condition:       # phmdoctest:omit\n    time.sleep(99)\n    time.sleep(99)\n```\n\n\u003c!--phmdoctest-label omit-result--\u003e\n```python\n# import time                      # phmdoctest:omit\n\ncondition = True\n# if condition:       # phmdoctest:omit\n#     time.sleep(99)\n#     time.sleep(99)\n```\n\nInline annotation processing counts the number of commented\nout sections and adds the count as the suffix\n`_N` to the name of the pytest function in the\ngenerated test file.\n\nInline annotations are similar, but less powerful\nthan the Python standard library **doctest** directive `#doctest+SKIP`.\nImproper use of `phmdoctest:omit` can cause Python syntax errors.\n\nThe examples above are snippets that illustrate how to\nuse inline annotations.\nHere is an example that produces a pytest file from Markdown.\nThe command below takes [inline_example.md](doc/inline_example.md) and generates\n[test_inline_example.py](doc/test_inline_example_py.md).\n\u003c!--phmdoctest-label inline-outfile--\u003e\n```\nphmdoctest doc/inline_example.md --outfile test_inline_example.py\n```\n\n\n## skipping blocks with skip option\n\nIf you don't want to generate test cases for Python\nblocks precede the block with a **skip** directive or\nuse the `--skip TEXT` option. More than one **skip** directive\nor`--skip TEXT`is allowed.\n\nThe following describes using `--skip TEXT`.\nThe code in each Python block gets searched\nfor the substring `TEXT`.  Zero, one or more blocks will contain\nthe substring. These blocks will not generate test cases in the\noutput file.\n\n- The Python code in the fenced code block gets searched.\n- The info string is **not** searched.\n- Output blocks are **not** searched.\n- Both Python code and session blocks get searched.\n- Case is significant.\n\nThe report shows which Python blocks get skipped\nin the test role column, and the Python blocks that\nmatched each --skip TEXT in the skips section.\n\nThis option makes it **very easy** to **inadvertently exclude**\nPython blocks from the test cases.  In the event no test cases get\ngenerated, the option `--fail-nocode` described below is useful.\n\nThree special `--skip TEXT` strings work a little differently.\nThey select one of the first, second, or last of the Python blocks.\nOnly Python blocks get counted.\n- `--skip FIRST` skips the first Python block.\n- `--skip SECOND` skips the second Python block.\n- `--skip LAST` skips the final Python block.\n\n## skip option\n\nThis command using `--skip`:\n\n\u003c!--phmdoctest-label skip-command--\u003e\n```\nphmdoctest doc/example2.md --skip \"Python 3.7\" --skip LAST --report --outfile test_example2.py\n```\n\nProduces the report\n\n\u003c!--phmdoctest-label skip-report--\u003e\n```\n            doc/example2.md fenced blocks\n-----------------------------------------------------\nblock     line  test          TEXT or directive\ntype    number  role          quoted and one per line\n-----------------------------------------------------\npython       9  code\n            14  output\npython      20  skip-code     \"Python 3.7\"\n            26  skip-output\n            31  --\npython      37  code\npython      44  code\n            51  output\nyaml        59  --\ntext        67  --\npy          75  session\npython      87  code\n            94  output\npy         102  skip-session  \"LAST\"\n-----------------------------------------------------\n5 test cases.\n1 skipped code blocks.\n1 skipped interactive session blocks.\n1 code blocks with no output block.\n\n  skip pattern matches (blank means no match)\n------------------------------------------------\nskip pattern  matching code block line number(s)\n------------------------------------------------\nPython 3.7    20\nLAST          102\n------------------------------------------------\n```\n\ncreates the output file [test_example2.py](doc/test_example2_py.md)\n\n\n## short form of skip option\n\nThis is the same command as above using the short `-s` form of the `--skip` option\nin two places.\nIt produces the same report and outfile.\n\u003c!--phmdoctest-label short-skip-command--\u003e\n```\nphmdoctest doc/example2.md -s \"Python 3.7\" -sLAST --report --outfile test_example2.py\n```\n\n## fail-nocode option\n\nThe `--fail-nocode` option produces a pytest file that will always\nfail when no Python code or session blocks get found.\n\nEvem if no Python code or session blocks exist in the\nMarkdown file a pytest file gets generated.\nThis also happens when `--skip` eliminates all the\nPython code blocks.\nThe generated pytest file will have the function\n`def test_nothing_passes()`.\n\nIf the option `--fail-nocode` is passed the\nfunction is `def test_nothing_fails()` which raises an\nassertion.\n\n## setup option\n\nA single Python code block can assign names visible to\nother code blocks by giving the `--setup TEXT` option.\nPlease see the [setup](#setup) directive above.\nThe rules for `TEXT` are the same as for `--skip TEXT` plus...\n\n- Only one block can match `TEXT`.\n- The block cannot match a block that is skipped.\n- The block cannot be a session block even though session\n  blocks get searched for `TEXT`.\n- It is ok if the block has an output block. It will be ignored.\n\n\n## teardown option\n\nA single Python code block can supply code run by the pytest\n`teardown_module()` fixture. Use the `--teardown TEXT` option.\nPlease see the [teardown](#teardown) directive above.\nThe rules for `TEXT` are the same as for `--setup` above except\n`TEXT` won't match a setup block.\n\n## Setup example\n\nFor the Markdown file [setup.md](doc/setup.md)\nrun this command to see how the blocks get tested.\n\n\u003c!--phmdoctest-label setup-command-report--\u003e\n```\nphmdoctest doc/setup.md --setup FIRST --teardown LAST --report\n```\n\n\u003c!--phmdoctest-label setup-report--\u003e\n```\n            doc/setup.md fenced blocks\n-------------------------------------------------\nblock     line  test      TEXT or directive\ntype    number  role      quoted and one per line\n-------------------------------------------------\npython       9  setup     \"FIRST\"\npython      20  code\n            27  output\npython      37  code\n            42  output\npython      47  code\n            51  output\npython      58  teardown  \"LAST\"\n-------------------------------------------------\n3 test cases.\n```\n\nThis command\n\u003c!--phmdoctest-label setup-command-outfile--\u003e\n```\nphmdoctest doc/setup.md --setup FIRST --teardown LAST --outfile test_setup.py\n```\ncreates the test file\n[test_setup.py](doc/test_setup_py.md)\n\n## Setup for sessions\nThe pytest option `--doctest-modules` is required to\nrun doctest on sessions.  pytest runs doctests in\na separate context.\nFor more on this see [Execution context](#execution-context) below.\n\nTo allow sessions to see the variables assigned by the `--setup`\ncode block, add the option `--setup-doctest`\n\nHere is an example with setup code and sessions\n[setup_doctest.md](doc/setup_doctest.md). The first part\nof this file is a copy of setup.md.\n\nThis command  uses the short form of setup and teardown.\n-u for set**up** and -d for tear**down**.\n\u003c!--phmdoctest-label setup-doctest-outfile--\u003e\n```\nphmdoctest doc/setup_doctest.md -u FIRST -d LAST --setup-doctest --outfile test_setup_doctest.py\n```\nIt creates the test file\n[test_setup_doctest.py](doc/test_setup_doctest_py.md)\n\n## Execution context\n\nWhen run without `--setup`\n\n- pytest and doctest determine the order of test case execution.\n- phmdoctest assumes test code and session execution is in file order.\n- Test case order is not significant.\n- Code and expected output run within a function body of a pytest test case.\n- If pytest is invoked with `--doctest-modules`:\n  - Sessions are run in a separate doctest execution context.\n  - Otherwise, sessions do not run.\n\n### With `--setup`\n\n- Names assigned by setup code are visible to code blocks.\n- Code blocks can modify the objects created by the setup code.\n- Code block test case order is significant.\n- Session order is not significant.\n- If pytest is run with `--doctest-modules`:\n  - pytest runs two separate contexts: one for sessions, one for code blocks.\n  - setup and teardown code gets run twice, once by each context.\n  - the names assigned by the setup code block\n    are `are not` visible to the sessions.\n\n### With `share-names`\n- Only following code blocks can modify the shared objects.\n- Shared objects will **not** be visible to sessions\n  if pytest is run with `--doctest-modules`.\n- After running a code block with `clear-names`\n  - Shared objects will no longer be visible.\n  - Names assigned by setup code will no longer be visible.\n\n### With `--setup` and `--setup-doctest`\nSame as the setup section plus:\n- names assigned by the setup code block\n  are visible to the sessions.\n- Sessions can modify the objects created by the setup code.\n- Session order is significant.\n- Sessions and code blocks are still running in separate contexts\n  isolated from each other.\n- A session can't affect a code block, and a code block can't affect\n  a session.\n- Names assigned by the setup code block are globally visible\n  to the entire test suite via the pytest doctest_namespace\n  fixture.  See hint near the end [Hints](#hints).\n\n### pytest live logging demo\nThe live logging demos reveals pytest execution contexts.\npytest Live Logs show the\nexecution order of setup_module(), test cases, sessions, and\nteardown_module().\nThere are 2 demo invocations in the workflow action\ncalled pytest Live Log Demo.  GitHub login required.\n\n\n## Send outfile to stdout\nTo redirect the above outfile to the standard output stream use one\nof these two commands.\n\nBe sure to leave out `--report` when sending --outfile to standard output.\n\u003c!--phmdoctest-label outfile-dash1--\u003e\n```\nphmdoctest doc/example2.md -s \"Python 3.7\" -sLAST --outfile -\n```\nor\n\u003c!--phmdoctest-label outfile-dash2--\u003e\n```\nphmdoctest doc/example2.md -s \"Python 3.7\" -sLAST --outfile=-\n```\n\n## Usage\n\n`phmdoctest --help`\n\n\u003c!--phmdoctest-label usage--\u003e\n```\nUsage: phmdoctest [OPTIONS] MARKDOWN_FILE\n\n  MARKDOWN_FILE may also be .toml, .cfg, or .ini configuration file.\n\nOptions:\n  --outfile TEXT       Write generated test case file to path TEXT. \"-\" writes\n                       to stdout.\n\n  -s, --skip TEXT      Any Python code or interactive session block that\n                       contains the substring TEXT is not tested. More than\n                       one --skip TEXT is ok. Double quote if TEXT contains\n                       spaces. For example --skip=\"python 3.7\" will skip every\n                       Python block that contains the substring \"python 3.7\".\n                       If TEXT is one of the 3 capitalized strings FIRST\n                       SECOND LAST the first, second, or last Python code or\n                       session block in the Markdown file is skipped.\n\n  --report             Show how the Markdown fenced code blocks are used.\n\n  --fail-nocode        This option sets behavior when the Markdown file has no\n                       Python fenced code blocks or interactive session blocks\n                       or if all such blocks are skipped. When this option is\n                       present the generated pytest file has a test function\n                       called test_nothing_fails() that will raise an\n                       assertion. If this option is not present the generated\n                       pytest file has test_nothing_passes() which will never\n                       fail.\n\n  -u, --setup TEXT     The Python code block that contains the substring TEXT\n                       is run at test module setup time. Variables assigned at\n                       the outer level are visible as globals to the other\n                       Python code blocks. TEXT should match exactly one code\n                       block. If TEXT is one of the 3 capitalized strings\n                       FIRST SECOND LAST the first, second, or last Python\n                       code or session block in the Markdown file is matched.\n                       A block will not match --setup if it matches --skip, or\n                       if it is a session block. Use --setup-doctest below to\n                       grant Python sessions access to the globals.\n\n  -d, --teardown TEXT  The Python code block that contains the substring TEXT\n                       is run at test module teardown time. TEXT should match\n                       exactly one code block. If TEXT is one of the 3\n                       capitalized strings FIRST SECOND LAST the first,\n                       second, or last Python code or session block in the\n                       Markdown file is matched. A block will not match\n                       --teardown if it matches either --skip or --setup, or\n                       if it is a session block.\n\n--setup-doctest        Make globals created by the --setup Python code block\n                       or setup directive visible to session blocks and only\n                       when they are tested with the pytest --doctest-modules\n                       option.  Please note that pytest runs doctests in a\n                       separate context that only runs doctests. This option\n                       is ignored if there is no --setup option.\n\n  --version            Show the version and exit.\n  --help               Show this message and exit.\n```\n\n## Run as a Python module\n\nTo run phmdoctest from the command line:\n\n`python -m phmdoctest doc/example2.md --report`\n\n\n## Python API\n\nCall **main.testfile()** to generate a pytest file in memory.\nPlease see the Python API [here][10].\nThe example generates a pytest file from doc/setup.md and\ncompares the result to doc/test_setup.py.\n\u003c!--phmdoctest-label main-testfile--\u003e\n```python\nfrom pathlib import Path\nimport phmdoctest.main\n\ngenerated_testfile = phmdoctest.main.testfile(\n    \"doc/setup.md\",\n    setup=\"FIRST\",\n    teardown=\"LAST\",\n)\nexpected = Path(\"doc/test_setup.py\").read_text(encoding=\"utf-8\")\nassert expected == generated_testfile\n```\n\n## pytest fixtures\n\nUse fixture **testfile_creator** to generate a test file in memory.\nPass the test file to fixture **testfile_tester** to run\nthe test file in the pytester environment.\n[Fixture API][10] | [Example](doc/project_test_py.md).\nSee more uses in tests/test_examples.py, tests/test_details.py, and\ntests/test_many_markdown.py.\nThe fixtures run pytest much faster than `run_and_pytest()`\nbelow since there is no subprocess call.\nIn the readthedocs documentation see the section Development tools API 1.4.0.\npytest's pytester is suitable for pytest plugin development.\n\n## Simulate command line\n\nTo simulate a command line call to phmdoctest from\nwithin a Python script `phmdoctest.simulator` offers the\nfunction `run_and_pytest()`.\n- it creates the --outfile in a temporary directory\n- optionally runs pytest on the outfile\n- pytest can return a JUnit XML report\n- useful during development to validate the command line\n  and prevent use of a stale --outfile\n\nPlease see the [Latest Development tools API section][10] or\nthe docstring of the function `run_and_pytest()` in the file `simulator.py.`\nPass pytest_options as a list of strings as shown below.\n\n\u003c!--phmdoctest-label simulator--\u003e\n```python\nimport phmdoctest.simulator\n\ncommand = \"phmdoctest doc/example1.md --report --outfile temporary.py\"\nsimulator_status = phmdoctest.simulator.run_and_pytest(\n    well_formed_command=command, pytest_options=[\"--doctest-modules\", \"-v\"]\n)\nassert simulator_status.runner_status.exit_code == 0\nassert simulator_status.pytest_exit_code == 0\n```\n\n## Hints\n\n- To read the Markdown file from the standard input stream.\n  Use `-` for MARKDOWN_FILE.\n- Write the test file to a temporary directory so that\n  it is always up to date.\n- In CI scripts the following shell command will create the temporary\n  directory **tmp** in the **tests** folder on Windows, Linux, and macOS.\n  ```bash\n  python -c \"from pathlib import Path; d = Path('tests') / 'tmp'; d.mkdir(mode=0o700)\"\n  ```\n- It is easy to use --output by mistake instead of `--outfile`.\n- If Python code block has no output, put assert statements in the code.\n- Use pytest option `--doctest-modules` to test the sessions.\n- Markdown indented code blocks ([Spec][8] section 4.4) are ignored.\n- simulator_status.runner_status.exit_code == 2 is the click\n  command line usage error.\n- Since phmdoctest generates code, the input file should be from a trusted\n  source.\n- An empty code block gets given the role `del-code`. It is not tested.\n- Use special TEXT values FIRST, SECOND, LAST for the command\n  line options `--setup` and `--teardown` since they only match one block.\n- The variable names `managenamespace`, `doctest_namespace`,\n  `capsys`, and `_phm_expected_str` should not be used in\n  Markdown Python code blocks since they may be used in generated code.\n- Setup and teardown code blocks cannot have expected output.\n- To have pytest collect a code block with the label directive\n  start the value with `test_`.\n- With the `--setup-doctest` option, names assigned by the setup code\n  block are globally visible to the entire test suite.\n  This is due to the scope of the pytest doctest_namespace\n  fixture.  Try using a separate pytest command to test\n  just the phmdoctest test.\n- The module **phmdoctest.fixture** is imported at pytest time\n  to support setup, teardown, share-names, and clear-names features.\n- The phmdoctest Markdown parser finds fenced code blocks enclosed by\n  html `\u003cdetails\u003e` and `\u003c/details\u003e` tags.\n  The tags may require a preceding and trailing blank line\n  to render correctly. See example in tests/test_details.py.\n- Try redirecting phmdoctest standard output into PYPI Pygments to\n  colorize the generated test file.\n  ```shell\n  python -m phmdoctest project.md --outfile - | pygmentize\n  ```\n\n- If the --outfile is written into a folder that pre-exists in the\n  repository, consider adding the outfile name to .gitignore. If\n  the outfile name later changes, the change will be needed in\n  .gitignore too.\n  ```\n  # Reserved for generated test file.\n  tests/test_readme.py\n  ```\n\n## Directive hints\n\n- Only put one of setup, teardown, share-names, or\n  clear-names on a code block.\n- Only one block can be setup. Only one block can be teardown.\n- The setup or teardown block can't have an expected output block.\n- Label directive does not generate a test\n  case name on setup and teardown blocks.\n- Directives displayed in the `--report` start with a dash like\n  this: `-label test_i_ratio`.\n- Code generated by Python blocks with setup and teardown\n  directives runs at the pytest fixture `scope=\"module\"` level.\n- Code generated by Python blocks with share-names and\n  clear-names directives are **collected** and run by pytest\n  like any other test case.\n- A malformed HTML comment ending is bad. Make sure\n  it ends with both dashes like `--\u003e`.  Running with `--report`\n  will expose that problem.\n- The setup, teardown, share-names, and clear-names directives\n  have logging. To see the log messages,\n  run pytest with the option:\n  `--log-cli-level=DEBUG --color=yes`\n- There is no limit to number of blank lines after\n  the directive HTML comment but before the fenced code block.\n- The directive `\u003c!--phmdoctest-mark.xfail--\u003e` might be useful as\n  an alternative to `\u003c!--phmdoctest-mark.skip--\u003e` for failing examples.\n- The directive `\u003c!--phmdoctest-mark.ATTRIBUTE--\u003e` will not be\n  effective when used with `\u003c!--phmdoctest-setup--\u003e` or\n   `\u003c!--phmdoctest-teardown--\u003e` because pytest marks can only\n  be applied to tests. They have no effect on fixtures.\n  Setup and teardown use fixtures.\n\n## Related projects\n- rundoc\n- byexample\n- sphinx.ext.doctest\n- sybil\n- doxec\n- egtest\n- pytest-phmdoctest\n- pytest-codeblocks\n\n[3]: https://github.github.com/gfm/#fenced-code-blocks\n[11]: https://github.github.com/gfm/#info-string\n[10]: https://phmdoctest.readthedocs.io/en/latest/doc/api.html\n[7]: https://pypi.org/project/commonmark\n[8]: https://spec.commonmark.org\n[9]: https://commonmark.org\n[4]: https://docs.python.org/3/library/doctest.html\n[6]: https://pypi.org/project/coverage\n[13]: https://ci.appveyor.com/project/tmarktaylor/phmdoctest\n[14]: https://travis-ci.org/tmarktaylor/monotable\n[15]: https://docs.pytest.org/en/stable\n[16]: https://pypi.org/project/pytest-phmdoctest\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftmarktaylor%2Fphmdoctest","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftmarktaylor%2Fphmdoctest","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftmarktaylor%2Fphmdoctest/lists"}