{"id":34066393,"url":"https://github.com/tmarktaylor/pytest-phmdoctest","last_synced_at":"2026-04-20T08:05:10.120Z","repository":{"id":42050027,"uuid":"426278686","full_name":"tmarktaylor/pytest-phmdoctest","owner":"tmarktaylor","description":"Python syntax highlighted Markdown doctest pytest plugin.","archived":false,"fork":false,"pushed_at":"2022-04-15T12:28:36.000Z","size":170,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-12-16T05:59:52.023Z","etag":null,"topics":["doctest","markdown","pytest-plugin","python"],"latest_commit_sha":null,"homepage":"https://tmarktaylor.github.io/pytest-phmdoctest/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tmarktaylor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-11-09T15:17:26.000Z","updated_at":"2021-11-10T20:06:56.000Z","dependencies_parsed_at":"2022-08-12T03:21:09.973Z","dependency_job_id":null,"html_url":"https://github.com/tmarktaylor/pytest-phmdoctest","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/tmarktaylor/pytest-phmdoctest","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmarktaylor%2Fpytest-phmdoctest","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmarktaylor%2Fpytest-phmdoctest/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmarktaylor%2Fpytest-phmdoctest/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmarktaylor%2Fpytest-phmdoctest/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tmarktaylor","download_url":"https://codeload.github.com/tmarktaylor/pytest-phmdoctest/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmarktaylor%2Fpytest-phmdoctest/sbom","scorecard":{"id":889650,"data":{"date":"2025-08-11","repo":{"name":"github.com/tmarktaylor/pytest-phmdoctest","commit":"4c37735c3273ff35dabc9516008965c21931b9b2"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.2,"checks":[{"name":"Code-Review","score":0,"reason":"Found 0/1 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/ci.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:42: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/pytest-phmdoctest/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:44: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/pytest-phmdoctest/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:72: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/pytest-phmdoctest/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:75: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/pytest-phmdoctest/ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:102: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/pytest-phmdoctest/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:115: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/pytest-phmdoctest/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:117: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/pytest-phmdoctest/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:135: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/pytest-phmdoctest/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:137: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/pytest-phmdoctest/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:167: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/pytest-phmdoctest/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:169: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/pytest-phmdoctest/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:183: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/pytest-phmdoctest/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:19: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/pytest-phmdoctest/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:21: update your workflow using https://app.stepsecurity.io/secureworkflow/tmarktaylor/pytest-phmdoctest/ci.yml/main?enable=pin","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:122","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:123","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:124","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:125","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:142","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:143","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:144","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:175","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:176","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:27","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:28","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:29","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:30","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:61","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:62","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:63","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:64","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:80","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:81","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:86","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:88","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:89","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:96","Info:   0 out of  13 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   1 third-party GitHubAction dependencies pinned","Info:   1 out of  24 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE.txt:0","Info: FSF or OSI recognized license: MIT License: LICENSE.txt:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":7,"reason":"3 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: PYSEC-2020-92 / GHSA-hj5v-574p-mj7c","Warn: Project is vulnerable to: PYSEC-2022-42969","Warn: Project is vulnerable to: PYSEC-2024-48 / GHSA-fj7x-q9j7-g6q6"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 30 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-24T11:17:59.172Z","repository_id":42050027,"created_at":"2025-08-24T11:17:59.172Z","updated_at":"2025-08-24T11:17:59.172Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32038456,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-20T00:18:06.643Z","status":"online","status_checked_at":"2026-04-20T02:00:06.527Z","response_time":94,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["doctest","markdown","pytest-plugin","python"],"created_at":"2025-12-14T06:34:39.699Z","updated_at":"2026-04-20T08:05:10.106Z","avatar_url":"https://github.com/tmarktaylor.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pytest-phmdoctest 1.0.0\n\n## Introduction\n\nPython syntax highlighted Markdown doctest pytest plugin.\n\nA [pytest][4] plugin based on the [phmdoctest][3] command line tool.\n\nIf you have Python syntax highlighted examples in Markdown\nlike this Python code...\n```python\nprint(\"Hello World!\")\n```\nplus expected output.\n```\nHello World!\n```\n\nThis [pytest][4] plugin will test them, as is, without edits.\nOn this file try the command ...\n```shell\npytest -v --phmdoctest README.md\n```\n\npytest console output ...\n```\nplugins: phmdoctest-1.0.0\ncollected 1 item\n\n::README.py::test_code_12_output_16 PASSED\n```\n\nThe plugin also tests Python interactive sessions\ndescribed by [doctest][1].\nSee [testing examples](#testing-python-interactive-sessions) below.\n\nThe `12` in the function name `test_code_12_output_16` is the\nline number of the first line\nof python code. `16` shows the line number of the expected\nterminal output.\n\n\n## Description\n\nThe plugin is based on the Python tool\n[phmdoctest][3] version \u003e= 1.4.0\nIt generates a pytest test file from a Markdown file.\n\n- Reads these from Markdown fenced code blocks:\n  - Python source code and expected terminal output.\n  - Python interactive sessions described by [doctest][1].\n- Simple use case is possible with no Markdown edits.\n- More features selected by adding HTML comment directives\n  to the Markdown. See Directives in [phmdoctest Directives][5].\n- Run on pytest \u003e= 5.4.3.\n\n### main branch status\n[![](https://img.shields.io/pypi/l/pytest-phmdoctest.svg)](https://github.com/tmarktaylor/pytest-phmdoctest/blob/main/LICENSE.txt)\n[![](https://img.shields.io/pypi/v/pytest-phmdoctest.svg)](https://pypi.python.org/pypi/pytest-phmdoctest)\n[![](https://img.shields.io/pypi/pyversions/pytest-phmdoctest.svg)](https://pypi.python.org/pypi/pytest-phmdoctest)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n\n[![CI Test](https://github.com/tmarktaylor/pytest-phmdoctest/actions/workflows/ci.yml/badge.svg)](https://github.com/tmarktaylor/pytest-phmdoctest/actions/workflows/ci.yml)\n[![Build status](https://ci.appveyor.com/api/projects/status/fa0frmueq4h94v23/branch/main?svg=true)](https://ci.appveyor.com/project/tmarktaylor/pytest-phmdoctest/branch/main)\n[![readthedocs](https://readthedocs.org/projects/pytest-phmdoctest/badge/?version=latest)](https://pytest-phmdoctest.readthedocs.io/en/latest/?badge=latest)\n[![codecov](https://codecov.io/gh/tmarktaylor/pytest-phmdoctest/branch/main/graph/badge.svg?token=j5uu3WJn6u)](https://codecov.io/gh/tmarktaylor/pytest-phmdoctest/branch/main)\n\n[Website](https://tmarktaylor.github.io/pytest-phmdoctest) |\n[Docs](https://pytest-phmdoctest.readthedocs.io/en/latest/) |\n[Repos](https://github.com/tmarktaylor/pytest-phmdoctest) |\n[pytest](https://ci.appveyor.com/project/tmarktaylor/pytest-phmdoctest/branch/main) |\n[Codecov](https://codecov.io/gh/tmarktaylor/pytest-phmdoctest/branch/main) |\n[License](https://github.com/tmarktaylor/pytest-phmdoctest/blob/main/LICENSE.txt)\n\n\n[Introduction](#introduction) |\n[Description](#description) |\n[Installation](#installation) |\n[Usage](#usage) |\n[Testing Python interactive sessions](#testing-python-interactive-sessions) |\n[Generate test files](#generate-test-files) |\n[Generate and collect test files](#generate-and-collect-test-files) |\n[Help](#help) |\n[Configure collection](#configure-collection) |\n[Hints](#hints) |\n[Related projects](#related-projects)\n\n\n[Changes](doc/recent_changes.md) |\n[Contributions](CONTRIBUTING.md)\n\n\n## Installation\nIt is advisable to install in a virtual environment.\n\n    python -m pip install pytest-phmdoctest\n\n## Usage\n\nConsider a project with the following files.\nNot all files are shown.\n\n```text\nREADME.md\ndoc/directive2.md\ndoc/nocode.md\ntests/test_example.py\n```\n\n\u003c!--phmdoctest-label plugin-inactive-command--\u003e\n```shell\npytest -v\n```\n\n\u003c!--phmdoctest-label plugin-inactive-output--\u003e\n```text\nplugins: phmdoctest-1.0.0\ncollected 1 item\n\ntests/test_example.py::test_example PASSED\n```\n\nUse `--phmdoctest` to also collect Markdown files.\n\n\u003c!--phmdoctest-label phmdoctest-option-command--\u003e\n```shell\npytest -v --phmdoctest\n```\n\n\u003c!--phmdoctest-label phmdoctest-option-output--\u003e\n```text\nplugins: phmdoctest-1.0.0\ncollected 6 items\n\n::README.py::test_code_10_output_17 PASSED\n::doc__directive2.py::test_code_25_output_32 PASSED\n::doc__directive2.py::test_code_42_output_47 PASSED\n::doc__directive2.py::test_code_52_output_56 PASSED\n::doc__project.py::test_code_12_output_19 PASSED\ntests/test_example.py::test_example PASSED\n```\n- The sample project above can be viewed on GitHub at `tests/sample`.\n- The doc__ indicates the Markdown file was collected from the\n  doc folder.\n- Markdown \".md\" files get discovered by pytest.\n  pytest finds them in the same way it finds Python test files.\n  For each Markdown file discovered, the plugin generates a pytest\n  test file.\n- A Markdown file that does not have any Python fenced code block examples\n  is not tested.\n- To avoid collecting .md files use pytest `--ignore` and `--ignore-glob`\n  on the command line or in the addopts part of the pytest ini file.\n  These commands work on .md files and use Unix shell-style wildcards.\n\nUse `--phmdoctest` in a pytest ini file instead of on the\ncommand line as shown below:\n```ini\n# pytest.ini and tox.ini\n[pytest]\naddopts = --phmdoctest\n```\n```toml\n# pyproject.toml\n[tool.pytest.ini_options]\naddopts = \"--phmdoctest\"\n```\n```ini\n# setup.cfg  Note: Use is discouraged by pytest docs.\n[tool:pytest]\naddopts = --phmdoctest\n```\n\n## Testing Python interactive sessions\n\nThe plugin also tests Python interactive sessions described by [doctest][1]\nlike this one:\n\n```python\n\u003e\u003e\u003e import math\n\u003e\u003e\u003e math.floor(9.1)\n9\n```\n\nUse the `--phmdoctest-docmod` option to collect both Python code/expected output\nand Python interactive sessions.\n\n\u003c!--phmdoctest-label phmdoctest-docmod-command--\u003e\n```shell\npytest -v --phmdoctest-docmod\n```\n\n\u003c!--phmdoctest-label  phmdoctest-docmod-output--\u003e\n```text\nplugins: phmdoctest-1.0.0\ncollected 10 items\n\n::README.py::README.session_00001_line_24 PASSED\n::README.py::test_code_10_output_17 PASSED\n::doc__directive2.py::test_code_25_output_32 PASSED\n::doc__directive2.py::test_code_42_output_47 PASSED\n::doc__directive2.py::test_code_52_output_56 PASSED\n::doc__project.py::doc__project.session_00001_line_31 PASSED\n::doc__project.py::doc__project.session_00002_line_46 PASSED\n::doc__project.py::doc__project.session_00003_line_55 PASSED\n::doc__project.py::test_code_12_output_19 PASSED\ntests/test_example.py::test_example PASSED\n```\n\n- The `--phmdoctest-docmod` option uses a non-public pytest class DoctestModule.\n  There is a slight chance a pytest future\n  major release changes the DoctestModule API.\n- DoctestModule works ok on pytest major releases 5, 6, and 7\n  as verified by tests in .github/workflows/ci.yml.\n- If  `--phmdoctest-docmod` doctest collection breaks,\n  the rest of the plugin `--phmdoctest` and `--phmdoctest-generate`\n  options still work. The test suite simulates such breaking\n  changes. See the test_readme and test_docmod.py test functions\n  that take the monkeypatch fixture.\n- The `line_24` in the function name `session_00001_line_24` is the\n  line number in tests/sample/README.md of the first line\n  of the interactive session. `00001` is a sequence number to\n  order the doctests.\n\nHere is simulated output captured when `--phmdoctest-docmod`\ndoctest collection breaks due to incompatible DoctestModule API.\n\n\u003c!--phmdoctest-label phmdoctest-bad-api-command--\u003e\n```shell\npytest -v --phmdoctest-docmod\n```\n\n\u003c!--phmdoctest-label phmdoctest-bad-api-output--\u003e\n```text\nplugins: phmdoctest-1.0.0\ncollected 8 items\n\n::README.py::test_code_10_output_17 PASSED\n::README.py::test_unable_to_collect_doctests FAILED\n::doc__directive2.py::test_code_25_output_32 PASSED\n::doc__directive2.py::test_code_42_output_47 PASSED\n::doc__directive2.py::test_code_52_output_56 PASSED\n::doc__project.py::test_code_12_output_19 PASSED\n::doc__project.py::test_unable_to_collect_doctests FAILED\ntests/test_example.py::test_example PASSED\n```\n- There is one FAILED test called for each Markdown file with\n  Python interactive sessions.\n- The Python code/expected output examples still run successfully.\n- The pytest test case in tests/test_example.py succeeds.\n\n## Generate test files\n\nSave generated test files to the file system. Do not collect them.\nThe plugin does not use the non-public DoctestModule when doing this.\n\n\u003c!--phmdoctest-label phmdoctest-generate-command--\u003e\n```shell\npytest -v --phmdoctest-generate .gendir\n```\n\n\u003c!--phmdoctest-label phmdoctest-generate-output--\u003e\n```text\nplugins: phmdoctest-1.0.0\ncollected 1 item\n\ntests/test_example.py::test_example PASSED\n```\n\n- Note that only test_example.py was collected.\n- With `--phmdoctest-generate` the test files\n  generated from Markdown do not get collected.\n- Run pytest again on the generated test files.\n- The generated test files become stale with time.\n- Test files should be regenerated after modifying the Markdown.\n- See below to generate and collect test files with\n  a single pytest command.\n- .gendir is cleaned of all *.py\n  files before writing new test files.\n  Pre-existing *.py files in the output directory get renamed. If\n  output_directory inadvertently gets pointed at a Python\n  source directory, the renamed files can be recovered by renaming them.\n  - The FILENAME.py files found in the output directory are renamed\n    to noFILENAME.sav.\n  - If a noFILENAME.sav already exists it is not modified.\n  - Files in target_dir with other extensions are not modified.\n  - A FILENAME.py pre-existing in target_dir is only renamed\n    and not deleted.\n- .gendir is cleaned of all *.md files as well.\n  Pre-existing FILENAME.md files in the output directory get renamed\n  to FILENAME_md.sav.\n- If .gendir was empty, it will now have these *.py files:\n\n\u003c!--phmdoctest-label gendir-files--\u003e\n```\ntest_doc__directive2.py\ntest_doc__project.py\ntest_README.py\n```\n\n## Generate and collect test files\n\nA single pytest command will generate test files and collect them.\nThe plugin does not use the non-public DoctestModule when doing this.\n\n\u003c!--phmdoctest-label generate-collect-command--\u003e\n```shell\npytest -v --phmdoctest-generate .gendir . .gendir --doctest-modules --ignore src\n```\n\n\u003c!--phmdoctest-label generate-collect-output--\u003e\n```text\nplugins: phmdoctest-1.0.0\ncollected 10 items\n\ntests/test_example.py::test_example PASSED\n.gendir/test_README.py::test_README.session_00001_line_24 PASSED\n.gendir/test_README.py::test_code_10_output_17 PASSED\n.gendir/test_doc__directive2.py::test_code_25_output_32 PASSED\n.gendir/test_doc__directive2.py::test_code_42_output_47 PASSED\n.gendir/test_doc__directive2.py::test_code_52_output_56 PASSED\n.gendir/test_doc__project.py::test_doc__project.session_00001_line_31 PASSED\n.gendir/test_doc__project.py::test_doc__project.session_00002_line_46 PASSED\n.gendir/test_doc__project.py::test_doc__project.session_00003_line_55 PASSED\n.gendir/test_doc__project.py::test_code_12_output_19 PASSED\n```\n\nHow it works:\n- The plugin writes the generated test files during\n  the pytest collection phase.\n- This happens while pytest is collecting from `\".\"`\n- pytest does not collect from `.gendir` until after discovering\n  and collecting files in `\".\"`.\n- The leading `\".\"` in `.gendir` prevents pytest from searching\n  there for test files while searching `\".\"`.\n  See norecursedirs default values in Pytest Documentation |\n  API reference | Configuration Options | norecursedirs.\n- The `--doctest-modules` option tells pytest to look for doctests in\n  docstrings of *.py files.\n- When doing `--doctest-modules`, the `--ignore src` option tells\n  pytest not to collect modules from the src folder. We only\n  want to collect doctests from .gendir.\n  This prevents pytest from importing modules\n  there to look for doctests.\n\nThese are the ini file settings:\n\n```ini\n# pytest.ini and tox.ini\n[pytest]\naddopts = --phmdoctest-generate=.gendir --doctest-modules --ignore src\n```\n```toml\n# pyproject.toml\n[tool.pytest.ini_options]\naddopts = \"--phmdoctest-generate=.gendir --doctest-modules --ignore src\"\n```\n```ini\n# setup.cfg  Note: Use is discouraged by pytest docs.\n[tool:pytest]\naddopts = --phmdoctest-generate=.gendir --doctest-modules --ignore src\n```\n\nHere is a demo that runs on a checked out copy of the repository.\n\n```\n# With a terminal in the tests/sample directory\n# The first line collects 5 items.\n# The second line collects 3 items.\n\npytest -v --phmdoctest-generate=.gendir \".\" .gendir --ignore README.md --ignore doc/directive2.md --doctest-modules --ignore src\npytest -v --phmdoctest-generate=.gendir \".\" .gendir  --ignore-glob */*.md --doctest-modules --ignore src\n```\n\n## Help\n\npytest `--help` contains a **phmdoctest:** group in the middle and\nan ini-option near the bottom.  The help contains:\n- `--phmdoctest`\n- `--phmdoctest-generate`\n- `--phmdoctest-docmod`\n- `phmdoctest-collect`\n\n## Configure collection\n\nAn optional `phmdoctest-collect = ` section can be placed in the pytest\nini file. It is a list of lines of the format\n\n    glob [options]\n\nConsider using the section to pass collection options described below.\nThe options have the same names and behave like the options\naccepted by [phmdoctest usage][6].\n\n- The Markdown file must match one of the globs.\n- The glob is processed by Path.glob() from the Python standard library pathlib.\n  Path.glob() offers a \"**\" recursive pattern that means\n  \"this directory and all subdirectories recursively.\"\n- The globs are checked from top to bottom. The first glob to match the Markdown\n  file determines the phmdoctest command line options.\n- If there is no match the file will **not** be collected.\n- A line can have just the glob and no options. The glob is required.\n- The options should look like and have the same spacing as the command line\n  options passed to the tool [phmdoctest usage][6].\n  - Use double quotes as needed in TEXT.\n  - The plugin does not support `\\\"` escaped double quote.\n  - Look for list of options in the next section.\n- If a line that does not parse is needed, the plugin collects\n  a special test file that contains a failing test case with\n  an embedded error message.\n\nExample\n```ini\n# pytest.ini\n[pytest]\naddopts = --phmdoctest-docmod\nphmdoctest-collect =\n    doc/project.md --skip greeting --skip enjoyment\n    **/*.md\n```\nThen run this pytest command on the project files from\nthe Usage section ...\n\n\u003c!--phmdoctest-label collect-section-command--\u003e\n```shell\npytest -v --ignore tests/test_example.py\n```\noutput\n\u003c!--phmdoctest-label collect-section-output--\u003e\n```text\nplugins: phmdoctest-1.0.0\ncollected 7 items\n\n::README.py::README.session_00001_line_24 PASSED\n::README.py::test_code_10_output_17 PASSED\n::doc__directive2.py::test_code_25_output_32 PASSED\n::doc__directive2.py::test_code_42_output_47 PASSED\n::doc__directive2.py::test_code_52_output_56 PASSED\n::doc__project.py::doc__project.session_00001_line_31 PASSED\n::doc__project.py::doc__project.session_00002_line_46 PASSED\n```\n\n- The example passes the options `--skip greeting` and `--skip enjoyment`\n  when testing doc/project.md. The `--skip greeting` and `--skip enjoyment`\n  options cause 2 examples to be skipped. These are the first and\n  last examples in the file.\n- The example tests the two Python interactive sessions in the midlle\n  of the file as the last two items above.\n- The \"**/*.md\" recursively matches other Markdown files.\n  There are no otions.\n- The ini file globs above only apply to .md files.\n  We ignore the Python pytest test file tests/test_example.py\n  by adding the `--ignore` option.\n- Look for more phmdoctest-collect examples on GitHub in\n  tests/test_collect_section.py.\n\n\n### phmdoctest-collect options\n\n#### `-s, --skip TEXT`\nDo not test blocks with substring TEXT. Allowed multiple times.\n\n#### `--fail-nocode`\nMarkdown file with no code blocks left after applying skips\ngenerates a failing test.\n\n#### `-u, --setup TEXT`\nRun block with substring TEXT at test module setup time.\n\n#### `-d, --teardown TEXT`\nRun block with substring TEXT at test module teardown time.\n\n#### `--setup-doctest`\nMake globals created by the `--setup` Python code block\nor setup directive visible to Python interactive session \u003e\u003e\u003e blocks.\n**Caution:** The globals are set at pytest Session scope.\nThe globals are visible to all doctests in the test suite.\nThis includes doctests collected by the plugin\nand doctests collected from other files due to `--doctest-modules`.\n\n### Notes\n\n- Fenced code blocks are searched for the substring TEXT.\n- `--skip TEXT` can apply to more than one block.\n- Exactly one block can match `--setup` and `--teardown`.\n\n### Equivalent directives\n\nThe HTML directive comments below placed in the Markdown file\ncan be used instead of specifying options in the\nphmdoctest-collect section.\nDirectives select a single fenced code block.\nThere are 10 directives in [phmdoctest Directives][5].\nHere are the directives equivalent to the collect section options.\n\n```\n  collect-section          HTML Directive\n  option                   comment\n------------------------------------------------\n-s, --skip TEXT        \u003c!--phmdoctest-skip--\u003e\n-u, --setup TEXT       \u003c!--phmdoctest-setup--\u003e\n-d, --teardown TEXT    \u003c!--phmdoctest-teardown--\u003e\n```\n\n## Hints\n\n- When invoking pytest, cwd must be in the subpath of the files to be collected\n  to avoid this error from pathlib.py in relative_to():\n  ValueError: `\u003cfile to be collected\u003e` is not in the subpath of `\u003cworking directory\u003e`\n- Note the plugin does not accept single quoted phmdoctest args in the\n  phmdoctest-collect section. A failing test will be collected.\n- Use underscore in conftest.py for pytest_plugins:\n  `pytest_plugins = [\"pytest_phmdoctest\"]`\n- An ImportPathMismatchError indicates two test files have the same name.\n- If using `--phmdoctest-generate` add `.gendir` to .gitignore.\n- pytest -vv output shows the path to the plugin temporary directory.\n- The `--report` option of the phmdoctest command lists all\n  fenced code blocks in the Markdown file.\n- phmdoctest can generate test files for multiple Markdown files\n  with one call by specifying a configuration file.\n- phmdoctest offers pytest pytester fixtures (suitable for plugin development)\n  to generate and run tests for a single Markdown file.\n\n## Related projects\n\n- phmdoctest\n- rundoc\n- byexample\n- sphinx.ext.doctest\n- sybil\n- doxec\n- egtest\n- pytest-codeblocks\n\n[1]: https://docs.python.org/3/library/doctest.html\n[3]: https://tmarktaylor.github.io/phmdoctest\n[4]: https://docs.pytest.org/en/stable\n[5]: https://tmarktaylor.github.io/phmdoctest#directives\n[6]: https://tmarktaylor.github.io/phmdoctest#usage\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftmarktaylor%2Fpytest-phmdoctest","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftmarktaylor%2Fpytest-phmdoctest","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftmarktaylor%2Fpytest-phmdoctest/lists"}