{"id":19442346,"url":"https://github.com/treebeardtech/nbmake","last_synced_at":"2025-05-15T09:04:49.660Z","repository":{"id":39583536,"uuid":"313931299","full_name":"treebeardtech/nbmake","owner":"treebeardtech","description":"📝 Pytest plugin for testing notebooks","archived":false,"fork":false,"pushed_at":"2025-03-18T18:27:37.000Z","size":676,"stargazers_count":196,"open_issues_count":15,"forks_count":18,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-14T22:16:29.954Z","etag":null,"topics":["jupyter","jupyter-book","jupyter-notebook","nbsphinx","notebook","papermill","pytest-plugin","test"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/nbmake/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/treebeardtech.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.MD","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-11-18T12:43:11.000Z","updated_at":"2025-04-02T15:03:30.000Z","dependencies_parsed_at":"2023-01-27T18:00:39.020Z","dependency_job_id":"505d818f-75b2-456d-b012-fd8e9dde3aa2","html_url":"https://github.com/treebeardtech/nbmake","commit_stats":{"total_commits":119,"total_committers":7,"mean_commits":17.0,"dds":0.2773109243697479,"last_synced_commit":"a3730bc55c9f342e4e4ec64ca8bfd833d23b037f"},"previous_names":[],"tags_count":35,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/treebeardtech%2Fnbmake","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/treebeardtech%2Fnbmake/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/treebeardtech%2Fnbmake/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/treebeardtech%2Fnbmake/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/treebeardtech","download_url":"https://codeload.github.com/treebeardtech/nbmake/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248968918,"owners_count":21191162,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["jupyter","jupyter-book","jupyter-notebook","nbsphinx","notebook","papermill","pytest-plugin","test"],"created_at":"2024-11-10T15:39:02.445Z","updated_at":"2025-04-14T22:16:44.310Z","avatar_url":"https://github.com/treebeardtech.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# nbmake\n[![codecov](https://img.shields.io/badge/chat-discord-blue?logo=discord\u0026logoColor=white)](https://discord.gg/QFjCpMjqRY)\n[![codecov](https://codecov.io/gh/treebeardtech/nbmake/branch/main/graph/badge.svg?token=9GuDM35FuO)](https://codecov.io/gh/treebeardtech/nbmake)\n[![PyPI versions](https://img.shields.io/pypi/pyversions/nbmake?logo=python\u0026logoColor=white)](https://pypi.org/project/nbmake)\n[![PyPI versions](https://img.shields.io/pypi/v/nbmake?logo=python\u0026logoColor=white)](https://pypi.org/project/nbmake)\n[![PyPI Downloads](https://img.shields.io/pypi/dm/nbmake)](https://pypi.org/project/nbmake)\n\n**What?** Pytest plugin for testing and releasing notebook documentation\n\n**Why?** To raise the quality of scientific material through better automation\n\n**Who is this for?** Research/Machine Learning Software Engineers who maintain packages/teaching materials with documentation written in notebooks.\n\n## Functionality\n\n1. Executes notebooks using pytest and nbclient, allowing parallel notebook testing\n2. Optionally writes back to the repo, allowing faster building of [nbsphinx](https://github.com/spatialaudio/nbsphinx) or [jupyter book](https://github.com/executablebooks/jupyter-book) docs\n\n## Quick Start\n\nIf you have a notebook that runs interactively using an ipython kernel,\nyou can try testing it automatically as follows:\n\n```sh\npip install pytest nbmake\npytest --nbmake **/*ipynb\n```\n\n## Configure Cell Timeouts\n\nYou can configure the cell timeout with the following pytest flag:\n\n```sh\npytest --nbmake --nbmake-timeout=3000 # allows each cell 3000 seconds to finish\n```\n\n## Allow Errors For a Whole Notebook\n\nThis configuration must be placed in the notebook's **top-level metadata** (not cell-level metadata).\n\nYour notebook should look like this:\n\n```json\n{\n  \"cells\": [ ... ],\n  \"metadata\": {\n    \"kernelspec\": { ... },\n    \"execution\": {\n      \"allow_errors\": true,\n      \"timeout\": 300\n    }\n  }\n}\n```\n\n## Allow a Cell to Throw an Exception\n\nA cell with the following metadata can throw an exception without failing the test:\n\n```json\n  \"metadata\": {\n    \"tags\": [\n      \"raises-exception\"\n    ]\n  }\n```\n\n## Ignore a Code Cell\n\nA cell with the following metadata will not be executed by nbmake\n\n```json\n{\n  \"language\": \"python\",\n  \"custom\": {\n    \"metadata\": {\n      \"tags\": [\n        \"skip-execution\"\n      ]\n    }\n  }\n}\n```\n\n## Override Notebook Kernels when Testing\n\nRegardless of the kernel configured in the notebook JSON, you can force nbmake to use a specific kernel when testing:\n\n```\npytest --nbmake --nbmake-kernel=mycustomkernel\n```\n\n## Add Missing Jupyter Kernel to Your CI Environment\n\nIf you are not using the flag above and are using a kernel name other than the default ‘python3’, you will see an error message when executing your notebooks in a fresh CI environment: `Error - No such kernel: 'mycustomkernel'`\n\nUse ipykernel to install the custom kernel:\n\n```sh\npython -m ipykernel install --user --name mycustomkernel\n```\n\nIf you are using another language such as c++ in your notebooks, you may have a different process for installing your kernel.\n\n## Parallelisation\n\nFor repos containing a large number of notebooks that run slowly, you can run each notebook\nin parallel using `pytest-xdist`.\n\n```sh\npip install pytest-xdist\n\npytest --nbmake -n=auto\n```\n\nIt is also possible to parallelise at a CI-level using strategies, see [example](https://github.com/LabForComputationalVision/plenoptic/blob/master/.github/workflows/treebeard.yml)\n\n### Build Jupyter Books Faster\n\nUsing xdist and the `--overwrite` flag let you build a large jupyter book repo faster:\n\n```sh\npytest --nbmake --overwrite -n=auto examples\njb build examples\n```\n\n## Find missing imports in a directory of failing notebooks\n\nIt's not always feasible to get notebooks running from top to bottom from the start.\n\nYou can however, use nbmake to check that there are no `ModuleNotFoundError`s:\n\n```sh\npytest \\\n  --nbmake \\\n  --nbmake-find-import-errors \\ # Ignore all errors except ModuleNotFoundError\n  --nbmake-timeout=20 # Skip past cells longer than 20s\n```\n\n## Mock out variables to simplify testing\n\nIf your notebook runs a training process that takes a long time to run, you can use nbmake's\nmocking feature to overwrite variables after a cell runs:\n\n```json\n{\n  \"cells\": [\n    ...,\n    {\n      \"cell_type\": \"code\",\n      \"execution_count\": null,\n      \"metadata\": {\n        \"nbmake\": {\n          \"mock\": {\n            // these keys will override global variables after this cell runs\n            \"epochs\": 2,\n            \"config\": \"/test/config.json\",\n            \"args\": {\n              \"env\": \"test\"\n            }\n          }\n        }\n      },\n      \"outputs\": [],\n      \"source\": [\n        \"epochs = 10\\n\",\n        \"...\"\n      ]\n    },\n    ...\n  ],\n  ...\n}\n```\n\n## Run test logic after a cell executes\n\nYou can fetch CI secrets and run assertions after any cell by putting scripts in the cell metadata under `nbmake.post_cell_execute`:\n\n```json\n{\n \"cells\": [\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": null,\n   \"metadata\": {\n    \"nbmake\": {\n     \"post_cell_execute\": [\n       \"y = 3\",\n       \"z = x+y\"\n     ]\n    }\n   },\n   \"outputs\": [],\n   \"source\": [\n    \"x = 1\\n\",\n    \"y = 2\\n\",\n    \"z = 0\\n\",\n    \"# this cell has a post_cell_execute that assigns y and z\"\n   ]\n  },\n```\n\n## Advice on Usage\n\nnbmake is best used in a scenario where you use the ipynb files only for development. Consumption of notebooks is primarily done via a docs site, built through jupyter book, nbsphinx, or some other means. If using one of these tools, you are able to write assertion code in cells which will be [hidden from readers](https://jupyterbook.org/interactive/hiding.html).\n\n### Pre-commit\n\nTreating notebooks like source files lets you keep your repo minimal. Some tools, such as plotly may drop several megabytes of javascript in your output cells, as a result, stripping out notebooks on pre-commit is advisable:\n\n```\n# .pre-commit-config.yaml\nrepos:\n  - repo: https://github.com/kynan/nbstripout\n    rev: master\n    hooks:\n      - id: nbstripout\n```\n\nSee https://pre-commit.com/ for more...\n\n## Disable Nbmake\n\nImplicitly:\n```\npytest\n```\n\nExplicitly:\n```\npytest -p no:nbmake\n```\n\n## See Also:\n\n* A more in-depth [intro to nbmake](https://semaphoreci.com/blog/test-jupyter-notebooks-with-pytest-and-nbmake) running on Semaphore CI\n* [nbmake action](https://github.com/treebeardtech/treebeard)\n* [pytest](https://pytest.org/)\n* [jupyter book](https://github.com/executablebooks/jupyter-book)\n* [jupyter cache](https://github.com/executablebooks/jupyter-cache)\n* [MyST-NB](https://github.com/executablebooks/MyST-NB)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftreebeardtech%2Fnbmake","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftreebeardtech%2Fnbmake","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftreebeardtech%2Fnbmake/lists"}