Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/lsys/runpynb

Run Jupyter notebooks in command-line and Makefiles.
https://github.com/lsys/runpynb

cli jupyter jupyter-notebook makefile notebook-jupyter python

Last synced: 3 months ago
JSON representation

Run Jupyter notebooks in command-line and Makefiles.

Awesome Lists containing this project

README 

        
 



# Run Jupyter notebooks quietly from command-line

[![PyPI](https://img.shields.io/pypi/v/runpynb?color=brightgreen&label=PyPI)](https://pypi.org/project/runpynb/)

![GitHub release (latest by date)](https://img.shields.io/github/v/release/lsys/runpynb?label=Latest%20release)




[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/runpynb?label=Python%203.6%2B)](https://pypi.org/project/runpynb/)




[![DOI](https://zenodo.org/badge/520408889.svg)](https://zenodo.org/badge/latestdoi/520408889)



`runPyNB` is a quick and dirty utility to run (and time) Jupyter notebooks from command-line and makefiles.



## Quickstart

Install from PyPI

```bash

pip install runpynb

```



General usage: `runpynb  [options]` (".ipynb" not required)



* `runpynb`: Run all notebooks in directory.



    

    $ runpynb


    ![](https://raw.githubusercontent.com/lsys/runpynb/main/assets/_docs/runall.gif)



(back to top)



## Usage



* `runpynb  -q`: Run quietly (`-q`).



    

    $ runpynb hello.ipynb -q


    ![](https://raw.githubusercontent.com/lsys/runpynb/main/assets/_docs/be-quiet.gif)

    

* `runpynb  -qs`: Run quietly (`-q`) as a sequence of workflow (`-s`). Errors (eg in `error.ipynb`) will break the workflow.



    

    $ runpynb error.ipynb hello.ipynb -qs


    ![](https://raw.githubusercontent.com/lsys/runpynb/main/assets/_docs/as-sequence.gif)

    

* `runpynb  -o`: Save output as separate notebook (`-o`), instead of overwriting existing notebook(s).



    

    $ runpynb hello.ipynb -o


    ![](https://raw.githubusercontent.com/lsys/runpynb/main/assets/_docs/output-as-separate-notebook.gif)

    

(back to top)



## Options

```bash

usage: runpynb [-h] [-t TIMEOUT] [-s] [-o] [-v VERSION] [-q] [notebooks ...]



Run (and time) Jupyter notebooks silently in command-line.



positional arguments:

  notebooks             List of Jupyter notebooks (*.ipynb) to be run

                        (default=all notebooks in path).



optional arguments:

  -h, --help            show this help message and exit

  -t TIMEOUT, --timeout TIMEOUT

                        Seconds until a cell in the notebook timesout, which

                        raises a Timeouterror exception (default is 3000=5

                        mins).

  -s, --sequence        Sequence implicit in notebook lists. If error occurs

                        somewhere, stop entire pipeline.

  -o, --output          Save output as a separate notebook with "-out"-suffix

                        (e.g. *-out.ipynb) instead of overwriting existing

                        file.

  -v VERSION, --version VERSION

                        Version of notebook to return (Default=No conversion).

                        Notebook will be converted if necessary.

  -q, --quiet           Be quiet and don't print messages (including run

                        time). Caution: Does not suppress error messages.

```

(back to top)



## Status

[![Documentation Status](https://readthedocs.org/projects/runpynb/badge/?version=latest)](https://runpynb.readthedocs.io/en/latest/?badge=latest)




[![Build Status](https://app.travis-ci.com/LSYS/runPyNB.svg?branch=main)](https://app.travis-ci.com/LSYS/runPyNB)




[![Tests](https://github.com/LSYS/runPyNB/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/LSYS/runPyNB/actions/workflows/tests.yml)

[![codecov](https://codecov.io/gh/LSYS/runPyNB/branch/main/graph/badge.svg?token=ZtC2IJ07Fa)](https://codecov.io/gh/LSYS/runPyNB)




[![CI](https://github.com/LSYS/runPyNB/actions/workflows/build.yml/badge.svg?branch=main)](https://github.com/LSYS/runPyNB/actions/workflows/build.yml)




[![CLI](https://github.com/LSYS/runPyNB/actions/workflows/cli.yml/badge.svg?branch=main)](https://github.com/LSYS/runPyNB/actions/workflows/cli.yml)




[![Doclinks](https://github.com/LSYS/runPyNB/actions/workflows/doclinks.yml/badge.svg?branch=main)](https://github.com/LSYS/runPyNB/actions/workflows/doclinks.yml)

(back to top)






## More on this package



This is a lightweight package that wraps around the official Jupyter [`nbformat`](https://nbformat.readthedocs.io/en/latest/) and [`nbconvert`](https://nbconvert.readthedocs.io/en/latest/) modules.



My workflow involves using [`Jupyter notebooks`](https://jupyter.org/) to clean, and analyze data.

I use this utility to run notebooks silently from the command-line and [`Makefiles`](#usage-with-makefiles) (without converting from `.ipynb` files to `.py` files). 



Related packages are [`guoquan/runnb`](https://github.com/guoquan/runnb) and [`vinayak-mehta/nbcommands`](https://github.com/vinayak-mehta/nbcommands) with a planned enhancement `nbtime` to run Jupyter notebooks from command-line.

(back to top)



## Usage with Makefiles

A minimal workflow where `get-data.ipynb` takes 5000 seconds to prepare `data.csv`.

And where `analyze.ipynb` uses `data.csv` to produce `output.png`.

```makefile

.DEFAULT_GOAL := output.png



data.csv: get-data.ipynb

	runpynb $^ -t 5000

	

output.png: analyze.ipynb data.csv

	runpynb $< 

```

(back to top)



## License

This package is licensed under the [MIT License](https://github.com/LSYS/runPyNB/blob/main/LICENSE).