Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/rhettbull/clirunner

A python test helper for invoking and testing command line interfaces (CLIs) based on Click's CliRunner
https://github.com/rhettbull/clirunner

cli command-line command-line-tool pytest python test testing unittest unittesting

Last synced: 12 months ago
JSON representation

A python test helper for invoking and testing command line interfaces (CLIs) based on Click's CliRunner

Awesome Lists containing this project

README 

          
# CliRunner



A test helper for invoking and testing command line interfaces (CLIs). This is adapted from the [Click](https://click.palletsprojects.com/) [CliRunner](https://click.palletsprojects.com/en/8.1.x/testing/) but modified to work with non-Click scripts, such as those using [argparse](https://docs.python.org/3/library/argparse.html) for parsing command line arguments.



## Installation



`python3 -m pip install clirunner`



## Source Code



The source code is available on [GitHub](https://github.com/RhetTbull/clirunner).



## Motivation



I write a lot of Python command line tools. I usually reach for Click to build the CLI but sometimes will use argparse or even just manual `sys.argv` parsing for simple scripts or where I do not want to introduce a dependency on Click. Click provides a very useful [CliRunner](https://click.palletsprojects.com/en/8.1.x/testing/) for testing CLIs, but it only works with Click applications. This project is a derivative of Click's CliRunner that works with non-Click scripts. The API is the same as Click's CliRunner, so it should be easy to switch between the two if you later refactor to use Click.



## Supported Platforms



Tested on macOS, Ubuntu Linux, and Windows using "*-latest" GitHub Workflow runners with Python 3.9 - 3.12.



## Documentation



Full documentation is available [here](https://rhettbull.github.io/clirunner/).



## Basic Testing



CliRunner can invoke your CLI's main function as a command line script. The CliRunner.invoke() method runs the command line script in isolation and captures the output as both bytes and binary data.



The return value is a Result object, which has the captured output data, exit code, and optional exception attached:



### hello.py



```python

"""Simple CLI """



import argparse



def hello():

    """Print Hello World"""

    argp = argparse.ArgumentParser(description="Print Hello World")

    argp.add_argument("-n", "--name", help="Name to greet")

    args = argp.parse_args()

    print(f"Hello {args.name or 'World'}!")



if __name__ == "__main__":

    hello()

```



### test_hello.py



```python

"""Test hello.py"""



from hello import hello



from clirunner import CliRunner



def test_hello_world():

    runner = CliRunner()

    result = runner.invoke(hello, ["--name", "Peter"])

    assert result.exit_code == 0

    assert result.output == "Hello Peter!\n"

```



Note that `result.output` will contain the combined output of `stdout` and `stderr`. If you want to capture `stdout` and `stderr` separately, use `result.stdout` and `result.stderr`.



## File System Isolation



For basic command line tools with file system operations, the `CliRunner.isolated_filesystem()` method is useful for setting the current working directory to a new, empty folder.



### cat.py



```python

"""Simple cat program for testing isolated file system"""



import argparse



def cat():

    argp = argparse.ArgumentParser()

    argp.add_argument("file", type=argparse.FileType("r"))

    args = argp.parse_args()

    print(args.file.read(), end="")



if __name__ == "__main__":

    cat()

```



### test_cat.py



```python

"""Test cat.py example."""



from cat import cat



from clirunner import CliRunner



def test_cat():

    runner = CliRunner()

    with runner.isolated_filesystem():

        with open("hello.txt", "w") as f:

            f.write("Hello World!\n")



        result = runner.invoke(cat, ["hello.txt"])

        assert result.exit_code == 0

        assert result.output == "Hello World!\n"

```



Pass `temp_dir` to control where the temporary directory is created. The directory will not be removed by `CliRunner` in this case. This is useful to integrate with a framework like Pytest that manages temporary files.



```python

def test_keep_dir(tmp_path):

    runner = CliRunner()



    with runner.isolated_filesystem(temp_dir=tmp_path) as td:

        ...

```



## Input Streams



The test wrapper can also be used to provide input data for the input stream (stdin). This is very useful for testing prompts, for instance:



### prompt.py



```python

"""Simple example for testing input streams"""



def prompt():

    foo = input("Foo: ")

    print(f"foo = {foo}")



if __name__ == "__main__":

    prompt()

```



### test_prompt.py



```python

"""Test prompt.py example"""



from prompt import prompt



from clirunner import CliRunner



def test_prompts():

    runner = CliRunner()

    result = runner.invoke(prompt, input="wau wau\n")

    assert not result.exception

    # note: unlike click.CliRunner, clirunner.CliRunner does not echo the input

    assert "foo = wau wau\n" in result.output

```



Note that the input will not be echoed to the output stream. This is different from the behavior of the `input()` function, which does echo the input and from click's `prompt()` function, which also echo's the input when under test.



## Environment Variable Isolation



The `CliRunner.invoke()` method can also be used to set environment variables for the command line script. This is useful for testing command line tools that use environment variables for configuration.



### hello_env.py



```python

"""Say hello to the world, shouting if desired."""



import os



def hello():

    """Say hello to the world, shouting if desired."""

    if os.getenv("SHOUT") == "1":

        print("HELLO WORLD!")

    else:

        print("Hello World!")



if __name__ == "__main__":

    hello()

```



### test_hello_env.py



```python

"""Test hello2.py showing how to set environment variables for testing."""



from hello_env import hello



from clirunner import CliRunner



def test_hello():

    """Test hello2.py"""

    runner = CliRunner()

    result = runner.invoke(hello)

    assert result.exit_code == 0

    assert result.output == "Hello World!\n"



def test_hello_shouting():

    """Test hello2.py"""

    runner = CliRunner()

    result = runner.invoke(hello, env={"SHOUT": "1"})

    assert result.exit_code == 0

    assert result.output == "HELLO WORLD!\n"

```



## Handling Exceptions



Normally the `CliRunner.invoke()` method will catch exceptions in the CLI under test. If an exception is raised, it will be available via the `Result.exception` property. This can be disabled by passing `catch_exceptions=False` to the `CliRunner.invoke()` method.



### raise_exception.py



```python

"""Simple script that raises an exception"""



def raise_exception():

    """Raises a ValueError exception"""

    raise ValueError("Exception raised")



if __name__ == "__main__":

    raise_exception()

```



### test_raise_exception.py



```python

"""Test raise_exception.py"""



import pytest

from raise_exception import raise_exception



from clirunner import CliRunner



def test_exception_caught():

    """CliRunner normally catches exceptions"""

    runner = CliRunner()

    result = runner.invoke(raise_exception)

    # exit code will not be 0 if exception is raised

    assert result.exit_code != 0

    assert isinstance(result.exception, ValueError)



def test_exception_not_caught():

    """CliRunner can be configured to not catch exceptions"""

    runner = CliRunner()

    with pytest.raises(ValueError):

        runner.invoke(raise_exception, catch_exceptions=False)

```



## Testing Click Applications



Do not use `clirunner.CliRunner` to test applications built with [Click](https://pypi.org/project/click/), [Typer](https://pypi.org/project/typer/), or another Click derivative. Instead, use Click's built-in [CliRunner](https://click.palletsprojects.com/en/8.1.x/testing) or [Typer's equivalent](https://typer.tiangolo.com/tutorial/testing/).



`clirunner.CliRunner` is designed for testing non-Click scripts such as those using [argparse](https://docs.python.org/3/library/argparse.html) or manual [sys.argv](https://docs.python.org/3/library/sys.html#sys.argv) argument parsing. It has also been tested with [pydantic-argparse](https://pydantic-argparse.supimdos.com/), [clipstick](https://github.com/sander76/clipstick), and [tyro](https://github.com/brentyi/tyro).



## License



CliRunner is a derivative work of Click's CliRunner, and so it is licensed under the same BSD 3-clause license as Click. See the [LICENSE](https://github.com/RhetTbull/clirunner/blob/main/LICENSE) and [LICENSE.Click](https://github.com/RhetTbull/clirunner/blob/main/LICENSE.Click) files for details.