https://github.com/tyleryep/pytest-idempotent
Pytest plugin for testing the idempotency of a function.
https://github.com/tyleryep/pytest-idempotent
Last synced: over 1 year ago
JSON representation
Pytest plugin for testing the idempotency of a function.
- Host: GitHub
- URL: https://github.com/tyleryep/pytest-idempotent
- Owner: TylerYep
- License: mit
- Created: 2021-11-26T01:45:22.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2025-03-17T18:45:53.000Z (over 1 year ago)
- Last Synced: 2025-03-19T09:40:58.093Z (over 1 year ago)
- Language: Python
- Size: 139 KB
- Stars: 3
- Watchers: 1
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# pytest-idempotent
[](https://www.python.org/downloads/release/python-380/)
[](https://badge.fury.io/py/pytest-idempotent)
[](https://github.com/TylerYep/pytest-idempotent/actions/workflows/test.yml)
[](https://github.com/TylerYep/pytest-idempotent/blob/main/LICENSE)
[](https://codecov.io/gh/TylerYep/pytest-idempotent)
[](https://pepy.tech/project/pytest-idempotent)
Pytest plugin for testing the idempotency of a function.
## Usage
```
pip install pytest-idempotent
```
## Documentation
Suppose we had the following function, that we (incorrectly) assumed was idempotent. How would we write a test for this?
First, we can label the function with a decorator:
```python
# abc.py
from pytest_idempotent import idempotent # or use your own decorator! See below.
@idempotent
def func(x: list[int]) -> None:
x += [9]
```
Note: this function is _not_ idempotent because calling it on the same list `x` grows the size of `x` by 1 each time. To be idempotent, we should be able to run `func` more than once without any adverse effects.
We can write an idempotency test for this function as follows:
```python
# tests/abc_test.py
import pytest
@pytest.mark.idempotent
def test_func() -> None:
x: list[int] = []
func(x)
assert x == [9]
```
Adding the `@pytest.mark.idempotent` mark automatically splits this test into two - one that tests the regular behavior, and one that tests that the function can be called twice without adverse effects.
```
❯❯❯ pytest
================= test session starts ==================
platform darwin -- Python 3.9.2, pytest-6.2.5
collected 2 items
tests/abc_test.py .F [100%]
===================== FAILURES ========================
------------- test_func[idempotency-check] -------------
@pytest.mark.idempotent
def test_func() -> None:
x: list[int] = []
func(x)
> assert x == [9]
E assert [9, 9] == [9]
E Left contains one more item: 9
E Use -v to get the full diff
tests/abc_test.py:19: AssertionError
=============== short test summary info ================
FAILED tests/abc_test.py::test_func[idempotency-check]
- assert [9, 9] == [9]
============= 1 failed, 1 passed in 0.16s ==============
```
## How It Works
Idempotency is a difficult pattern to enforce. To solve this issue, **pytest-idempotent** takes the following approach:
- Introduce a decorator, `@idempotent`, to functions.
- This decorator serves as a visual aid. If this decorator is commonly used in the codebase, it is much easier to consider idempotency for new and existing functions.
- At runtime, this decorator is a no-op.
- At test-time, if the feature is enabled, we will run the decorated function twice with the same parameters in all test cases.
- We can also assert that the second run returns the same result using an additional parameter to the function's decorator: `@idempotent(equal_return=True)`.
- For all tests marked using `@pytest.mark.idempotent`, we run each test twice: once normally, and once with the decorated function called twice.
- Both runs need to pass all assertions.
- We return the first result because the first run will complete the processing. The second will either return exact the same result or be a no-op.
- To disable idempotency testing for a test or group of tests, add the Pytest marker:
`@pytest.mark.idempotent(enabled=False)`
## Enforcing Tests Use `@pytest.mark.idempotent`
By default, any test that calls an `@idempotent` function must also be decorated with the marker `@pytest.mark.idempotent`.
To disable idempotency testing for a test or group of tests, use:
`@pytest.mark.idempotent(enabled=False)`, or add the following config to your project:
```python
def pytest_idempotent_enforce_tests() -> bool:
return False
```
To disable enforced idempotency testing for a specific function, you can also pass the flag into the decorator:
```python
# abc.py
from pytest_idempotent import idempotent
@idempotent(enforce_tests=False)
def func() -> None:
return
```
Or, you can automatically add the marker based on the test name by adding to `conftest.py`:
```python
# conftest.py
def pytest_collection_modifyitems(items):
for item in items:
if "idempotent" in item.nodeid:
item.add_marker(pytest.mark.idempotent)
```
## @idempotent decorator
By default, the `@idempotent` decorator does nothing during runtime. We do not want to add overhead to production code to run tests.
```python
from typing import Any, Callable, TypeVar
_F = TypeVar("_F", bound=Callable[..., Any])
def idempotent(func: _F) -> _F:
"""
No-op during runtime.
This marker allows pytest-idempotent to override the decorated function
during test-time to verify the function is idempotent.
"""
return func
```
To use your own `@idempotent` decorator, you can override the `pytest_idempotent_decorator` function in your `conftest.py` to return the module path to your implementation.
```python
# conftest.py
# Optional: you can define this to ensure the plugin is correctly installed
pytest_plugins = ["pytest_idempotent"]
def pytest_idempotent_decorator() -> str:
# This links to my custom implementation of @idempotent.
return "src.utils.idempotent"
```