{"id":13772220,"url":"https://github.com/mristin/icontract-hypothesis","last_synced_at":"2025-04-10T17:11:06.942Z","repository":{"id":38379530,"uuid":"321988111","full_name":"mristin/icontract-hypothesis","owner":"mristin","description":"Combine contracts and automatic testing.","archived":false,"fork":false,"pushed_at":"2022-06-05T20:45:22.000Z","size":185,"stargazers_count":79,"open_issues_count":2,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-24T14:51:23.229Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mristin.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-12-16T13:17:58.000Z","updated_at":"2025-02-23T20:14:09.000Z","dependencies_parsed_at":"2022-08-25T05:51:18.633Z","dependency_job_id":null,"html_url":"https://github.com/mristin/icontract-hypothesis","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mristin%2Ficontract-hypothesis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mristin%2Ficontract-hypothesis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mristin%2Ficontract-hypothesis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mristin%2Ficontract-hypothesis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mristin","download_url":"https://codeload.github.com/mristin/icontract-hypothesis/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248261406,"owners_count":21074222,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T17:01:01.450Z","updated_at":"2025-04-10T17:11:06.911Z","avatar_url":"https://github.com/mristin.png","language":"Python","funding_links":[],"categories":["Property Based Testing"],"sub_categories":[],"readme":"icontract-hypothesis\n====================\n\n.. image:: https://github.com/mristin/icontract-hypothesis/workflows/CI/badge.svg\n    :target: https://github.com/mristin/icontract-hypothesis/actions?query=workflow%3ACI\n    :alt: Continuous integration\n\n.. image:: https://coveralls.io/repos/github/mristin/icontract-hypothesis/badge.svg?branch=main\n    :target: https://coveralls.io/github/mristin/icontract-hypothesis?branch=main\n    :alt: Test coverage\n\n.. image:: https://badge.fury.io/py/icontract-hypothesis.svg\n    :target: https://badge.fury.io/py/icontract-hypothesis\n    :alt: PyPI - version\n\n.. image:: https://img.shields.io/pypi/pyversions/icontract-hypothesis.svg\n    :alt: PyPI - Python Version\n\nIcontract-hypothesis combines design-by-contract with automatic testing.\n\nIt is an integration between\n`icontract \u003chttps://github.com/Parquery/icontract\u003e`_\nlibrary for design-by-contract and\n`Hypothesis \u003chttps://github.com/HypothesisWorks/hypothesis\u003e`_ library for\nproperty-based testing.\n\nThe result is a powerful combination that allows you to automatically test\nyour code. Instead of writing manually the Hypothesis search strategies for\na function, icontract-hypothesis infers them based on\nthe function's precondition. This makes automatic testing as effortless as it\ngoes.\n\nYou can use icontract-hypothesis:\n\n* As a library, to write succinct unit tests,\n* As a command-line tool or as a tool integrated in your IDE\n  (*e.g.*, `icontract-hypothesis-vim \u003chttps://github.com/mristin/icontract-hypothesis-vim\u003e`__,\n  `icontract-hypothesis-pycharm \u003chttps://github.com/mristin/icontract-hypothesis-pycharm\u003e`__ and\n  `icontract-hypothesis-vscode \u003chttps://github.com/mristin/icontract-hypothesis-vscode\u003e`__).\n\n  This allows you to automatically test functions during the development and\n  use it in your continuous integration,\n* As a ghostwriter utility giving you a starting point for your more elaborate\n  Hypothesis strategies.\n\nSince the contracts live close to the code, evolving the code also automatically\nevolves the tests.\n\nUsage\n-----\nLibrary\n~~~~~~~\nThere are two ways to integrate icontract-hypothesis in your tests as a library.\n\n**Only assume.** First, you can use it for defining the assumptions of the test based on the\nprecondition:\n\n.. code-block:: python\n\n    \u003e\u003e\u003e from hypothesis import given\n    \u003e\u003e\u003e import hypothesis.strategies as st\n\n    \u003e\u003e\u003e import icontract\n    \u003e\u003e\u003e import icontract_hypothesis\n\n    \u003e\u003e\u003e @icontract.require(lambda x: x \u003e 0)\n    ... @icontract.ensure(lambda result: result \u003e 0)\n    ... def some_func(x: int) -\u003e int:\n    ...     return x - 1000\n\n    \u003e\u003e\u003e assume_preconditions = icontract_hypothesis.make_assume_preconditions(\n    ...     some_func)\n\n    \u003e\u003e\u003e @given(x=st.integers())\n    ... def test_some_func(x: int) -\u003e None:\n    ...    assume_preconditions(x)\n    ...    some_func(x)\n\n    \u003e\u003e\u003e test_some_func()\n    Traceback (most recent call last):\n        ...\n    icontract.errors.ViolationError: File \u003cdoctest README.rst[4]\u003e, line 2 in \u003cmodule\u003e:\n    result \u003e 0:\n    result was -999\n    x was 1\n\nThe function ``assume_preconditions`` created by\n``icontract_hypothesis.make_assume_preconditions`` will reject all the input\nvalues which do not satisfy the pre-conditions of ``some_func``.\n\n**Infer strategy**. Second, you can automatically infer the strategy and test the function:\n\n.. code-block:: python\n\n    \u003e\u003e\u003e import icontract\n    \u003e\u003e\u003e import icontract_hypothesis\n\n    \u003e\u003e\u003e @icontract.require(lambda x: x \u003e 0)\n    ... @icontract.ensure(lambda result: result \u003e 0)\n    ... def some_func(x: int) -\u003e int:\n    ...     return x - 1000\n\n    \u003e\u003e\u003e icontract_hypothesis.test_with_inferred_strategy(some_func)\n    Traceback (most recent call last):\n        ...\n    icontract.errors.ViolationError: File \u003cdoctest README.rst[10]\u003e, line 2 in \u003cmodule\u003e:\n    result \u003e 0:\n    result was -999\n    x was 1\n\nWhich approach to use depends on how you want to write your tests.\nThe first approach, using ``assume_preconditions``, is practical if you already\ndefined your search strategy and you only want to exclude a few edge cases.\nThe second approach, automatically inferring test strategies, is useful if you\njust want to test your function without specifying any particular search strategy.\n\nIcontract-hypothesis guarantees that the inferred strategy must satisfy the preconditions.\nIf it does not, you should consider it a bug in which case\nplease `create an issue \u003chttps://github.com/mristin/icontract-hypothesis/issues/new\u003e`_\nso that we can fix it.\n\nIf you want to inspect the strategy or further refine it programmatically, use\n``icontract_hypothesis.infer_strategy``:\n\n.. code-block:: python\n\n    \u003e\u003e\u003e import math\n\n    \u003e\u003e\u003e import icontract\n    \u003e\u003e\u003e import icontract_hypothesis\n\n    \u003e\u003e\u003e @icontract.require(lambda x: x \u003e 0)\n    ... @icontract.require(lambda x: x \u003e math.sqrt(x))\n    ... def some_func(x: float) -\u003e int:\n    ...     pass\n\n    \u003e\u003e\u003e icontract_hypothesis.infer_strategy(some_func)\n    fixed_dictionaries({'x': floats(min_value=0, exclude_min=True).filter(lambda x: x \u003e math.sqrt(x))})\n\nTesting Tool\n~~~~~~~~~~~~\nWe provide ``pyicontract-hypothesis test`` command-line tool which you can use\nto automatically test a module.\n\n.. Help starts: pyicontract-hypothesis test --help\n.. code-block::\n\n    usage: pyicontract-hypothesis test [-h] -p PATH\n                                       [--settings [SETTINGS [SETTINGS ...]]]\n                                       [--inspect] [-i [INCLUDE [INCLUDE ...]]]\n                                       [-e [EXCLUDE [EXCLUDE ...]]]\n\n    optional arguments:\n      -h, --help            show this help message and exit\n      -p PATH, --path PATH  Path to the Python file to test\n      --settings [SETTINGS [SETTINGS ...]]\n                            Specify settings for Hypothesis\n\n                            The settings are assigned by '='.\n                            The value of the setting needs to be encoded as JSON.\n\n                            Example: max_examples=500\n      --inspect             Only show the strategy and the settings\n\n                            No tests are executed.\n      -i [INCLUDE [INCLUDE ...]], --include [INCLUDE [INCLUDE ...]]\n                            Regular expressions, lines or line ranges of the functions to process\n\n                            If a line or line range overlaps the body of a function,\n                            the function is considered included.\n\n                            Example 1: ^do_something.*$\n                            Example 2: 3\n                            Example 3: 34-65\n      -e [EXCLUDE [EXCLUDE ...]], --exclude [EXCLUDE [EXCLUDE ...]]\n                            Regular expressions, lines or line ranges of the functions to exclude\n\n                            If a line or line range overlaps the body of a function,\n                            the function is considered excluded.\n\n                            Example 1: ^do_something.*$\n                            Example 2: 3\n                            Example 3: 34-65\n\n.. Help ends: pyicontract-hypothesis test --help\n\nNote that ``pyicontract-hypothesis test`` can be trivially integrated with\nyour IDE if you can pass in the current cursor position and the\ncurrent file name.\n\nGhostwriting Tool\n~~~~~~~~~~~~~~~~~\nWriting property-based tests by hand is tedious and can be partially automated.\nTo that end, we implemented a ghostwriter utility ``pyicontract-hypothesis ghostwrite``\nthat generates a first draft based on pre-conditions that you manually refine further:\n\n.. Help starts: pyicontract-hypothesis ghostwrite --help\n.. code-block::\n\n    usage: pyicontract-hypothesis ghostwrite [-h] (-m MODULE | -p PATH)\n                                             [-o OUTPUT] [--explicit] [--bare]\n                                             [-i [INCLUDE [INCLUDE ...]]]\n                                             [-e [EXCLUDE [EXCLUDE ...]]]\n\n    optional arguments:\n      -h, --help            show this help message and exit\n      -m MODULE, --module MODULE\n                            Module to ghostwrite the unit tests for\n      -p PATH, --path PATH  Path to the module to ghostwrite the unit tests for.\n\n                            If the file represents a module reachable through\n                            sys.path, use the qualified module name in\n                            the unit test.\n\n                            Otherwise, the module is represented as the stem\n                            of the path with all non-identifier characters\n                            replaced with an underscore (\"_\").\n      -o OUTPUT, --output OUTPUT\n                            Path to the file where the output should be written.\n\n                            If '-', writes to STDOUT.\n      --explicit            Write the inferred strategies explicitly\n\n                            This is practical if you want to tune and\n                            refine the strategies and just want to use\n                            ghostwriting as a starting point.\n\n                            Mind that pyicontract-hypothesis does not\n                            automatically fix imports as this is\n                            usually project-specific. You have to fix imports\n                            manually after the ghostwriting.\n      --bare                Print only the body of the tests and omit header/footer\n                            (such as TestCase class or import statements).\n\n                            This is useful when you only want to inspect a single test or\n                            include a single test function in a custom test suite.\n      -i [INCLUDE [INCLUDE ...]], --include [INCLUDE [INCLUDE ...]]\n                            Regular expressions, lines or line ranges of the functions to process\n\n                            If a line or line range overlaps the body of a function,\n                            the function is considered included.\n\n                            Example 1: ^do_something.*$\n                            Example 2: 3\n                            Example 3: 34-65\n      -e [EXCLUDE [EXCLUDE ...]], --exclude [EXCLUDE [EXCLUDE ...]]\n                            Regular expressions, lines or line ranges of the functions to exclude\n\n                            If a line or line range overlaps the body of a function,\n                            the function is considered excluded.\n\n                            Example 1: ^do_something.*$\n                            Example 2: 3\n                            Example 3: 34-65\n\n.. Help ends: pyicontract-hypothesis ghostwrite --help\n\nThe examples of ghostwritten tests are available at:\n`tests/pyicontract_hypothesis/samples \u003chttps://github.com/mristin/icontract-hypothesis/blob/main/tests/pyicontract_hypothesis/samples\u003e`_\n\nRunning Tools as Module\n~~~~~~~~~~~~~~~~~~~~~~~\n\nIf for some reason you want to run the tools as a module, just invoke:\n\n.. Help starts: python -m icontract_hypothesis --help\n.. code-block::\n\n    usage: icontract_hypothesis [-h] {test,ghostwrite} ...\n\n    Combine property-based testing with contracts of a Python module.\n\n    positional arguments:\n      {test,ghostwrite}  Commands\n        test             Test the functions automatically by inferring search\n                         strategies from preconditions\n        ghostwrite       Ghostwrite the unit tests with inferred search strategies\n\n    optional arguments:\n      -h, --help         show this help message and exit\n\n.. Help ends: python -m icontract_hypothesis --help\n\nInstallation\n------------\nicontract-hypothesis is available on PyPI at\nhttps://pypi.org/project/icontract-hypothesis, so you can use ``pip``:\n\n.. code-block::\n\n    pip3 install icontract-hypothesis\n\n\nSearch Strategies\n-----------------\nA naive approach to fuzzy testing is to randomly sample input data, filter it\nbased on pre-conditions and ensure post-conditions after the run. However,\nif your acceptable band of input values is narrow, the rejection sampling\nwill become impractically slow.\n\nFor example, assume a pre-condition ``5 \u003c x \u003c 10``.\nSampling from all possible integers for ``x`` will rarely hit\nthe pre-condition (if ever) thus wasting valuable computational time.\nThe problem is exacerbated as the number of arguments grow due to\n`the curse of dimensionality \u003chttps://en.wikipedia.org/wiki/Curse_of_dimensionality\u003e`_.\n\nIcontract-hypothesis tries to address the search strategies\na bit more intelligently:\n\n* The pre-conditions are matched against common code patterns to define\n  the strategies. For example, ``5 \u003c x \u003c 10`` gives a search strategy\n  ``hypothesis.strategies.integers(min=6, max=9)``.\n\n  We currently match bounds on all available Hypothesis types\n  (``int``, ``float``, ``datetime.date`` *etc*.).\n  We also match regular expressions on ``str`` arguments.\n\n* Pre-conditions which could not be matched, but operate on a single argument\n  are inferred based on the type hint and composed with Hypothesis\n  ``FilteredStrategy``.\n\n* The remainder of the pre-conditions are enforced by filtering on the whole\n  fixed dictionary which is finally passed into the function as keyword arguments.\n\nThere is an ongoing effort to move the strategy matching code into Hypothesis and\ndevelop it further to include many more cases. See\n`this Hypothesis issue \u003chttps://github.com/HypothesisWorks/hypothesis/issues/2701\u003e`_.\n\nNote that static analysis of the source code may not determine all the defined names in various\nscopes as they can also be injected dynamically (*e.g.*, setting ``__globals__`` attribute or\n``globals()[random.choice(\"abc\")] = 1``).\nAs long as you keep fancy dynamic acrobatics out of your contracts,\nthe strategy inference by icontract-hypothesis should work fine.\n\nClasses\n~~~~~~~\nHypothesis automatically builds composite input arguments (classes, dataclasses,\nnamed tuples *etc*.). If your class enforces pre-conditions in the constructor\nmethod (``__init__``), make sure that it inherits from ``icontract.DBC``.\n\nThat way icontract-hypothesis will use\n`hypothesis.strategies.register_type_strategy \u003chttps://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.register_type_strategy\u003e`__\nto register your class with Hypothesis and consider pre-conditions when building\nits instances.\n\nIt is important that you should *not* use\n`hypothesis.strategies.builds \u003chttps://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.builds\u003e`_\nwith the classes using contracts in their constructors as\n`builds \u003chttps://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.builds\u003e`_\nwill disregard the registered strategy. You should use\n`hypothesis.strategies.from_type \u003chttps://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.from_type\u003e`_\ninstead. See\n`this comment on an Hypothesis issue \u003chttps://github.com/HypothesisWorks/hypothesis/issues/2708#issuecomment-749393747\u003e`_\nand\n`the corresponding answer \u003chttps://github.com/HypothesisWorks/hypothesis/issues/2708#issuecomment-749397758\u003e`_.\n\nMany times default inferred strategies for the constructors should be enough, though you\nare of course not restricted to them. You can register your own strategies with\n`hypothesis.strategies.register_type_strategy \u003chttps://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.register_type_strategy\u003e`__\n. Icontract-hypothesis will respect the previous registrations and will not overwrite them.\n\nIDE Plug-ins\n------------\n* `icontract-hypothesis-vim \u003chttps://github.com/mristin/icontract-hypothesis-vim\u003e`__ for\n  `VIM \u003chttps://www.vim.org/\u003e`_\n* `icontract-hypothesis-pycharm \u003chttps://github.com/mristin/icontract-hypothesis-pycharm\u003e`__ for\n  `PyCharm \u003chttps://www.jetbrains.com/pycharm/\u003e`_\n* `icontract-hypothesis-vscode \u003chttps://github.com/mristin/icontract-hypothesis-vscode\u003e`__ for\n  `Visual Studio Code \u003chttps://code.visualstudio.com/\u003e`_\n\nRelated Libraries\n-----------------\nPython design-by-contract libraries\n`deal \u003chttps://github.com/life4/deal\u003e`_ and\n`dpcontracts \u003chttps://github.com/deadpixi/contracts\u003e`_\nintegrate directly with Hypothesis (see\n`this page \u003chttps://deal.readthedocs.io/basic/tests.html\u003e`_ and\n`that page \u003chttps://hypothesis.readthedocs.io/en/latest/extras.html#hypothesis-dpcontracts\u003e`_,\nrespectively).\n\nAs of 2020-12-16:\n\n**Behavioral subtyping.** Neither of the two libraries handles behavioral sub-typing correctly\n(*i.e.*, they do not weaken and strengthen the pre-conditions, and\npost-conditions and invariants, respectively).\nHence they can not be used with class hierarchies as the contracts are not\nproperly inherited.\n\nThis is not strictly related to property-based testing,\nbut presents an inherent flaw in how they implement contracts.\nHence even if you manually supply a search strategy that\nfulfills behavioral subtyping, these two libraries would\nreport (or ignore) an error.\n\nConsider this example with `deal \u003chttps://github.com/life4/deal\u003e`__:\n\n.. code-block:: Python\n\n    class A:\n        @deal.post(lambda result: result % 2 == 0)\n        def some_func(self) -\u003e int:\n            return 2\n\n    class B(A):\n        @deal.post(lambda result: result % 3 == 0)\n        def some_func(self) -\u003e int:\n            # The result 9 satisfies the postcondition of B.some_func,\n            # but not the postcondition of A.some_func thus\n            # breaking the behavioral subtyping.\n            return 9\n\n    b = B()\n    # The correct behavior would be to throw an exception here.\n    b.some_func()\n\n\n**Rejection sampling.** The two libraries only provide rejection sampling which is insufficient\nfor many practical use cases. For example, the computational time grows exponentially with the\nnumber of arguments (see Section \"Search Strategies\").\n\n**Propagation of contracts.** Finally, the existing libraries do not automatically propagate\npre-conditions of constructors to Hypothesis so automatic testing with composite inputs\n(such as instances of classes) is currently not possible with these two libraries. The user can,\nof course, manually design search strategies that satisfy the contracts.\nIn contrast, icontract-hypothesis does that hard-lifting for you automatically.\n\n\nBenchmarks\n~~~~~~~~~~\nWe run benchmarks against `deal` and `dpcontracts` libraries as part of our continuous integration.\n\nWe benchmark against functions using 1, 2 and 3 arguments, respectively, with the precondition that\nthe argument should be positive (*e.g.*, ``a \u003e 0``). We sampled 100 inputs per each run.\n\n.. Benchmark report starts.\n\n\nThe following scripts were run:\n\n* `benchmarks/compare_with_others.py \u003chttps://github.com/Parquery/icontract/tree/master/benchmarks/compare_with_others.py\u003e`_\n\nThe benchmarks were executed on Intel(R) Xeon(R) E-2276M  CPU @ 2.80GHz.\nWe used Python 3.8.5, icontract 2.4.1, deal 4.4.0 and dpcontracts 0.6.0.\n\nThe following tables summarize the results.\n\nBenchmarking Hypothesis testing:\n\n\nArgument count: 1\n\n==========================================  ============  ==============  =======================\nCase                                          Total time    Time per run    Relative time per run\n==========================================  ============  ==============  =======================\n`benchmark_icontract_inferred_strategy`           0.48 s        48.29 ms                     100%\n`benchmark_icontract_assume_preconditions`        0.79 s        78.75 ms                     163%\n`benchmark_dpcontracts`                           1.06 s       106.17 ms                     220%\n`benchmark_deal`                                  0.83 s        82.63 ms                     171%\n==========================================  ============  ==============  =======================\n\nArgument count: 2\n\n==========================================  ============  ==============  =======================\nCase                                          Total time    Time per run    Relative time per run\n==========================================  ============  ==============  =======================\n`benchmark_icontract_inferred_strategy`           0.63 s        63.45 ms                     100%\n`benchmark_icontract_assume_preconditions`        1.65 s       165.05 ms                     260%\n`benchmark_dpcontracts`                           2.10 s       209.51 ms                     330%\n`benchmark_deal`                                  1.61 s       161.09 ms                     254%\n==========================================  ============  ==============  =======================\n\nArgument count: 3\n\n==========================================  ============  ==============  =======================\nCase                                          Total time    Time per run    Relative time per run\n==========================================  ============  ==============  =======================\n`benchmark_icontract_inferred_strategy`           0.72 s        71.66 ms                     100%\n`benchmark_icontract_assume_preconditions`        3.30 s       330.20 ms                     461%\n`benchmark_dpcontracts`                           4.23 s       423.31 ms                     591%\n`benchmark_deal`                                  3.20 s       319.57 ms                     446%\n==========================================  ============  ==============  =======================\n\n\n\n.. Benchmark report ends.\n\nVersioning\n==========\nWe follow `Semantic Versioning \u003chttp://semver.org/spec/v1.0.0.html\u003e`_.\nThe version X.Y.Z indicates:\n\n* X is the major version (backward-incompatible),\n* Y is the minor version (backward-compatible), and\n* Z is the patch version (backward-compatible bug fix).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmristin%2Ficontract-hypothesis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmristin%2Ficontract-hypothesis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmristin%2Ficontract-hypothesis/lists"}