{"id":32599689,"url":"https://github.com/testflows/testflows-combinatorics","last_synced_at":"2025-10-30T06:51:29.342Z","repository":{"id":182371835,"uuid":"668402577","full_name":"testflows/TestFlows-Combinatorics","owner":"testflows","description":"TestFlows.com Open-Source Software Testing Framework Combinatorics ","archived":false,"fork":false,"pushed_at":"2023-07-20T11:50:15.000Z","size":46,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-18T12:46:01.305Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/testflows.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-07-19T18:10:40.000Z","updated_at":"2025-01-09T13:22:19.000Z","dependencies_parsed_at":"2023-07-19T18:53:40.094Z","dependency_job_id":null,"html_url":"https://github.com/testflows/TestFlows-Combinatorics","commit_stats":null,"previous_names":["testflows/testflows-combinatorics"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/testflows/TestFlows-Combinatorics","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/testflows%2FTestFlows-Combinatorics","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/testflows%2FTestFlows-Combinatorics/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/testflows%2FTestFlows-Combinatorics/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/testflows%2FTestFlows-Combinatorics/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/testflows","download_url":"https://codeload.github.com/testflows/TestFlows-Combinatorics/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/testflows%2FTestFlows-Combinatorics/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":281305242,"owners_count":26478374,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-27T02:00:05.855Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-30T06:51:06.406Z","updated_at":"2025-10-30T06:51:29.337Z","avatar_url":"https://github.com/testflows.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"`TestFlows.com Open-Source Software Testing Framework`_ Combinatorics\n---------------------------------------------------------------------\n\n.. image:: https://raw.githubusercontent.com/testflows/TestFlows-ArtWork/master/images/logo.png\n   :width: 20%\n   :alt: test bug\n   :align: center\n\nThe **testflows.combinatorics** module provides a convenient collection of tools\nused for combinatorial testing to check different combinations of the input parameters\nincluding calculating **covering arrays** for **pairwise** and **n-wise** testing using the `IPOG`_ algorithm.\n\n.. contents:: Read more about:\n   :backlinks: top\n   :depth: 2\n\nFeatures\n********\n\nProvides support for calculating the following:\n\n* pure Python implementation of covering arrays based on the `IPOG`_ algorithm that can be used with any Python test framework\n* permutations of input parameters\n* combinations with and without replacement\n* cartesian product of input iterables\n* binomial coefficients\n\nInstallation\n************\n\nWhen used with `TestFlows.com Open-Source Software Testing Framework`_ the **testflows.combinatorics** module\ncomes by default as a part of the **testflows.core** module. However, if you would like to use\nit with other test frameworks, it can be installed separately as follows:\n\n.. code-block:: bash\n\n   pip3 install --update testflows.combinatorics\n\n\nCovering Arrays - (Pairwise, N-wise) Testing\n********************************************\n\nThe **Covering(parameters, strength=2)** or **CoveringArray(parameters, strength=2)** class allows you to calculate a covering array\nfor some **k** parameters having the same or different number of possible values.\n\nThe class uses `IPOG`_, an in-parameter-order, algorithm as described in `IPOG: A General Strategy for T-Way Software Testing`_ paper by Yu Lei et al.\n\nFor any non-trivial number of parameters, exhaustively testing all possibilities is not feasible.\nFor example, if we have **10** parameters ($k=10$) that each has **10** possible values ($v=10$), the\nnumber of all possibilities is $v^k=10^{10} = {10}_{billion}$ thus requiring 10 billion tests for complete coverage.\n\nGiven that exhaustive testing might not be practical, a covering array could give us a much smaller\nnumber of tests if we choose to check all possible interactions only between some fixed number\nof parameters at least once, where an interaction is some specific combination, where order does not matter,\nof some **t** number of parameters, covering all possible values that each selected parameter could have.\n\n:Note:\n   You can find out more about covering arrays by visiting the US National Institute of Standards and Technology's (NIST)\n   `Introduction to Covering Arrays \u003chttps://math.nist.gov/coveringarrays/coveringarray.html\u003e`_ page.\n \n\nThe **Covering(parameters, strength=2)** takes the following arguments\n\nwhere,\n\n* **parameters**\n   specifies parameter names and their possible values and\n   is specified as a **dict[str, list[value]]**, where *key* is the parameter name and\n   *value* is a list of possible values for a given parameter.\n* **strength**\n   specifies the strength **t** of the covering array that indicates the number of parameters\n   in each combination, for which all possible interactions will be checked.\n   If **strength** equals the number of parameters, then you get the exhaustive case.\n\nThe return value of the **Covering(parameters, strength=2)** is a **CoveringArray** object that is an iterable\nof tests, where each test is a dictionary, with each key being the parameter name and its value\nbeing the parameter value.\n\nFor example,\n\n.. code-block:: python\n\n   from testflows.combinatorics import Covering\n\n   parameters = {\"a\": [0, 1], \"b\": [\"a\", \"b\"], \"c\": [0, 1, 2], \"d\": [\"d0\", \"d1\"]}\n\n   print(Covering(parameters, strength=2))\n\n\n.. code-block::\n\n   CoveringArray({'a': [0, 1], 'b': ['a', 'b'], 'c': [0, 1, 2], 'd': ['d0', 'd1']},2)[\n   6\n   a b c d\n   -------\n   0 b 2 d1\n   0 a 1 d0\n   1 b 1 d1\n   1 a 2 d0\n   0 b 0 d0\n   1 a 0 d1\n   ]\n\n\nGiven that in the example above, the **strength=2**, all possible 2-way (pairwise)\ncombinations of parameters **a**, **b**, **c**, and **d** are the following:\n\n.. code-block::\n\n   [('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'c'), ('b', 'd'), ('c', 'd')]\n\n\nThe six tests that make up the covering array cover all the possible interactions\nbetween the values of each of these parameter combinations. For example, the **('a', 'b')**\nparameter combination covers all possible combinations of the values that\nparameters **a** and **b** can take.\n\nGiven that parameter **a** can have values **[0, 1]**, and parameter **b** can have values **['a', 'b']**\nall possible interactions are the following:\n\n.. code-block::\n\n   [(0, 'a'), (0, 'b'), (1, 'a'), (1, 'b')]\n\n\nwhere the first element of each tuple corresponds to the value of the parameter **a**, and the second\nelement corresponds to the value of the parameter **b**.\n\nExamining the covering array above, we can see that all possible interactions of parameters\n**a** and **b** are indeed covered at least once. The same check can be done for other parameter combinations.\n\nChecking Covering Array\n~~~~~~~~~~~~~~~~~~~~~~~\n\nThe **check()** method of the **CoveringArray** can be used to verify that the tests\ninside the covering array cover all possible t-way interactions at least once, and thus\nmeet the definition of a covering array.\n\nFor example,\n\n.. code-block:: python\n\n   from testflows.combinatorics import Covering\n\n   parameters = {\"a\": [0, 1], \"b\": [\"a\", \"b\"], \"c\": [0, 1, 2], \"d\": [\"d0\", \"d1\"]}\n   tests = Covering(parameters, strength=2)\n\n   print(tests.check())\n\n\nDumping Covering Array\n~~~~~~~~~~~~~~~~~~~~~~\n\nThe **CoveringArray** object implements a custom **__str__** method, and therefore it can be easily converted into\na string representation similar to the format used in the `NIST covering array tables \u003chttps://math.nist.gov/coveringarrays/ipof/ipof-results.html\u003e`_.\n\nFor example,\n\n.. code-block:: python\n\n   print(Covering(parameters, strength=2))\n\n.. code-block::\n\n   CoveringArray({'a': [0, 1], 'b': ['a', 'b'], 'c': [0, 1, 2], 'd': ['d0', 'd1']},2)[\n   6\n   a b c d\n   -------\n   0 b 2 d1\n   0 a 1 d0\n   1 b 1 d1\n   1 a 2 d0\n   0 b 0 d0\n   1 a 0 d1\n   ]\n\n\nCombinations\n************\n\nThe **combinations(iterable, r, with_replacement=False)** function can be used to calculate\nall r-length combinations of elements in a specified iterable.\n\nFor example,\n\n.. code-block:: python\n\n   from testflows.combinatorics import combinations\n\n   parameters = {\"a\": [0, 1], \"b\": [\"a\", \"b\"], \"c\": [0, 1, 2], \"d\": [\"d0\", \"d1\"]}\n\n   print(list(combinations(parameters.keys(), 2)))\n\n\n.. code-block::\n\n   [('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'c'), ('b', 'd'), ('c', 'd')]\n\n:Note:\n   This function is equivalent to the `itertools.combinations \u003chttps://docs.python.org/3/library/itertools.html#itertools.combinations\u003e`_\n\nWith Replacement\n~~~~~~~~~~~~~~~~\n\nYou can calculate all combinations with replacement by setting the **with_replacement** argument to **True**.\n\nFor example,\n\n.. code-block:: python\n\n   from testflows.combinatorics import combinations\n\n   parameters = {\"a\": [0, 1], \"b\": [\"a\", \"b\"], \"c\": [0, 1, 2], \"d\": [\"d0\", \"d1\"]}\n\n   print(list(combinations(parameters.keys(), 2, with_replacement=True)))\n\n\n.. code-block::\n\n   [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'b'), ('b', 'c'), ('b', 'd'), ('c', 'c'), ('c', 'd'), ('d', 'd')]\n\n:Note:\n   The **with_replacement=True** option is equivalent to `itertools.combinations_with_replacement \u003chttps://docs.python.org/3/library/itertools.html#itertools.combinations_with_replacement\u003e`_\n\nCartesian Product\n*****************\n\nYou can calculate all possible combinations of elements from different iterables using\nthe cartesian **product(*iterables, repeat=1)** function.\n\nFor example,\n\n.. code-block:: python\n\n   from testflows.combinatorics import *\n\n   parameters = {\"a\": [0, 1], \"b\": [\"a\", \"b\"], \"c\": [0, 1, 2], \"d\": [\"d0\", \"d1\"]}\n\n   print(list(product(parameters[\"a\"], parameters[\"b\"])))\n\n\n.. code-block::\n\n   [(0, 'a'), (0, 'b'), (1, 'a'), (1, 'b')]\n\n:Note:\n   This function is equivalent to the `itertools.product \u003chttps://docs.python.org/3/library/itertools.html#itertools.product\u003e`_\n\n\nPermutations\n************\n\nThe **permutations(iterable, r=None)** function can be used to calculate\nthe r-length permutations of elements for a given iterable.\n\n:Note:\n   Permutations are different from **combinations**. In a combination, the elements\n   don't have any order, but in a permutation, the order of elements is important.\n\nFor example,\n\n.. code-block:: python\n\n   from testflows.combinatorics import *\n\n   parameters = {\"a\": [0, 1], \"b\": [\"a\", \"b\"], \"c\": [0, 1, 2], \"d\": [\"d0\", \"d1\"]}\n\n   print(list(permutations(parameters.keys(), 2)))\n\n\n.. code-block::\n\n   ('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'a'), ('b', 'c'), ('b', 'd'), ('c', 'a'), ('c', 'b'), ('c', 'd'), ('d', 'a'), ('d', 'b'), ('d', 'c')]\n\n\nand as we can see, both **('a', 'b')** and **('b', 'a')** elements are present.\n\n:Note:\n   This function is equivalent to the `itertools.permutations \u003chttps://docs.python.org/3/library/itertools.html#itertools.permutations\u003e`_\n\nBinomial Coefficients\n*********************\n\nYou can calculate the binomial coefficient, which is the same as\nthe number of ways to choose **k** items from **n** items without repetition and without order.\n\nBinomial coefficient is defined as\n\n.. image:: https://latex.codecogs.com/svg.image?%5Cfrac%7Bn!%7D%7Bk!(n-k!)%7D=%5Cbinom%7Bn%7D%7Bk%7D\n\nwhen $k \u003c= n$ and is zero when $k \u003e n$\n\nFor example,\n\n.. code-block:: python\n\n   from testflows.combinatorics import *\n\n   print(binomial(4,2))\n\n.. code-block::\n\n   6\n\n\nwhich means that there are 6 ways to choose 2 elements out of 4.\n\n:Note:\n   This function is equivalent to the  `math.comb \u003chttps://docs.python.org/3/library/math.html#math.comb\u003e`_\n\n.. _`IPOG`: https://citeseerx.ist.psu.edu/document?repid=rep1\u0026type=pdf\u0026doi=1362e14b8210a766099a9516491693c0c08bc04a\n.. _`IPOG: A General Strategy for T-Way Software Testing`: https://citeseerx.ist.psu.edu/document?repid=rep1\u0026type=pdf\u0026doi=1362e14b8210a766099a9516491693c0c08bc04a\n.. _`TestFlows.com Open-Source Software Testing Framework`: https://testflows.com\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftestflows%2Ftestflows-combinatorics","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftestflows%2Ftestflows-combinatorics","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftestflows%2Ftestflows-combinatorics/lists"}