{"id":19082223,"url":"https://github.com/waikato-datamining/wai-test","last_synced_at":"2026-06-18T04:32:13.403Z","repository":{"id":62588117,"uuid":"200544574","full_name":"waikato-datamining/wai-test","owner":"waikato-datamining","description":"Python library for unit tests.","archived":false,"fork":false,"pushed_at":"2025-04-14T04:48:01.000Z","size":22,"stargazers_count":0,"open_issues_count":3,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-08-30T12:37:20.731Z","etag":null,"topics":["python","unit-testing"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/waikato-datamining.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.rst","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-08-04T21:44:53.000Z","updated_at":"2025-04-14T04:13:25.000Z","dependencies_parsed_at":"2022-11-03T17:01:33.381Z","dependency_job_id":null,"html_url":"https://github.com/waikato-datamining/wai-test","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/waikato-datamining/wai-test","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waikato-datamining%2Fwai-test","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waikato-datamining%2Fwai-test/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waikato-datamining%2Fwai-test/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waikato-datamining%2Fwai-test/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/waikato-datamining","download_url":"https://codeload.github.com/waikato-datamining/wai-test/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waikato-datamining%2Fwai-test/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34476727,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-18T02:00:06.871Z","response_time":128,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["python","unit-testing"],"created_at":"2024-11-09T02:42:37.946Z","updated_at":"2026-06-18T04:32:13.387Z","avatar_url":"https://github.com/waikato-datamining.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# wai-test\nPython library for unit tests.\n\n## Testing Philosophy\nThis test library is designed around the testing of an individual `subject`,\nwhere a `subject` is usually either a class or a function. For each `subject`\nto be tested, one or more test classes should be written, each of which\nderives from the base `AbstractTest` class.\n\nEach test class defines the `subject` it tests, and a number of tests for\nthat subject to pass. Each test is a method of the test class, configured\nwith some of the decorators provided by this library. Furthermore, common\nconfiguration between tests can be provided by overriding class methods of\nthe base `AbstractTest` class.\n\n---\n\n## Test Types\nThere are 3 types of test currently supported by this library: standard tests,\nregression tests, and exception tests.\n\n### Standard Tests\nA standard test involves testing the subject against a known value. It is\nspecified by decorating the test method with the `@Test` decorator. The method\nshould use the test subject to produce some value, which is then compared\nto the expected value with one of the `unittest` assertions via the `self`\nreference, e.g. `self.assertEqual(subject_value, expected_value, msg)`.\n\n### Regression Tests\nA regression test involves comparing the subject to a previously-produced\nversion of the subject. A regression test is specified by marking the test\nsubject with the `@RegressionTest` decorator. The test method should return\na dictionary from regression labels to the tested value for that label. This\nway multiple related values can be regression-tested at once.\n\nBecause a regression test must have a past value to compare the current value\nagainst, regression results are saved to disk. In order to facilitate this,\nresult serialisers which can write the result to disk need to be specified.\nThis can be done either at a test-class level (providing common serialisers\nfor all tests) and/or at a per-test level, allowing specific serialisation\nfor certain tests.\n\n### Exception Tests\nException tests are similar to standard tests, but in that they should not\ncompare a result via value, instead raising an exception to determine pass/fail\nstatus. Exception tests can be specified by adding the `@ExceptionTest` decorator\nto the test method.\n\n---\n\n## Decorators\nDecorators are used to define the type of test that a test method performs,\nas well as specify test-specific configuration for the test.\n\n### Test Type Decorators\nThese decorators specify what type of test the decorated method defines.\nOnly one of these decorators should be applied to each test method.\n\n* **`@Test`** - Specifies a standard test method. Takes no arguments.\n* **`@RegressionTest`** - Specifies a regression test method. Takes no\n                          arguments.\n* **`@ExceptionTest`** - Specifies an exception test method. Takes as\n                         argument a series of exception types. The test will\n                         pass if any of the given types of exception is thrown,\n                         and fail if not.\n                         \n### Test-Specfic Configuration Decorators\nThese decorations configure the decorated test in some way.\n\n* **`@Skip`** - Specifies that this test should be skipped.\n* **`@SubjectArgs`** - Overrides the default arguments to the test subject\n                       for this test. Takes the same arguments as the subject,\n                       and forwards them directly. If this decorator is specified\n                       multiple times on the same method, only the top-most decorator\n                       will apply.\n* **`@ExpectedFailure`** - Marks the decorated method as a test that is\n                           expected to fail. Inverts the success criteria for\n                           this test. I.e. A test that passes under normal\n                           conditions is considered to have failed, and a test\n                           that fails under normal conditions is considered to\n                           have passed.\n* **`@WithSerialiser`** - This test sets the serialiser for a given result-type\n                          for the decorated test only. Can be used to override\n                          the common serialisers for the test class. Takes as\n                          arguments the type of result being serialised, and the\n                          serialiser to use for serialising that type in this\n                          test method.\n\n---\n\n## Common Test Configuration\nDefining common configuration for all tests in a given test class is done by\noverriding specific class methods of the `AbstractTest` base-class.\n\n* **`subject_type`** - This method specifies the subject being tested by this\n                       test-class. It should return a callable to use to\n                       instantiate a test-subject, usually the class or the\n                       function itself.\n* **`common_resources`** - This method should return any common resources that\n                           are required by all test methods in the test-class.\n                           The resources will be passed to each test method\n                           as positional arguments, so can be captured via\n                           the `*args` pattern or by unpacking into individual\n                           named arguments.\n* **`common_serialisers`** - This method should return a dictionary from types\n                             to serialisers for those types. This serialiser\n                             mapping is added to the basic serialisers for string\n                             and bytes (possibly overriding them). Individual tests\n                             can override or define additional serialisers with the\n                             `@WithSerialiser` decorator.\n* **`common_arguments`** - This method should return the positional and keyword\n                           arguments that will be supplied to the subject on\n                           instantiation for each test. By default no arguments\n                           are supplied. Can be overridden on a per-test basis\n                           via the `@SubjectArgs` decorator.\n                           \n---\n## Serialisers\nSerialisers are used by regression tests to save a regression reference to disk, and\nto load it again in future to compare to the results of future runs of the test. Two\nbasic serialisers are provided with the library, `StringSerialiser` and\n`BytesSerialiser`. Custom serialisers can be defined by sub-classing the\n`RegressionSerialiser` base-class. Serialisers can also define their own notion of\nequality between the result of the current test and the reference value by overriding\nthe `compare` method.\n\n---\n\n## Test Method Signatures\nAll tests in a test-class should have one of the following signatures.\n\n### Standard Test\n```python\n@Test\n... additional decorators ...\ndef standard_test_name(self,\n                       subject,\n                       named_resource_1, named_resource_2,\n                       *unnamed_resources):\n    ... test code ...\n    \n    self.assertSomething(some_value)\n\n```\n\n### Regression Test\n```python\n@RegressionTest\n... additional decorators ...\ndef regression_test_name(self,\n                         subject,\n                         named_resource_1, named_resource_2,\n                         *unnamed_resources):\n    ... test code ...\n    \n    return {\n        \"regression_name_1\": regression_value_1,\n        \"regression_name_2\": regression_value_2,\n        ... additional regressions ...\n    }\n\n```\n\n### Exception Test\n```python\n@ExceptionTest(ExceptionType1, ExceptionType2, ...)\n... additional decorators ...\ndef exception_test_name(self,\n                        subject,\n                        named_resource_1, named_resource_2,\n                        *unnamed_resources):\n    ... test code ...\n    \n    if condition1:\n        raise ExceptionType1(msg)\n        \n    ... more test code ..\n    \n    if condition2:\n        raise ExceptionType2(msg)\n        \n    ...\n\n```\n\n---\n\n## Example\nThe following is an example of a basic class and the setup that will test it.\n\nThe class being tested:\n```python\nclass MyClass:\n    def __init__(self, value: int = 0):\n        self.value = value\n\n    def transform(self, x: int, y: int) -\u003e int:\n        if y \u003c 0:\n            raise ValueError(\"y can't be less than 0\")\n\n        return x + y + self.value\n\n\n```\n\nA couple of different serialiser for integers:\n```python\nclass IntSerialiser(RegressionSerialiser[int]):\n    @classmethod\n    def binary(cls) -\u003e bool:\n        return False\n\n    @classmethod\n    def extension(cls) -\u003e str:\n        return \"int\"\n\n    @classmethod\n    def serialise(cls, result: int, file: IO[str]):\n        file.write(str(result))\n\n    @classmethod\n    def deserialise(cls, file: IO[str]) -\u003e int:\n        return int(file.read())\n        \n        \nclass IntSerialiser2(IntSerialiser):\n    @classmethod\n    def serialise(cls, result: int, file: IO[str]):\n        super().serialise(result + 1, file)\n\n    @classmethod\n    def deserialise(cls, file: IO[str]) -\u003e int:\n        return super().deserialise(file) - 1\n\n```\n\nThe test class:\n```python\nclass MyClassTest(AbstractTest):\n    @classmethod\n    def subject_type(cls):\n        return MyClass  # Testing the MyClass class\n\n    @classmethod\n    def common_resources(cls):\n        return 1,  # All tests take an initial value of 1\n\n    @classmethod\n    def common_serialisers(cls):\n        return {\n            int: IntSerialiser  # Default serialiser for int values\n        }\n\n    @classmethod\n    def common_arguments(cls):\n        return (15,), {}  # Most tests will initialise MyClass with 15\n\n    # Standard test with no additional decorators\n    @Test\n    def standard_test(self, subject: MyClass, x: int):\n        self.assertEqual(subject.transform(x, 2),\n                         18)\n\n    # Same as above, but with the wrong result value\n    # so we expect it to fail\n    @Test\n    @ExpectedFailure\n    def standard_test_failure(self, subject: MyClass, x: int):\n        self.assertEqual(subject.transform(x, 2),\n                         17)\n\n    # Same as above, but overriding subject arguments\n    @Test\n    @SubjectArgs(10)\n    def standard_test_with_10(self, subject: MyClass, x: int):\n        self.assertEqual(subject.transform(x, 2),\n                         13)\n\n    # Exception test, make sure MyClass raises ValueError\n    # on negative y\n    @ExceptionTest(ValueError)\n    def exception_test(self, subject: MyClass, x: int):\n        return subject.transform(x, -1)\n\n    # Regression test with two regression outputs\n    @RegressionTest\n    def regression_test(self, subject: MyClass, x: int):\n        return {\n            \"transform5\": subject.transform(x, 5),\n            \"transform2\": subject.transform(x, 2)\n        }\n\n    # Same as above, but overriding the serialiser\n    @RegressionTest\n    @WithSerialiser(int, IntSerialiser2)\n    def regression_test_2(self, subject: MyClass, x: int):\n        return {\n            \"transform5\": subject.transform(x, 5),\n            \"transform2\": subject.transform(x, 2)\n        }\n\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwaikato-datamining%2Fwai-test","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwaikato-datamining%2Fwai-test","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwaikato-datamining%2Fwai-test/lists"}