{"id":13739096,"url":"https://github.com/cryptocode/marble","last_synced_at":"2025-04-09T22:53:31.781Z","repository":{"id":44326662,"uuid":"471977713","full_name":"cryptocode/marble","owner":"cryptocode","description":"A metamorphic testing library for Zig","archived":false,"fork":false,"pushed_at":"2025-03-04T11:22:59.000Z","size":24,"stargazers_count":39,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-09T22:53:26.250Z","etag":null,"topics":["metamorphic-testing","testing","testing-tool","testing-tools","zig","zig-library","zig-package"],"latest_commit_sha":null,"homepage":"","language":"Zig","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cryptocode.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-03-20T12:47:11.000Z","updated_at":"2025-03-25T01:37:34.000Z","dependencies_parsed_at":"2024-01-25T00:11:56.607Z","dependency_job_id":"8aac18fa-4830-479b-a21e-5ba062c2e3f3","html_url":"https://github.com/cryptocode/marble","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cryptocode%2Fmarble","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cryptocode%2Fmarble/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cryptocode%2Fmarble/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cryptocode%2Fmarble/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cryptocode","download_url":"https://codeload.github.com/cryptocode/marble/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248125634,"owners_count":21051766,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["metamorphic-testing","testing","testing-tool","testing-tools","zig","zig-library","zig-package"],"created_at":"2024-08-03T04:00:25.021Z","updated_at":"2025-04-09T22:53:31.766Z","avatar_url":"https://github.com/cryptocode.png","language":"Zig","readme":"\u003cimg align=\"right\" width=\"320\" src=\"https://user-images.githubusercontent.com/34946442/159163318-432052e3-69c7-4598-aaac-74d54f67c8b4.png\"\u003e\n\nMarble is a [metamorphic testing](https://en.wikipedia.org/wiki/Metamorphic_testing) library for Zig.\n\nThis library tracks Zig master and was last tested on `0.14.0-dev.3187+d4c85079c`\n\nMetamorphic testing is a powerful technique that provides additional test coverage by applying a number of transformations to test input, and then checking if certain relations still hold between the outputs. Marble will automatically run through all possible combinations of these transformations.\n\nHere's a [great introduction by](https://www.cockroachlabs.com/blog/metamorphic-testing-the-database/) Cockroach Labs. I highly recommend reading before using this library.\n\nThe repository contains a few [test examples](https://github.com/cryptocode/marble/blob/main/src/example_tests.zig)\n\n## Resources\n* [Hillel Wayne's blog post on Metamorphic Testing (highly recommended)](https://www.hillelwayne.com/post/metamorphic-testing/)\n* [Test your Machine Learning Algorithm with Metamorphic Testing](https://medium.com/trustableai/testing-ai-with-metamorphic-testing-61d690001f5c)\n* [Original paper by T.Y. Chen et al](https://www.cse.ust.hk/~scc/publ/CS98-01-metamorphictesting.pdf)\n* [Case study T.Y. Chen et al](http://grise.upm.es/rearviewmirror/conferencias/jiisic04/Papers/25.pdf)\n* [Metamorphic Testing and Beyond T.Y. Chen et al](https://www.cs.hku.hk/data/techreps/document/TR-2003-06.pdf)\n* [Survey on Metamorphic Testing](http://www.cs.ecu.edu/reu/reufiles/read/metamorphicTesting-16.pdf)\n* [Performance Metamorphic Testing](http://www.lsi.us.es/~jtroya/publications/NIER17_at_ICSE17.pdf)\n* [Experiences from Three Fuzzer Tools](https://johnwickerson.github.io/papers/dreamingup_MET21.pdf)\n* [Monarch, a similar library for Rust](https://github.com/zmitchell/monarch/blob/master/src/runner.rs)\n\n## Building\n\nTo build and run test examples:\n\n```bash\nzig build\nzig build test\n```\n\n## Importing the library\nAdd Marble as a Zig package in your build file, or simply import it directly after vendoring/adding a submodule:\n\n```zig\nconst marble = @import(\"marble/main.zig\");\n```\n\n## Writing tests\n\nA metamorphic Zig test looks something like this:\n\n```zig\nconst SinusTest = struct {\n    const tolerance = std.math.epsilon(f64) * 20;\n\n    /// This test has a single value, but you could also design the test to take an\n    /// array as input. The transformations, check and execute functions would then\n    /// loop through them all. Alternatively, the test can be run multiple times\n    /// with different inputs.\n    value: f64,\n\n    /// The mathematical property \"sin(x) = sin(π − x)\" must hold\n    pub fn transformPi(self: *SinusTest) void {\n        self.value = std.math.pi - self.value;\n    }\n\n    /// Adding half the epsilon must still cause the relation to hold given the tolerance\n    pub fn transformEpsilon(self: *SinusTest) void {\n        self.value = self.value + std.math.epsilon(f64) / 2.0;\n    }\n\n    /// A metamorphic relation is a relation between outputs in different executions.\n    /// This relation must hold after every execution of transformation combinations.\n    pub fn check(_: *SinusTest, original_output: f64, transformed_output: f64) bool {\n        return std.math.approxEqAbs(f64, original_output, transformed_output, tolerance);\n    }\n\n    /// Called initially to compute the baseline output, and after every transformation combination\n    pub fn execute(self: *SinusTest) f64 {\n        return std.math.sin(self.value);\n    }\n};\n\ntest \"sinus\" {\n    var i: f64 = 1;\n    while (i \u003c 100) : (i += 1) {\n        var t = SinusTest{ .value = i };\n        try std.testing.expect(try marble.run(SinusTest, \u0026t, .{}));\n    }\n}\n```\n\nYou will get compile time errors if the requirements for a metamorphic test are not met.\n\nIn short, you must provide a `value` field, a `check` function, an `execute` function and one or more `transform...` functions.\n\n### Writing transformations\nAdd one or more functions starting with `transform...`\n\nMarble will execute all combinations of the transformation functions. After every\ncombination, `execute` is called followed by `check`.\n\nTransformations should change the `value` property - Marble will remember what it was originally. The transformations must be such that `check`\nsucceeds. That is, the relations between the inital output and the transformed output must still hold.\n\n### Checking if relations still hold\nYou must provide a `check` function to see if one or more relations hold, and return true if so. If false is returned, the test fails with a print-out of the current transformation-combination.\n\nRelation checks may be conditional; check out the tests for examples on how this works.\n\n### Executing\nYou must provide an `execute` function that computes a result based on the current value. The simplest form will simply return the current value, but you can\ndo any arbitrary operation here. This function is called before any transformations to form a baseline. This baseline is passed as the first argument to `check`\n\n### Optional before/after calls\n\nBefore and after the test, and every combination, `before(...)` and `after(...)` is called if present. This is useful to reset state, initialize test cases, and perform clean-up.\n\n### What happens during a test run?\n\nUsing the example above, the following pseudocode runs will be performed:\n\n```\nbaseline = execute()\n\n// First combination\ntransformPi()\nout = execute()\ncheck(baseline, out)\n\n// Second combination\ntransformEpsilon()\nout = execute()\ncheck(baseline, out)\n\n// Third combination\ntransformPi()\ntransformEpsilon()\nout = execute()\ncheck(baseline, out)\n```\n\n### Configuring runs\n\nThe `run` function takes a `RunConfiguration`:\n\n```zig\n/// If set to true, only run each transformation once separately\nskip_combinations: bool = false,\n\n/// If true, print detailed information during the run\nverbose: bool = false,\n```\n\n### Error reporting\n\nIf a test fails, the current combination being executed is printed. For instance, the following tells us that the combination of `transformAdditionalTerm` and `transformCase` caused the metamorphic relation to fail:\n\n```\nTest [2/2] test \"query\"... Test case failed with transformation(s):\n  \u003e\u003e transformAdditionalTerm\n  \u003e\u003e transformCase\n```\n\n### Terminology\n\n* Source test case output: The output produced by `execute()` on the initial input. This is also known as the baseline.\n* Derived test case output: The output produced by `execute()` after applying a specific combination of transformations.\n* Metamorphic relation: A property that must hold when considering a source test case and a derived test case.\n","funding_links":[],"categories":["Development Tools"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcryptocode%2Fmarble","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcryptocode%2Fmarble","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcryptocode%2Fmarble/lists"}