{"id":15403650,"url":"https://github.com/rogervinas/mutation-testing","last_synced_at":"2025-04-16T03:43:51.709Z","repository":{"id":115435706,"uuid":"539349011","full_name":"rogervinas/mutation-testing","owner":"rogervinas","description":"🧟‍♂️ Mutation Testing with Pitest and Kotlin","archived":false,"fork":false,"pushed_at":"2025-03-26T13:27:52.000Z","size":1688,"stargazers_count":4,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-26T14:40:50.189Z","etag":null,"topics":["kotlin","mutation-testing","pitest","testing"],"latest_commit_sha":null,"homepage":"https://dev.to/rogervinas/mutation-testing-2cfd","language":"Kotlin","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rogervinas.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-09-21T06:56:58.000Z","updated_at":"2025-03-26T13:27:54.000Z","dependencies_parsed_at":"2023-12-28T11:32:09.974Z","dependency_job_id":"295b08ae-97a9-47b3-a32c-eb9376d9f7d1","html_url":"https://github.com/rogervinas/mutation-testing","commit_stats":{"total_commits":43,"total_committers":3,"mean_commits":"14.333333333333334","dds":0.4418604651162791,"last_synced_commit":"c3a2208e5ba09810fd8ee12e22ab705d9ad13577"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rogervinas%2Fmutation-testing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rogervinas%2Fmutation-testing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rogervinas%2Fmutation-testing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rogervinas%2Fmutation-testing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rogervinas","download_url":"https://codeload.github.com/rogervinas/mutation-testing/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249191902,"owners_count":21227674,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kotlin","mutation-testing","pitest","testing"],"created_at":"2024-10-01T16:09:31.865Z","updated_at":"2025-04-16T03:43:51.692Z","avatar_url":"https://github.com/rogervinas.png","language":"Kotlin","readme":"[![CI](https://github.com/rogervinas/mutation-testing/actions/workflows/ci.yml/badge.svg)](https://github.com/rogervinas/mutation-testing/actions/workflows/ci.yml)\n![Java](https://img.shields.io/badge/Java-21-blue?labelColor=black)\n![Kotlin](https://img.shields.io/badge/Kotlin-2.x-blue?labelColor=black)\n![Pitest](https://img.shields.io/badge/Pitest-1.15.0-blue?labelColor=black)\n\n# Mutation Testing with Pitest and Kotlin\n\n[Mutation Testing](https://en.wikipedia.org/wiki/Mutation_testing) is a way to evaluate the quality of our tests by modifying (\"mutating\") our code and counting how many of these modifications (\"mutations\") do not pass the tests (\"are killed\"). \n\n**The greater the number of mutations killed the better!**\n\nLet me visualize it with this [The Walking Dead](https://www.imdb.com/title/tt1520211/) reference:\n\n![Mutation-Testing](doc/mutation-testing.png)\n\nIn this PoC I want to use [Pitest](https://pitest.org/) and [Kotlin](https://kotlinlang.org/) to see how this works in practice.\n\n## Gradle project setup\n\nFirst we start creating a gradle project with kotlin DSL and [gradle-pitest-plugin](https://plugins.gradle.org/plugin/info.solidsoft.pitest).\n\nIn `build.gradle.kts` we declare the plugin:\n```kotlin\nplugins {\n    id(\"info.solidsoft.pitest\") version \"1.9.11\"\n}\n```\n\nAnd we [configure it](https://github.com/szpak/gradle-pitest-plugin#plugin-configuration):\n```kotlin\nconfigure\u003cPitestPluginExtension\u003e {\n  junit5PluginVersion.set(\"1.0.0\")\n  avoidCallsTo.set(setOf(\"kotlin.jvm.internal\"))\n  mutators.set(setOf(\"STRONGER\"))\n  targetClasses.set(setOf(\"org.rogervinas.*\"))\n  targetTests.set(setOf(\"org.rogervinas.*\"))\n  threads.set(Runtime.getRuntime().availableProcessors())\n  outputFormats.set(setOf(\"XML\", \"HTML\"))\n  mutationThreshold.set(75)\n  coverageThreshold.set(60)\n}\n```\n* We specify in `targetClasses` where our \"to be mutated\" code is and in `targetTests` where our tests are.\n* We can specify a `mutationThreshold` in % so any value below that will make the test fail. In this case I expect at least 75% of the mutations to be killed.\n* We can also specify a `coverageThreshold` in % for the minimum line coverage that we want to ensure. In this example I've set it to 60%.\n\n## Implementation\n\nThe implementation we are going to test is quite simple:\n```kotlin\nclass MyImpl {\n  fun doSomething(a: Int, b: Int) = when {\n    (a \u003c 0 || b \u003c 0) -\u003e \"Either A or B are negative\"\n    (a == 0 \u0026\u0026 b == 0) -\u003e \"Both A and B are zero\"\n    (a \u003e b) -\u003e \"A is greater than B\"\n    (b \u003e a) -\u003e \"B is greater than A\"\n    else -\u003e \"A and B are equal\"\n  }\n}\n```\n\n## Test\n\nFor the test we use a [Junit5 Parameterized test](https://junit.org/junit5/docs/current/user-guide/#writing-tests-parameterized-tests):\n```kotlin\ninternal class MyImplTest {\n  @ParameterizedTest\n  @CsvSource(\n    value = [\n      \"0, 0, Both A and B are zero\",\n      // other cases ...\n    ]\n  )\n  fun `should do something`(a: Int, b: Int, expectedResult: String) {\n    assertThat(MyImpl().doSomething(a, b)).isEqualTo(expectedResult)\n  }\n}\n```\n\n## Mutation Testing with Pitest\n\nLet's execute **Pitest** but only with one test case `0, 0, Both A and B are zero` and see what happens:\n\n```shell\n\u003e ./gradlew pitest\n\n================================================================================\n- Statistics\n================================================================================\n\u003e\u003e Line Coverage: 5/8 (63%)\n\u003e\u003e Generated 17 mutations Killed 7 (41%)\n\u003e\u003e Mutations with no coverage 6. Test strength 64%\n\u003e\u003e Ran 11 tests (0.65 tests per mutation)\nEnhanced functionality available at https://www.arcmutate.com/\nException in thread \"main\" java.lang.RuntimeException:\nMutation score of 41 is below threshold of 75\n```\n\nThe test fails because from 17 mutations generated only 7 were killed, that is 41%, below the expected threshold of 75% 😟\n\nIf we examine the **Pitest** report generated in `build/reports/pitest` we can check all the mutations applied to each line:\n\n![Pitest-Report-1](doc/pitest-report-1.png)\n\nThen, what if we add all the test cases?\n\n```shell\n\u003e ./gradlew pitest\n\n================================================================================\n- Statistics\n================================================================================\n\u003e\u003e Line Coverage: 8/8 (100%)\n\u003e\u003e Generated 17 mutations Killed 15 (88%)\n\u003e\u003e Mutations with no coverage 0. Test strength 88%\n\u003e\u003e Ran 44 tests (2.59 tests per mutation)\n```\n\nWe are on the right path, now our line coverage is 100% and from 17 mutations generated all except 2 were killed, that is 88%, above our expected threshold of 75% 😃\n\nBut wait a minute ... why are there 2 surviving mutations if I added all the possible test cases? 🤔\n\nLet's check the **Pitest** report again:\n\n![Pitest-Report-2](doc/pitest-report-2.png)\n\nIf we add these two more test cases ...\n```\n\"1, 0, A is greater than zero\",\n\"0, 1, B is greater than zero\",\n```\n\nAnd we execute **Pitest** one more time:\n\n```shell\n\u003e ./gradlew pitest\n\n================================================================================\n- Statistics\n================================================================================\n\u003e\u003e Line Coverage: 8/8 (100%)\n\u003e\u003e Generated 17 mutations Killed 17 (100%)\n\u003e\u003e Mutations with no coverage 0. Test strength 100%\n\u003e\u003e Ran 44 tests (2.59 tests per mutation)\n```\n\nAll 17 mutations were killed! 🎉\n\nThanks and happy coding! 💙\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frogervinas%2Fmutation-testing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frogervinas%2Fmutation-testing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frogervinas%2Fmutation-testing/lists"}