{"id":13846982,"url":"https://github.com/peter-evans/mutation-testing","last_synced_at":"2025-05-07T02:25:37.252Z","repository":{"id":42222200,"uuid":"142961852","full_name":"peter-evans/mutation-testing","owner":"peter-evans","description":"Mutation Testing – featuring an example using Stryker, a framework for the JavaScript ecosystem","archived":false,"fork":false,"pushed_at":"2025-02-01T10:51:19.000Z","size":721,"stargazers_count":10,"open_issues_count":5,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-31T05:24:45.987Z","etag":null,"topics":["code-coverage","fault-based-testing","mutation-testing","stryker","stryker-testframework","unit-testing"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/peter-evans.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"peter-evans"}},"created_at":"2018-07-31T04:22:12.000Z","updated_at":"2025-02-01T10:51:17.000Z","dependencies_parsed_at":"2023-02-05T02:00:59.514Z","dependency_job_id":"f2f2ec72-e508-4f1f-9f0a-d9fbd6f4ac44","html_url":"https://github.com/peter-evans/mutation-testing","commit_stats":{"total_commits":100,"total_committers":4,"mean_commits":25.0,"dds":0.69,"last_synced_commit":"9951bf2c524db7ca619c9b10d76d189e31c14ff8"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peter-evans%2Fmutation-testing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peter-evans%2Fmutation-testing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peter-evans%2Fmutation-testing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peter-evans%2Fmutation-testing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/peter-evans","download_url":"https://codeload.github.com/peter-evans/mutation-testing/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252800139,"owners_count":21806103,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["code-coverage","fault-based-testing","mutation-testing","stryker","stryker-testframework","unit-testing"],"created_at":"2024-08-04T18:00:51.692Z","updated_at":"2025-05-07T02:25:37.232Z","avatar_url":"https://github.com/peter-evans.png","language":"JavaScript","funding_links":["https://github.com/sponsors/peter-evans"],"categories":["JavaScript"],"sub_categories":[],"readme":"# Mutation Testing [\u003cimg align=\"right\" alt=\"The blog of Peter Evans: Mutation Testing\" title=\"View blog post\" src=\"https://peterevans.dev/img/blog-published-badge.svg\"\u003e](https://peterevans.dev/posts/mutation-testing/)\n[![CI](https://github.com/peter-evans/create-or-update-project-card/workflows/CI/badge.svg)](https://github.com/peter-evans/create-or-update-project-card/actions?query=workflow%3ACI)\n[![Coverage Status](https://coveralls.io/repos/github/peter-evans/mutation-testing/badge.svg?branch=master)](https://coveralls.io/github/peter-evans/mutation-testing?branch=master)\n\n## What is mutation testing?\n\nMutation testing is a type of testing designed to assess the quality of unit tests.\nThis method is also sometimes described as \"fault based testing\" as it deliberately creates faults in software.\n\n#### How mutation testing frameworks work\n\n\u003cimg align=\"right\" width=\"353\" height=\"580\" src=\"mutation-testing.png\"\u003e\n\n1. Small syntactic changes are made to the application code. Each change, or mutation, is applied to a separate copy of the code thus creating many versions. These versions of the application code are described as \"mutants.\"\n2. The unit test suite is run against the original application code and all the mutant copies.\n3. The unit test results are compared between the original application code and mutants. If the unit test results are different then that mutant is said to be \"killed.\" If the unit test results are the same, the mutant is said to have \"survived.\"\n4. The tally of killed and survived mutants is displayed along with the code mutation for each surviving mutant.\n\nHere is an example where an if statement is modified to always equate to false.\nDespite this change the unit test results did not change. This indicates the unit testing is poor and not covering all logical cases.\n```\nMutator: IfStatement\n-       if(result.length \u003c arr[x].length) {\n+       if(false) {\n\nRan all tests for this mutant.\nMutant survived!\n```\n\n## Why use mutation testing?\n\nFor engineers, mutation testing is a great tool to support the development of unit test suites. \nIn both Test Driven Development (TDD) and writing unit tests after development, mutation testing helps fill in gaps in test suites.\nFurthermore, difficult to kill mutations can often highlight problems with the code itself.\nThis forces engineers to rethink their logic and refactor, ultimately leading to higher quality code.\n\nMutation testing can also be beneficial for managers or anybody involved in evaluating code quality and individual performance.\nIn an ideal world, all code should be peer reviewed to a high standard, but it can be difficult to achieve and maintain within a team.\nCode quality is often crudely evaluated by the unit test suite coverage of a codebase.\nHowever, code coverage is a very poor indictor of the quality of the codebase and its test suite.\nMutation testing can provide a metric, albeit not perfect, for the quality of unit testing that is verifiable without needing to look directly at the code itself.\n\n## Mutation Testing Frameworks\n\nSee [here](https://github.com/theofidry/awesome-mutation-testing) for a good list of frameworks for many different languages.\n\n## Mutation testing with [Stryker](https://stryker-mutator.io/)\n\nThis repository contains a mutation testing example using [Stryker](https://stryker-mutator.io/), a mutation testing framework for the JavaScript ecosystem.\nThe System under test (SUT) is a Node.js module containing a simple function that returns the longest word in a sentence.\n\n[src/index.js](src/index.js) snippet:\n```javascript\nmodule.exports.longestWord = function(str) {\n  if (typeof str !== \"string\") return \"\";\n\n  var arr = str.match(/\\w[a-z]{0,}/gi);\n  if (arr === null || arr.length === 0) return \"\";\n\n  var result = arr[0];\n  for(var x = 1; x \u003c arr.length; x++) {\n    if(result.length \u003c arr[x].length) {\n      result = arr[x];\n    } \n  }\n  return result;\n}\n```\n\nThe following test alone provides close to 100% code coverage but doesn't actually verify that the function does what it is supposed to do.\nThis test with no assertions is an extreme example and would no doubt be caught by code review, but just shows how easy it is to create poor unit tests.\n\n[test/test.js](test/test.js) snippet:\n```javascript\n  it('should return the longest word', function(done) {\n    var sentence = 'Ladies and gentlemen, the truth is that mutants are very real, and that they are among us.';\n    stringFormatter.longestWord(sentence);\n    done();\n  });\n```\n\nRunning the Stryker mutation test framework reveals that the unit testing is very poor.\n20 mutants survived!\n\n```\nRan 1.00 tests per mutant on average.\n----------|---------|----------|-----------|------------|----------|---------|\nFile      | % score | # killed | # timeout | # survived | # no cov | # error |\n----------|---------|----------|-----------|------------|----------|---------|\nAll files |    9.09 |        2 |         0 |         20 |        0 |       0 |\nindex.js  |    9.09 |        2 |         0 |         20 |        0 |       0 |\n----------|---------|----------|-----------|------------|----------|---------|\n```\n\nKilling those 20 mutants required four unit tests that can be seen [here](test/test.js).\nFurther unit tests could be written but for the purposes of this example only those required to kill the mutants are shown.\n\n## Mutation testing in production\n\nIf you have a large codebase you will soon find that testing against hundreds or thousands of mutants takes a long time!\nRather than hooking it directly in to your CI pipeline you might consider running the process once per night.\n\n## License\n\nMIT License - see the [LICENSE](LICENSE) file for details\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeter-evans%2Fmutation-testing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpeter-evans%2Fmutation-testing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeter-evans%2Fmutation-testing/lists"}