{"id":15471415,"url":"https://github.com/sobolevn/heisenbug-2019","last_synced_at":"2025-05-07T06:06:24.093Z","repository":{"id":147569181,"uuid":"172949781","full_name":"sobolevn/heisenbug-2019","owner":"sobolevn","description":"Examples for Heisenbug 2019","archived":false,"fork":false,"pushed_at":"2019-04-18T10:41:51.000Z","size":97,"stargazers_count":12,"open_issues_count":0,"forks_count":1,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-05-07T06:06:19.364Z","etag":null,"topics":["code-quality","conference","mutation-testing","mutmut","python","talk","testing"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sobolevn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2019-02-27T16:24:03.000Z","updated_at":"2023-02-13T16:18:18.000Z","dependencies_parsed_at":null,"dependency_job_id":"b2ef017f-1eb9-44e9-975b-4f117f4f1d89","html_url":"https://github.com/sobolevn/heisenbug-2019","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sobolevn%2Fheisenbug-2019","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sobolevn%2Fheisenbug-2019/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sobolevn%2Fheisenbug-2019/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sobolevn%2Fheisenbug-2019/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sobolevn","download_url":"https://codeload.github.com/sobolevn/heisenbug-2019/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252823918,"owners_count":21809713,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["code-quality","conference","mutation-testing","mutmut","python","talk","testing"],"created_at":"2024-10-02T02:19:20.931Z","updated_at":"2025-05-07T06:06:24.072Z","avatar_url":"https://github.com/sobolevn.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Heisenbug 2019\n\n[![wemake.services](https://img.shields.io/badge/%20-wemake.services-green.svg?label=%20\u0026logo=data%3Aimage%2Fpng%3Bbase64%2CiVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAMAAAAoLQ9TAAAABGdBTUEAALGPC%2FxhBQAAAAFzUkdCAK7OHOkAAAAbUExURQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP%2F%2F%2F5TvxDIAAAAIdFJOUwAjRA8xXANAL%2Bv0SAAAADNJREFUGNNjYCAIOJjRBdBFWMkVQeGzcHAwksJnAPPZGOGAASzPzAEHEGVsLExQwE7YswCb7AFZSF3bbAAAAABJRU5ErkJggg%3D%3D)](https://wemake.services)\n[![Build Status](https://travis-ci.org/sobolevn/heisenbug-2019.svg?branch=master)](https://travis-ci.org/sobolevn/heisenbug-2019)\n[![Coverage](https://coveralls.io/repos/github/sobolevn/heisenbug-2019/badge.svg?branch=master)](https://coveralls.io/github/sobolevn/heisenbug-2019?branch=master)\n[![wemake-python-styleguide](https://img.shields.io/badge/style-wemake-000000.svg)](https://github.com/sobolevn/heisenbug-2019)\n\nDemo app that shows how can we effectively use mutation testing.\n\n## Contents\n\n### missing_assert.py\n\nThis is a very basic example of test that does not do anything.\nBecause we have missed that our `assert` is commented out by accident!\n\n```\n⠙ 2/2  🎉 0  ⏰ 0  🤔 0  🙁 2\n```\n\nTo solve this, just uncomment the `assert` in [test_missing_assert.py](https://github.com/sobolevn/heisenbug-2019/blob/master/tests/test_missing_assert.py)\n\n### simple.py\n\nThis example is used to illustrate the simplest use-case possible.\nWe only have one function with a single statement.\nThis statement will be mutated, our mutation test will catch that.\n\n```\n⠼ 1/1  🎉 0  ⏰ 0  🤔 0  🙁 1\n```\n\nThe solution to this case is available here: [test_simple.py](https://github.com/sobolevn/heisenbug-2019/blob/master/tests/test_simple.py)\n\n### constants.py\n\nUsing the same constants for testing and production\ndoes not actually test anything.\n\n```\n⠇ 6/6  🎉 2  ⏰ 0  🤔 1  🙁 3\n```\n\nSince we will miss cases like this one:\n\n```diff\n--- heisenbug/constants.py\n+++ heisenbug/constants.py\n@@ -1,7 +1,7 @@\n # -*- coding: utf-8 -*-\n\n\n-BAD_FUNCTION_NAMES = ['dir', 'vars', 'locals', 'globals']\n+BAD_FUNCTION_NAMES = ['dir', 'vars', 'locals', 'XXglobalsXX']\n```\n\nThe tests will move in sync with the production code, following the bugs introduced there!\n\nSolution: duplicate your constants in tests, see [test_constants.py](https://github.com/sobolevn/heisenbug-2019/blob/master/tests/test_constants.py)\n\n### flask_app.py\n\nThis example represents a regular `flask` application.\nIt is configured to swallow exceptions and log them somewhere.\n\nIt seems to be fully tested, but has one big mistake inside.\n\n```\n⠋ 10/10  🎉 2  ⏰ 0  🤔 0  🙁 8\n```\n\nThis is the case we are looking for:\n\n```diff\n--- heisenbug/flask_app.py\n+++ heisenbug/flask_app.py\n@@ -18,7 +18,7 @@\n @app.route('/{int:index}')\n def hello(index):\n     \"\"\"View that will fail in production.\"\"\"\n-    return 'Hello, world! {0} faith in you.'.format(1 * index)\n+    return 'Hello, world! {0} faith in you.'.format(1 / index)\n```\n\nSince we swallow exceptions and test only partial of our output,\nwe will never be able to find the problem of when input is zero and causes a divide by zero.\n\nMain take-away: high-level integrations tests are not good enough on their own.\n\nSolution: do not use `200` code where something goes wrong,\nmake sure that you write enough unit-tests for your view logic.\n\n### algorithm.py\n\nThis is a simple `bubble_sort` algorithm.\n\nI have taken an example algorithm as is from [`TheAlgorithms/Python`](https://github.com/TheAlgorithms/Python/blob/master/sorts/bubble_sort.py),\nwhich is a very popular `python` library.\n\nThis is the result (please, take a not that it has several false positives):\n\n```\n⠼ 20/20  🎉 15  ⏰ 0  🤔 0  🙁 5\n```\n\nYou can see that we even use property-based tests [here](https://github.com/sobolevn/heisenbug-2019/blob/master/tests/test_algorithm.py).\n\nBut it still does not save us from the false-positives of mutation testing.\n\nMain take-away: it is really hard to use mutation testing\nto test algorithms built with performance and optimisations in mind.\n\n### opensource_disl_case.ipynb\n\nThis is a real-world example about\nmy open-source project called [`docker-image-size-limit`](https://github.com/wemake-services/docker-image-size-limit).\n\nIt is a perfectly working app with just 70 lines for simple `python` code.\nFully tested with E2E and unit tests, typed, 100% covered.\nAnd it has [~10 surviving mutants](https://github.com/sobolevn/heisenbug-2019/blob/master/heisenbug/opensource_disl_case.ipynb).\n\n```\n⠇ 29/29  🎉 19  ⏰ 0  🤔 2  🙁 8\n```\n\nThis article shows you exactly what happened and how to solve this case.\n\n\n## Resources\n\nHere I have collected different resource that you might also be interested in.\n\n### Articles\n\n- https://hackernoon.com/mutmut-a-python-mutation-testing-system-9b9639356c78\n- https://nedbatchelder.com/blog/201903/mutmut.html\n\n### Talks\n\n- https://vimeo.com/191364978\n- https://vimeo.com/171319754\n\n\n## License\n\nContent is licensed under [CC BY-NC](https://creativecommons.org/licenses/by-nc/4.0/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsobolevn%2Fheisenbug-2019","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsobolevn%2Fheisenbug-2019","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsobolevn%2Fheisenbug-2019/lists"}