{"id":19657730,"url":"https://github.com/esss/pytest-update-test-results","last_synced_at":"2026-05-07T16:16:09.229Z","repository":{"id":61598513,"uuid":"545011266","full_name":"ESSS/pytest-update-test-results","owner":"ESSS","description":"Pytest plugin that merge test results in an existing JUnit XML file","archived":false,"fork":false,"pushed_at":"2024-04-08T19:27:40.000Z","size":21,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":13,"default_branch":"master","last_synced_at":"2025-01-10T00:54:34.294Z","etag":null,"topics":["hacktoberfest","pytest-plugin"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ESSS.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.rst","contributing":"CONTRIBUTING.rst","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-03T16:19:55.000Z","updated_at":"2022-12-30T17:04:48.000Z","dependencies_parsed_at":"2024-04-08T20:42:10.691Z","dependency_job_id":"7cabde37-7fe0-48ee-a4a6-ff012ec43273","html_url":"https://github.com/ESSS/pytest-update-test-results","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ESSS%2Fpytest-update-test-results","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ESSS%2Fpytest-update-test-results/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ESSS%2Fpytest-update-test-results/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ESSS%2Fpytest-update-test-results/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ESSS","download_url":"https://codeload.github.com/ESSS/pytest-update-test-results/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240965478,"owners_count":19885940,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacktoberfest","pytest-plugin"],"created_at":"2024-11-11T15:33:24.584Z","updated_at":"2026-05-07T16:16:04.165Z","avatar_url":"https://github.com/ESSS.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pytest-update-test-results\n\nDeal with flaky tests by re-running failed tests and overwrite test results\non a test report from a previous run.\n\nThe primary purpose of this plugin is to avoid flaky tests to break a CI\nbuild by executing them again after a failed run. The plugin does not rerun\ntests by itself, but allow the user to leverage the pytest `--last-failed`\noption to execute failed tests in a separate run and provides `--update-xml`\noption so if previously failed tests passes, the XML test report will be\nupdated (and as result, your CI would flag the test as successfull).\n\nIn summary, your CI script would have something like:\n\n```bash\n#/bin/bash\npytest tests/my-test-suite --junix-xml pytest.xml\n\nif [ $? -eq 2 ]  # pytest exit code for test failure\nthen\n    # re-run failed test and update previous run report.\n    pytest tests/my-test-suite --last-failed --update-xml pytest.xml\nfi\n```\n\n## Rationale\n\nIn our particular case, most of the flakiness comes from using a shared\nstorage on our virtualized on-premise CI cluster, which made for random\nfile access times to varies in order of 10x-100x depending on\nthe cluster loading. This does not influence small unit tests, but greatly\naffects execution time of integration and end-to-end tests, resulting\nin flaky tests due timeout or race condition.\n\nOur first attempt to solve the problem was to mark individual tests as flaky\n(using [flaky]). But that felt like beating a dead horse, since new flaky\ntests were constantly popping up.\n\nThen, we tried [pytest-rerunfailures] (which automatically\nre-run all test failures), but that didn't work either, due the way\n[it deals with session fixtures][1].\n\nSo we opted to develop this plugin to re-run only the failed tests from\nthe previous run in a new pytest execution (using pytest built-in option `--last-failed`) and update the XML report. Besides working just fine with\nsession fixtures, re-running failed tests in a separate process has other\nadvantages:\n\n1. By starting a new process for the test run, we make sure no \"test\ncontamination\" occurs (memory leaks vanish, objects that live during the\nentire test session are reset).\n1. Because we heavily parallelize the test suite (using **pytest-xdist**),\nsome flaky tests can occur due high CPU/IO usage. In the re-run\nwe can reduce number of parallel processes to make more resources\navailable for each test.\n\n## At ESSS\n\nTo enable this for our internal projects:\n\n1. Add `test_py_retest: True` to your project's `spec` file.\n2. Add `pytest-update-test-results` dependency. \n\n[flaky]: https://github.com/box/flaky\n[pytest-rerunfailures]: https://github.com/pytest-dev/pytest-rerunfailures\n[1]: https://github.com/pytest-dev/pytest-rerunfailures/issues/51\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fesss%2Fpytest-update-test-results","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fesss%2Fpytest-update-test-results","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fesss%2Fpytest-update-test-results/lists"}