{"id":30114273,"url":"https://github.com/cicatriiz/flakewall","last_synced_at":"2026-04-20T03:33:08.257Z","repository":{"id":308777468,"uuid":"1034087545","full_name":"Cicatriiz/flakewall","owner":"Cicatriiz","description":"Language-agnostic flaky test guard: parse JUnit, score flakiness, quarantine, and retry (pytest/jest). Lightweight CLI for CI.","archived":false,"fork":false,"pushed_at":"2025-08-07T21:30:36.000Z","size":43,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-14T03:29:24.641Z","etag":null,"topics":["ci","cli","developer-tools","flaky-tests","github-actions","jest","junit","pytest","quarantine","test-retry"],"latest_commit_sha":null,"homepage":"https://github.com/Cicatriiz/flakewall","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Cicatriiz.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-07T20:26:55.000Z","updated_at":"2025-08-07T21:30:39.000Z","dependencies_parsed_at":"2025-08-07T21:43:19.175Z","dependency_job_id":null,"html_url":"https://github.com/Cicatriiz/flakewall","commit_stats":null,"previous_names":["cicatriiz/flakewall"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/Cicatriiz/flakewall","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cicatriiz%2Fflakewall","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cicatriiz%2Fflakewall/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cicatriiz%2Fflakewall/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cicatriiz%2Fflakewall/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Cicatriiz","download_url":"https://codeload.github.com/Cicatriiz/flakewall/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cicatriiz%2Fflakewall/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32031583,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-20T00:18:06.643Z","status":"online","status_checked_at":"2026-04-20T02:00:06.527Z","response_time":94,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ci","cli","developer-tools","flaky-tests","github-actions","jest","junit","pytest","quarantine","test-retry"],"created_at":"2025-08-10T07:39:32.162Z","updated_at":"2026-04-20T03:33:08.237Z","avatar_url":"https://github.com/Cicatriiz.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# flakewall\n\n[![CI](https://github.com/Cicatriiz/flakewall/actions/workflows/ci.yml/badge.svg)](https://github.com/Cicatriiz/flakewall/actions/workflows/ci.yml)\n[![PyPI](https://img.shields.io/pypi/v/flakewall.svg)](https://pypi.org/project/flakewall/)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)\n[![Python](https://img.shields.io/badge/python-3.9%2B-blue.svg)](pyproject.toml)\n\nLanguage-agnostic flaky test guard for CI. Parse JUnit XML, score flakiness, quarantine known flakes, and selectively retry tests (pytest/jest) – all via a tiny CLI.\n\n## What it does (simple explanation)\nFlaky tests are tests that sometimes pass and sometimes fail without code changes. `flakewall` helps you:\n- Identify flaky candidates by scanning JUnit XML across runs\n- Prevent known flakes from breaking CI (quarantine)\n- Optionally re-run failing tests to prove they’re flaky (then auto-quarantine)\n\n## Why it’s useful\n- Works with any language that can emit JUnit XML\n- No vendor lock-in, no background service – just a small CLI\n- Plays nicely with your existing CI and test runner\n\n## Install\nPick one of the following (no PyPI required):\n\nFrom source (recommended for dev):\n```bash\ngit clone https://github.com/Cicatriiz/flakewall.git\ncd flakewall\nmake dev   # creates venv and installs in editable mode\n# or\npython -m venv .venv \u0026\u0026 . .venv/bin/activate \u0026\u0026 pip install -e .[dev]\n```\n\npipx (isolated, global CLI without polluting system site-packages):\n```bash\npipx install git+https://github.com/Cicatriiz/flakewall.git\n```\n\nDocker (no local Python required):\n```bash\ndocker run --rm -v \"$PWD\":/work -w /work python:3.11-slim bash -lc \\\n  \"pip install git+https://github.com/Cicatriiz/flakewall.git \u0026\u0026 flakewall --help\"\n```\n\nFrom PyPI (optional):\n```bash\npip install flakewall\n```\n\n## Quick start\n```bash\n# 1) Generate JUnit XML in your test run\npytest --junitxml=reports/junit.xml\n\n# 2) Initialize config files\nflakewall init\n\n# 3) Guard CI: exit non‑zero only if non‑quarantined failures exist\nflakewall guard --junit \"reports/**/*.xml\"\n\n# 4) Score flakiness across multiple reports (optional)\nflakewall score --junit \"reports/**/*.xml\" --min-total 2 --json \u003e flakewall_score.json\n\n# 5) Retry specific failing tests (pytest/jest) and optionally auto‑quarantine\nflakewall retry --framework pytest --from-junit \"reports/**/*.xml\" --max-retries 1 --auto-quarantine\n```\n\n## Commands\n- `init` – creates `.flakewall/config.yml` and `.flakewall/quarantine.yml`\n- `report` – lists current failing test IDs and marks quarantined ones\n- `guard` – exits 0 if failures ⊆ quarantine; else exits 1. Flags: `--gh-annotations`, `--slack-webhook URL`\n- `score` – detects pass/fail flips across JUnit files; supports `--json` and `--rich` (adds flips, instability index, and streaks)\n- `auto-quarantine` – adds flip-prone tests above a `--threshold` to quarantine; optional `--ttl-runs N` to auto-unquarantine after N runs\n- `retry` – re-runs tests up to `--max-retries`; can `--auto-quarantine` proofs of flakiness; supports `--json`; optional `--junit-out path` to write a merged JUnit\n\n## Configuration\nFiles created by `init` under `.flakewall/`:\n- `config.yml` – for example:\n  ```yaml\n  retries: 0\n  report_glob: \"**/junit*.xml\"\n  ```\n- `quarantine.yml` – list of quarantined test IDs:\n  ```yaml\n  quarantined:\n    - package.module::TestClass::test_name\n  ```\n\n## CI examples\n- GitHub Actions: `examples/ci/github-actions.yml`\n- GitLab CI: `examples/ci/gitlab-ci.yml`\n- CircleCI: `examples/ci/circleci-config.yml`\n- Azure Pipelines: `examples/ci/azure-pipelines.yml`\n- Jenkins: `examples/ci/Jenkinsfile`\n\n## Supported inputs and runners\n- Input: JUnit XML (generic, from any language)\n- Retry adapters:\n  - pytest (default; `-k \u003cClass and name\u003e`)\n  - jest (`-t \u003cname\u003e`; include file path via `path/to/test.spec.ts::test name` if needed)\n  - vitest (`vitest run -t \u003cname\u003e [file]`)\n  - go (`go test -run \u003cname\u003e ./...`)\n  - dotnet (`dotnet test --filter FullyQualifiedName~\u003cname\u003e`)\n  - shell (generic template; provide `--cmd '...{test}...'`)\n\n## Notes and limitations\n- Quarantine is a local YAML list – keep it under review in code review.\n- TTL support: run `flakewall quarantine-tick` in CI to decrement TTL and auto-remove expired entries.\n- Flake scoring is minimal (requires only JUnit XML). It doesn’t depend on long CI history.\n- Jest retry selection uses `-t \u003cname\u003e`; include file path in test ID if multiple names collide.\n\n## Contributing\nSee `CONTRIBUTING.md` for dev setup and releasing.\n\n## License\nMIT – see `LICENSE`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcicatriiz%2Fflakewall","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcicatriiz%2Fflakewall","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcicatriiz%2Fflakewall/lists"}