{"id":48290907,"url":"https://github.com/mustafaautomation/flaky-test-detective","last_synced_at":"2026-04-04T23:05:51.360Z","repository":{"id":343053543,"uuid":"1165016663","full_name":"mustafaautomation/flaky-test-detective","owner":"mustafaautomation","description":"Detect, track, and quarantine flaky tests with pattern analysis and trend reporting","archived":false,"fork":false,"pushed_at":"2026-04-04T16:08:21.000Z","size":144,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-04T18:21:51.743Z","etag":null,"topics":["ci-cd","flaky-tests","junit","nodejs","pattern-detection","playwright","qa","quarantine","test-stability","testing","trend-analysis","typescript"],"latest_commit_sha":null,"homepage":"https://quvantic.com","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mustafaautomation.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-23T18:28:01.000Z","updated_at":"2026-04-04T16:08:26.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/mustafaautomation/flaky-test-detective","commit_stats":null,"previous_names":["mustafaautomation/flaky-test-detective"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/mustafaautomation/flaky-test-detective","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mustafaautomation%2Fflaky-test-detective","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mustafaautomation%2Fflaky-test-detective/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mustafaautomation%2Fflaky-test-detective/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mustafaautomation%2Fflaky-test-detective/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mustafaautomation","download_url":"https://codeload.github.com/mustafaautomation/flaky-test-detective/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mustafaautomation%2Fflaky-test-detective/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31418288,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-04T20:09:54.854Z","status":"ssl_error","status_checked_at":"2026-04-04T20:09:44.350Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ci-cd","flaky-tests","junit","nodejs","pattern-detection","playwright","qa","quarantine","test-stability","testing","trend-analysis","typescript"],"created_at":"2026-04-04T23:05:50.817Z","updated_at":"2026-04-04T23:05:51.350Z","avatar_url":"https://github.com/mustafaautomation.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Flaky Test Detective\n\n[![CI](https://github.com/mustafaautomation/flaky-test-detective/actions/workflows/ci.yml/badge.svg)](https://github.com/mustafaautomation/flaky-test-detective/actions)\n[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)\n[![Node.js](https://img.shields.io/badge/Node.js-18+-339933.svg?logo=node.js\u0026logoColor=white)](https://nodejs.org)\n[![TypeScript](https://img.shields.io/badge/TypeScript-strict-3178c6.svg?logo=typescript\u0026logoColor=white)](https://www.typescriptlang.org)\n[![Docker](https://img.shields.io/badge/Docker-Ready-2496ED.svg?logo=docker\u0026logoColor=white)](Dockerfile)\n\nDetect, track, and quarantine flaky tests with pattern analysis and trend reporting.\n\nQA teams lose 6-8 hours per week on flaky tests. Flaky Test Detective analyzes your test results across multiple runs to identify which tests are flaky, why they're flaky, and automatically quarantines the worst offenders.\n\n## Features\n\n- **Multi-framework support** — Playwright JSON and JUnit XML parsers with auto-detection\n- **Flakiness scoring** — `1 - (max(passCount, failCount) / totalRuns)` across historical runs\n- **Pattern detection** — Timing variance, network errors, and order-dependent failures\n- **Quarantine mode** — Auto-tag tests exceeding threshold, output as `--grep-invert` pattern\n- **Trend tracking** — SQLite-backed historical data with trend analysis\n- **Multiple reporters** — Console (colorized), JSON, HTML (dark dashboard with SVG charts), GitHub (PR comment markdown)\n\n## Quick Start\n\n```bash\nnpm install flaky-test-detective\n\n# Initialize config\nnpx flaky-detective init\n\n# Analyze test results\nnpx flaky-detective analyze ./test-results/results.json\n\n# Analyze multiple runs for better detection\nnpx flaky-detective analyze ./results/run-*.json\n\n# Generate HTML report\nnpx flaky-detective analyze ./results/*.json --reporter html,console\n\n# Get quarantine pattern\nnpx flaky-detective quarantine\n```\n\n## CLI Commands\n\n### `analyze \u003cfiles...\u003e`\n\nIngest test result files and compute flakiness.\n\n```bash\nnpx flaky-detective analyze results.json [results2.json...] [options]\n\nOptions:\n  -c, --config \u003cpath\u003e     Path to config file\n  -r, --reporter \u003ctype\u003e   console, json, html, github (comma-separated)\n  -o, --output \u003cdir\u003e      Output directory (default: .flaky-detective/reports)\n  -v, --verbose           Enable debug logging\n```\n\n### `report`\n\nGenerate report from previously stored data.\n\n### `quarantine`\n\nOutput grep pattern for quarantined tests. Use with Playwright:\n\n```bash\nnpx playwright test --grep-invert \"$(npx flaky-detective quarantine)\"\n```\n\n### `trends`\n\nShow flakiness trends over time.\n\n### `init`\n\nCreate a default configuration file.\n\n## Configuration\n\n```json\n{\n  \"storage\": {\n    \"type\": \"sqlite\",\n    \"path\": \".flaky-detective/data.db\"\n  },\n  \"thresholds\": {\n    \"flakiness\": 0.1,\n    \"quarantine\": 0.3,\n    \"minRuns\": 3,\n    \"timingCv\": 0.5\n  },\n  \"parsers\": [\"auto\"],\n  \"reporters\": [{ \"type\": \"console\" }],\n  \"outputDir\": \".flaky-detective/reports\"\n}\n```\n\n### Thresholds\n\n| Threshold | Default | Description |\n|-----------|---------|-------------|\n| `flakiness` | 0.1 | Minimum score to flag as flaky |\n| `quarantine` | 0.3 | Score threshold for auto-quarantine |\n| `minRuns` | 3 | Minimum runs before analysis |\n| `timingCv` | 0.5 | Coefficient of variation for timing detection |\n\n## How It Works\n\n### Flakiness Score\n\n```\nscore = 1 - (max(passCount, failCount) / totalRuns)\n```\n\n- Score of 0 = perfectly stable (all pass or all fail)\n- Score of 0.5 = maximum flakiness (equal pass/fail)\n\n### Pattern Detection\n\n| Pattern | Detection Method |\n|---------|-----------------|\n| **Timing** | High coefficient of variation in test durations |\n| **Network** | Error messages matching network keywords (ECONNREFUSED, ETIMEDOUT, etc.) |\n| **Ordering** | Position variance across runs suggesting order dependency |\n\n### Storage\n\n- **SQLite** (default) — Persistent storage with efficient trend queries\n- **JSON** — File-based fallback, no native dependencies\n\n## CI Integration\n\n### GitHub Actions\n\n```yaml\n- name: Run tests\n  run: npx playwright test --reporter=json --output-file=results.json\n  continue-on-error: true\n\n- name: Analyze flakiness\n  run: npx flaky-detective analyze results.json --reporter github\n```\n\n## Programmatic Usage\n\n```typescript\nimport { Analyzer, DEFAULT_CONFIG } from 'flaky-test-detective';\n\nconst analyzer = new Analyzer({\n  ...DEFAULT_CONFIG,\n  storage: { type: 'json', path: './flaky-data.json' },\n});\n\nanalyzer.init();\nconst report = analyzer.analyze(['./results/run1.json', './results/run2.json']);\n\nconsole.log(`Flaky tests: ${report.flakyTests}`);\nconsole.log(`Quarantined: ${report.quarantinedTests}`);\n\nanalyzer.close();\n```\n\n## License\n\nMIT\n\n---\n\nBuilt by [Quvantic](https://quvantic.com)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmustafaautomation%2Fflaky-test-detective","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmustafaautomation%2Fflaky-test-detective","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmustafaautomation%2Fflaky-test-detective/lists"}