https://github.com/treebeardtech/get-flakes
A CI tool that alerts teams when flaky tests are slowing down development
https://github.com/treebeardtech/get-flakes
Last synced: 10 months ago
JSON representation
A CI tool that alerts teams when flaky tests are slowing down development
- Host: GitHub
- URL: https://github.com/treebeardtech/get-flakes
- Owner: treebeardtech
- License: apache-2.0
- Created: 2021-02-22T11:52:06.000Z (almost 5 years ago)
- Default Branch: main
- Last Pushed: 2021-06-11T13:17:00.000Z (over 4 years ago)
- Last Synced: 2025-04-17T14:19:01.741Z (10 months ago)
- Language: Python
- Homepage:
- Size: 1.53 MB
- Stars: 9
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
# get-flakes 🍦
**under construction, do not attempt to use.**
**A CI tool that finds flaky GitHub Actions checks on recent pull requests**
---
get-flakes lets you automatically alert the team when flaky tests are hurting productivity.
It does this by querying your GitHub repo's recent GitHub Actions. When an action has been restarted and gives a different result, get-flakes marks this as flaky.
Excessive test retrying slows delivery, wastes resources, and hides real bugs.
## Quickstart
Create this file in your repo so you can manually trigger get-flakes and have it run periodically.
If more than 5% of commits in pull requests exhibit flakiness then the check fails.
```yaml
# .github/workflows/get-flakes.yml
on:
push: # remove this once you are setup
workflow_dispatch:
inputs:
tags:
description: 'Run get-flakes'
schedule:
- cron: 0 0 * * */7
jobs:
get-flakes:
runs-on: ubuntu-latest
steps:
- uses: actions/setup-python@v2
- run: pip install get-flakes
- run: |
get-flakes \
--days 8 \
--alarm-threshold-percentage=5
--token='${{ secrets.GITHUB_TOKEN }}'
```
This will write a markdown report which can be published back to GitHub:
## Use outside of GitHub Actions
Get-Flakes currently only works for GitHub Action results for GitHub repos. You can run this locally though to try it out.
```sh
pip install get-flakes
```
```sh
get-flakes report \
--days=9 \
--github_token=<...> \
--repo=treebeardtech/get-flakes \
--check_sha=<...> # The commit where the report goes, default is HEAD commit
```
## Contribute to this Design
This project is not yet ready for consumption but is available for ideation/feedback.
Feature requests and feedback welcome via email and issues.