https://github.com/deploysentinel/cypress-quarantine
Cypress plugin that helps skip/quarantine flaky tests dynamically
https://github.com/deploysentinel/cypress-quarantine
cypress flaky-tests plugin testing
Last synced: about 1 month ago
JSON representation
Cypress plugin that helps skip/quarantine flaky tests dynamically
- Host: GitHub
- URL: https://github.com/deploysentinel/cypress-quarantine
- Owner: DeploySentinel
- Created: 2023-02-12T00:53:22.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-02-15T07:53:48.000Z (over 2 years ago)
- Last Synced: 2024-11-03T22:38:47.489Z (7 months ago)
- Topics: cypress, flaky-tests, plugin, testing
- Language: TypeScript
- Homepage: https://www.deploysentinel.com/
- Size: 24.4 MB
- Stars: 7
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
Awesome Lists containing this project
README
# DeploySentinel Cypress Quarantine
DeploySentinel Cypress Quarantine is a plugin for Cypress that helps developers quarantine tests dynamically.
With this plugin, you can dynamically skip tests that are unstable, flaky, or not ready to run based off an API call made at test time.
## Installation
To install the Cypress plugin, run:
```sh
npm install -D @deploysentinel/cypress-quarantine
```
## Usage
### Add Support File
In your Cypress project's **support** file (ex. `cypress/support/index.js`) add:
```js
// ❗ Must be declared at the top of the file ❗
import "@deploysentinel/cypress-quarantine/support";
...
import "./commands";
```
### Add Plugin
In your Cypress project's **plugin** file (ex. `cypress.config.ts`) add:
```ts
import { defineConfig } from 'cypress';
import cyQuarantine from '@deploysentinel/cypress-quarantine/plugin';
export default defineConfig({
...
e2e: {
setupNodeEvents(on, config) {
cyQuarantine(on, config, {
// your custom API endpoint that returns which tests to skip per spec file (required)
apiUrl: 'http://localhost:8000/ci/quarantine-tests',
meta: {
testFramework: 'cypress',
// or any custom static metadata (optional)
},
// specify the method that generates unique test id based on titles
// ex: titles: ['describe A', 'test case B'] -> 'describe A > test case B' (stored in db)
// default to only test case name (leaf node); 'test case B' in this case
getTestId: (titles: string[]) => titles.join(' > '),
topLevelKey: 'xxx', // (optional)
});
return config;
},
},
});
```
## How Does it Work ?
When you use the plugin, it will make an `POST` API request to your server to retrieve quarantined tests for a specific file.
The resulting API response should look like
```
{
'describe A > test case B': true,
'describe A > test case C': false,
...
}
```
The key is a unique test ID derived from `getTestId`.
And the value indicates whether the test case should be skipped (quarantined).
### Top Level API Key
In case the API response has top level key attached, for example:
```
{
data: {
'describe A > test case B': true,
'describe A > test case C': false,
...
}
}
```
You can set `topLevelKey` field in config to `data` in this case.
### Custom Metadata
The API request body includes a few default custom fields by default.
You can utilize this information to develop a test quarantine algorithm.
```ts
{
cypressVesion: string;
// test environment variables
envs: Record;
// git information
commitInfo: {
authorEmail: string | null;
authorName: string | null;
branch: string | null;
defaultBranch: string | null;
message: string | null;
remoteBranch: string | null;
remoteOrigin: string | null;
sha: string | null;
timestamp: number | null;
metadata: unknown | null;
},
}
```