https://github.com/vitalets/sheeva

Concurrent Automation Test Runner
https://github.com/vitalets/sheeva

test-automation test-runner testing

Last synced: 3 months ago
JSON representation

Concurrent Automation Test Runner

• Host: GitHub
• URL: https://github.com/vitalets/sheeva
• Owner: vitalets
• Created: 2016-11-14T18:28:20.000Z (almost 9 years ago)
• Default Branch: master
• Last Pushed: 2017-08-09T09:17:34.000Z (about 8 years ago)
• Last Synced: 2025-04-04T18:48:11.033Z (6 months ago)
• Topics: test-automation, test-runner, testing
• Language: JavaScript
• Homepage:
• Size: 531 KB
• Stars: 3
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          


  



Sheeva



  

  

  





  Concurrent Automation Test Runner



***

## Intro

Sheeva is a test-runner with focus on optimal test execution on parallel targets with limited concurrency.

Targets are browsers, mobile phones, tablets and any other configurable entities.

## Features  

- **Parallelization** 

  - per browser 

  - per mobile device

  - per suite 

  - per test

- **Annotations** 

  - ignore/run test for particular target  

  - automatically re-try with increasing timeout 

  - run subset of tests by tag

  - attach custom data to test

- **Syntax**

  - BDD compatible with [Mocha](https://mochajs.org), [Jasmine]() and [Jest]()

  - Nested groups allowed

- **Integrations**

  - [Selenium-webdriver](https://www.npmjs.com/package/selenium-webdriver)

  - [Webdriverio](http://webdriver.io)

  - [Sauce Labs](https://saucelabs.com)

- **Can run in**

  - Node.JS

  - Browser

## Contents

 * [Demo](#demo)

 * [Install](#)

 * [Getting started](#)

 * [Configuration](#)

 * [API](#)

   * [Tests](#)

   * [Annotations](#)

 * [Examples](#)

 * [Why special runner?](#whyspecialrunner)

## Demo

## Why special runner?

Automation tests are different from unit tests.

Their requirements to runner are:

* **Parallelization per environment**  

   Run the same tests on different browsers, smartphones and platforms.

* **Parallelization per test**   

   Split suites and run each test/sub-suite in parallel.

   

* **Conditional skip/only**  

   Skip or run particular test in particular environment.

   

* **Concurrency limit**  

   Run on pre-defined number of concurrent sessions.

   

* **Nested groups**   

   Create nested test groups with common before/after hooks to reduce boilerplate code.

* **Smart tests splitting**  

   Smart algorithm for splitting tests between parallel sessions to have minimal execution time.

   

* **Progressive retries**  

   Give test a chance to re-run if it tightly depends on extra conditions (e.g. network state).

   

* **Tags**   

   Group and run tests by tags (e.g. *smoke*, *normal*, *full*) 

      

Sheeva tries to fulfill all these requirements.

## Errors handling

There are several types of errors that may occur while running tests:  

1. **Error in test**  

  It can be assertion error or error in test code itself. In that case all `afterEach` hooks are called,

  error is reported via `TEST_END` event and runner goes to the next test.

 

2. **Error in beforeEach hook**  

   Runner will not call test itself but will call all needed `afterEach` 

   hooks for proper cleanup. Then runner goes to the end of error suite, calls all needed `after` hooks

   and starts next suite.

   

2. **Error in afterEach hook**  

   Runner will anyway call all needed `afterEach` hooks for proper cleanup. 

   Then runner goes to the end of error suite, calls all needed `after` hooks

   and starts next suite. 

 

3. **Error in before hook**  

   Runner will not call any tests of that suite. Instead it goes to the end of that suite, 

   calls all needed `after` hooks and starts next suite.

3. **Error in after hook**  

   Runner will anyway call all needed `after` hooks for proper cleanup and start next suite.

4. **Error in runner**  

   This may be internal error in Sheeva itself or in configuraiton methods such as `config.startSession`. 

   In that case runner terminates immediately and `sheeva.run()` rejects with that error.

  

*Note 1:* reporter is responsible for displaying all errors except runner error.  

*Note 2:* if `config.breakOnError` is enabled then runner will terminate on any error.  

*Note 3:* there can be several errors at once, e.g. error in `before` hook can cause error in `after` hook.  

## License

MIT @ [Vitaliy Potapov](https://github.com/vitalets)