Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/jparkerweb/bivariate

An opinionated interface for writing, running, and saving BackstopJS tests
https://github.com/jparkerweb/bivariate

backstopjs browser-automation css javascript layout regression-testing visualtesting

Last synced: 3 days ago
JSON representation

An opinionated interface for writing, running, and saving BackstopJS tests

Awesome Lists containing this project

README 

        
# ![Bivariate](bivariate.png)



***An opinionated interface for writing, running, and saving BackstopJS tests***

  

## Goal



**Bivariate**'s goal is to allow for an approachable Visual Regression Testing suite that can be organized to accommodate small and large projects without overwhelming complexity.

  

![Bivariate App](./documentation/menu.gif)  



This goal is achieved by enforcing an opinionated grouping structure, providing a method to easily write tests via manageable object files, as well as allowing for all of BackstopJS's commands to be run from an interface.

  

## Installation



**Bivariate** runs in [Node](https://nodejs.org).



* Install [NodeJS](https://nodejs.org)



* Install the Latest version of Bivariate via NPM.  

It is *recommended* to install Bivariate *globally*, but it can run locally if required:  

  

  global install (*recommended*):  

  `npm install bivariate -g`



  local install:  

  `npm install bivariate`



* Ensure you have version 59 or greater of [Chrome](https://www.google.com/chrome/browser/) installed.

  Bivariate utilizes headless Chrome which started shipping in Chrome v59



* From your project directory, run Bivariate:  

  if installed globally:  

  `bivariate`



  if only installed locally:  

  `npx bivariate`

  

* Generate `bivariate_data`:  

  If Bivariate doesn't detect any existing Bivarte tests it will ask you would like to generate the starting configuration files.  



  ![Bivariate App](./documentation/first-run.gif)  

  

## Folder Structure



*All tests, scripts, and configuration files are stored in the `bivariate_data` parent folder.*



`bivariate_data`  

|  

+---- `test_scripts` holds user defined configuration and tests used to instruct BackstopJS  

|  

+---- `engine_scripts` holds user defined Puppeteer scripts for interacting with the Chrome DOM before saving a screen shot  

|  

+---- `bitmaps_reference_archive` holds archived *references* that can be restored and tested against  

  

### test_scripts



Out of the box, BackstopJS gets all of its config and test data from a single JSON file, which isn't very maintainable over time.  Luckily, **Bivariate** takes advantage of Node's module system to break this all apart and just return what is needed (*a simple array of objects*).



#### Configuration files



All configuration files are prefixed with a double underscore: \_\_  



##### \_\_config-baseURLs.js



holds the *base* URLs for all References and Tests to be run.  



```js

...

  // do not use a trailing slash in the base URLs

  theURLS.baseURL = "http://your-base-url";

  theURLS.baseRefURL = "http://your-base-reference-url";

...

```



##### \_\_config-common.js  



set of *common* config values (rendering engine, ports, etc.) that shouldn't need to be adjusted in most cases.



##### \_\_config-viewports.js  



 configure any number of *viewports* to test against (this can include any number of defined screen resolutions).

  

#### Individual Tests  



All individual tests are prefixed with a single underscore: \_  



Use the example tests as a template to create your own.  Tests are easy to setup and for the most part only require you to fill out the value for a few variables:



* label

* route

* selectors



`_example-test--home.js` :



````js

// -------------------

// - test definition -

// -------------------



// * tests should be saved as: '_test-name.js'

//   if you have a lot of tests you can store

//   related tests in named subdirectories

//   for better organization



var label = "Example Test - Home Page"

var route = "/index.html"

var readySelector = ""

var hideSelectors = []

var removeSelectors = []

var selectors = [ "document", "h1", ".hero", ".nav", ".body-content" ]

let hoverSelector = null

let hoverSelectors = []

let clickSelector = null

let clickSelectors = []

let postInteractionWait = 100

let scrollToSelector = null

let delay = 300

var onBeforeScript = 'onBefore-Example.js'

var onReadyScript = 'onReady-Example.js'

let viewports = []



// ---------

// - label -

// ---------

// [required]

// used on reports and screen shot file names (should be unique between tests)



// ---------

// - route -

// ---------

// [required]

// the route for this test (start with a "/")



// -----------------

// - readySelector -

// -----------------

// Selector to look for before continuing



// -----------------

// - hideSelectors -

// -----------------

// hide elements from view by changing its "visibility" to "hidden"



// -------------------

// - removeSelectors -

// -------------------

// remove elements from the DOM before screen capture



// -------------

// - selectors - (array or strings)

// -------------

// selectors for elements to be "captured" (CSS selector syntax)



// -----------------

// - hoverSelector -

// -----------------

// Move the pointer over the specified DOM element prior to screen shot.



// ------------------

// - hoverSelectors - (array)

// ------------------

// *Puppeteer only* takes array of selectors -- simulates multiple

// sequential hover interactions.



// -----------------

// - clickSelector -

// -----------------

// Click the specified DOM element prior to screen shot.



// ------------------

// - clickSelectors - (array)

// ------------------

// *Puppeteer only* takes array of selectors -- simulates

// multiple sequential click interactions.



// -----------------------

// - postInteractionWait -

// -----------------------

// Wait for a selector after interacting with hoverSelector or clickSelector

// (optionally accepts wait time in ms. Idea for use with a click or hover

// element transition. available with default onReadyScript)



// --------------------

// - scrollToSelector -

// --------------------

// Scrolls the specified DOM element into view prior to screen shot

// (available with default onReadyScript)



// ---------

// - delay -

// ---------

// Wait for x milliseconds



// ------------------

// - onBeforeScript -

// ------------------

// Runs before each scenario

// use for setting cookies or other env state

// (.js suffix is optional / looks for file in "engine_scripts" dir)



// -----------------

// - onReadyScript -

// -----------------

// Runs after onReady event on all scenarios

// use for simulating interactions

// (.js suffix is optional / looks for file in "engine_scripts" dir)



// -------------

// - viewports -

// -------------

// overwrite array of viewports

// example:

// let viewports = [

//    {

//        name: "huge-vertical-space",

//        width: 1920,

//        height: 4500

//    }

// ]

  



// -------------------------------------------------------------------

// - advanced options can be overwritten in the options object below -

// -------------------------------------------------------------------

module.exports = function(baseURLs) {

	var url = (baseURLs.baseURL + route)

	var referenceUrl = baseURLs.baseRefURL === null ? null : (baseURLs.baseRefURL + route)

	var options = {

		"label": label,									// [required] Tag saved with your reference images

		"url": url,										// [required] Tag saved with your reference images

		"referenceUrl": referenceUrl,					// Specify a different state or environment when creating reference.

		"readySelector": readySelector,					// Wait until this selector exists before continuing.

		"hideSelectors": hideSelectors,					// Array of selectors set to visibility: hidden

		"removeSelectors": removeSelectors,				// Array of selectors set to display: none

		"selectors": selectors,							// Array of selectors to capture. Defaults to document if omitted. Use "viewport" to capture the viewport size.

		"selectorExpansion": true,						// If you want BackstopJS to find and take screenshots of all matching selector instances then set to true.

		"readyEvent": null,								// Wait until this string has been logged to the console.

		"hoverSelector": hoverSelector,					// Move the pointer over the specified DOM element prior to screen shot.

		"hoverSelectors": hoverSelectors,				// *Puppeteer only* takes array of selectors -- simulates multiple sequential hover interactions.

		"clickSelector": clickSelector,					// Click the specified DOM element prior to screen shot.

		"clickSelectors": clickSelectors,				// *Puppeteer only* takes array of selectors -- simulates multiple sequential click interactions.

		"postInteractionWait": postInteractionWait,		// Wait for a selector after interacting with hoverSelector or clickSelector (optionally accepts wait time in ms. Idea for use with a click or hover element transition. available with default onReadyScript)

		"scrollToSelector": scrollToSelector,			// Scrolls the specified DOM element into view prior to screen shot (available with default onReadyScript)

		"delay": delay,									// Wait for x milliseconds

		"misMatchThreshold": 0.1,						// Percentage of different pixels allowed to pass test

		"onBeforeScript": onBeforeScript,				// Used to set up browser state e.g. cookies.

		"onReadyScript": onReadyScript,					// After the above conditions are met -- use this script to modify UI state prior to screen shots e.g. hovers, clicks etc.

		"requireSameDimensions": false,					// If set to true -- any change in selector size will trigger a test failure.

		"viewports": viewports							// An array of screen size objects your DOM will be tested against. This configuration will override the viewports property assigned at the config root.

	}



	if (baseURLs.baseRefURL === null) {

		delete options.referenceUrl

	}



	return options

}

````



#### Test Groups  



Bivariate presents and runs tests using a grouping concept.  A `test group` is a collection of `tests` that are run together.  A test group is a .js file that does not start with any underscores.  



Use the provide file `example-test-group.js` as a template for your own.  Note that all that is required is to fill in the `Scenarios` section to include which tests you want run.



`example-test-group.js` :



````js

// ----------------

// -- Test Group --

// ----------------



let mixIn = require("./../libs/mout-mixin/mixIn");

let testGroup = __filename.slice(__dirname.length + 1, -3);

let configCommon = require('./__config-common')(testGroup);

let baseURLs = require("./__config-baseURLs");



module.exports = mixIn(

    {

        // ---------------

        // -- Scenarios --

        // ---------------

        "scenarios": [

            require('./_example-site--home')(baseURLs),

            require('./_example-site--paints')(baseURLs)

        ],

    },

        configCommon

);

````

  

### engine_scripts

  

engine scripts are used to interact with your web pages using the `before` and `on ready` events.  Each test you create has an optional parameter of `onBeforeScript` & `onReadyScript`.  These can simply point to script files in the 'engine_scripts' directory.  The two example scripts found in the `engine_scripts` directory should be self explanatory (`onBefore-Example.js` & `onReady-Example.js`).  In addition you can refer to the [Puppeteer Docs](https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md) for more advanced examples.

  

### bitmaps_reference_archive

  

The `bitmaps_reference_archive` folder holds archived `references` which can be created, archived, and restored using the **Bivariate** app.

  

----



## App



Example run of the test scripts generated by Bivariate:  



![Bivariate App Example Run](./documentation/example-run.gif)  

  

For more info, reference `example-site` README for a walk through example of **Bivariate** in action.  



### [Example Site Docs](./example-site/README.md)



Detailed docs for what the `reference`, `test`, and `approve` commands do under the hood can be found on the BackstopJS Github page: [BackstopJS](https://github.com/garris/BackstopJS).