Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/codelenny/ava-test-style

:book: AVA Test Style Guide
https://github.com/codelenny/ava-test-style

Last synced: 4 months ago
JSON representation

:book: AVA Test Style Guide

Awesome Lists containing this project

README 

          
= Ryan's AVA Testing Organization Style

Ryan Leonard

:ava: AVA

:ava-link: https://github.com/avajs/ava

:aval: link:{ava-link}[{ava}]

:ava-assertions: link:https://github.com/avajs/ava#assertions[assertions]

:ava-macros: test macros

:ava-macrosl: link:https://github.com/avajs/ava#test-macros[{ava-macros}]

:ava-plan: assertion planning

:ava-planl: link:https://github.com/avajs/ava#assertion-planning[{ava-plan}]

:mocha: Mocha

:mochal: link:https://mochajs.org/[{mocha}]

:chai: Chai

:chail: link:http://chaijs.com/[{chai}]

:node: Node.js

:jsverify: JSVerify

:jsverifyl: link:https://github.com/jsverify/jsverify[{jsverify}]

:express: Express

:expressl: link:http://expressjs.com/[{express}]

:seleniumdrive: Selenium WebDriver

:seleniumdrivel: link:http://www.seleniumhq.org/projects/webdriver/[{seleniumdrive}]

:webdriver: WebDriverIO

:webdriverl: link:http://webdriver.io/[{webdriver}]

:babel: Babel

:babell: link:https://babeljs.io/[{babel}]

:supertest: SuperTest

:supertestl: link:https://github.com/visionmedia/supertest[{supertest}]

:status-code: HTTP status code

:status-codel: link:https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html[{status-code}]

:guide-link-title: Ryan's AVA Test Style

:guide-link: https://github.com/CodeLenny/ava-test-style

:guide-badge: https://img.shields.io/badge/AVA%20Test%20Style-@CodeLenny-blue.svg

:do: image:https://img.shields.io/badge/-_DO_-brightgreen.svg[]

:dotitle: DO:

:dont: image:https://img.shields.io/badge/-DON'T-red.svg[]

:donttitle: DON'T:

:sectanchors:

:sectlinks:

:toc: preamble



{aval} is a powerful testing platform that I've been using in most of my repositories instead of {mochal}.

After trying out {ava} in many repositories, I've started to settle into a standard style that works well for creating

clean, understandable, and simple testing files.



This document attempts to record my current style for writing {ava} tests.

Some of my style is intuitive to {ava},

like using the built-in {ava-assertions} instead of external assertion libraries like {chail},

but other stylistic decisions are my own.



This guide is intended to serve as a reference for libraries that follow my style choices.

If you have differing style choices to mine, feel free to fork this style guide and tweak to your liking.

You can also submit issues or pull requests and discuss pros and cons to different choices, and I may tweak my style if

convinced that another way is better.



.Embedable Badge image:{guide-badge}[]

[source,md,subs="attributes"]

----

[![AVA Test Style Guide]({guide-badge})]({guide-link})

----



.Text Reference

[source,md,subs="attributes"]

----

This module is tested using [{ava}]({ava-link}).

Tests should conform to [{guide-link-title}]({guide-link}).

----



== Highlights



- Tests are code, and should conform to lint rules.

- The file-system reflects the organization of tests, and follows a <>.

- Unit tests are separated from integration tests.

- Tests are designed for other humans.

  Describe tests from the point of view of a user, without referencing implementation details.

- Use property-based testing (such as {jsverifyl}) to find edge-cases and write concise tests.

- Put all setup in `before`/`beforeEach` statements.

- Duplicating documentation should be avoided at all costs.

- Use built-in test organization messages over inline comments or `README` descriptions when possible.



== Top-Level Test Organization



Tests are categorized by scope, ranging from small unit tests to larger integration tests.

This guide attempts to provide clear distinctions between unit and integration tests, and has named a few other test

categories.



unit tests::

Independently testing a method, mocking all external dependencies.

HTTP unit test::

Simple usage or edge case testing of an HTTP method.

interface integration test::

Testing a method against a single instance of an external dependency.

All other resources should be mocked.

integration test::

A full-scale test involving almost no mocked components, that tests an entire flow from the user's perspective.

Run with AVA in parallel to unit tests, before packaging.

package integration test::

If the application is packaged (such as a webserver packaged in a Docker image),

package integration tests run on the fully packaged build

(such as using a Chrome instance to connect to the webserver running in Docker).



The test groups listed above are ordered from least to most expensive to run - starting a Docker image to test a smaller

component takes a lot longer than just running a short unit test with all external resources mocked.



Therefore, unit tests should be used for most purposes.

Unit tests should cover the basic usage of the method (give correct input, get correct response)

and cover how the method handles edge cases (given invalid input, asked for non-existent resource, etc.).



Unit tests should be validating the documented signature - expected parameters, calling other methods with the proper

signature, and returning the expected result.



All tests beyond unit tests should be minimal flows through the application.



For interface integration tests, when one class (`CallingClass`) calls another (`CalledClass`),

we've already checked that `CallingClass` calls `CalledClass` with the documented parameters,

and separately tested that `CalledClass` returns the correct input when given the correct parameters.



Therefore, the interface integration test is only testing that the two classes are in sync with each other, and don't

have any mistakes with basic usage.



Integration tests also test that already-tested components are wired together correctly, from a user's point of view.

Integration tests treat the entire application as a black box, and merely test input at one end compared to output at

the other.



Package integration tests should be even more minimal, just ensuring that the packaged application does start without

any errors (such as missing dependencies or differences between the testing and packaged environments),

and continues working with one or two basic flows through the application.



[%hardbreaks]

{do} Carefully organize testing files.

{do} Focus on unit tests.

{dont} Duplicate tests.



=== File-System Structure



Most of this documentation will assume a monolithic repository structure that includes several libraries packaged as NPM

packages.



We'll use an webserver that deals with users and projects.



----

repo/

  package.json

  web-server/

    package.json

    Dockerfile

    Server.js

    test/

    package-test/

  users/

    auth/

      AuthenticationMethod.js

      PasswordAuthentication.js

    package.json

    Users.js

    User.js

    test/

  projects/

    package.json

    Projects.js

    Project.js

    test/

----



`users`, and `projects` are each NPM libraries that are installed in `web-server`.



`web-server/Server.js` will setup an {expressl} webserver

with routes that use `Users` and `Projects` to store and retrieve data for clients.



`web-server` has a `package-test/` directory that contains tests that will run in browsers

(such as through {seleniumdrivel} or {webdriverl}) against `Server` running in a Docker instance.



All other tests will be located in the `test/` directory for each module.



=== Unit Test Organization



Unit tests should be stored in `/test///.js`.



For instance, tests that confirm `Users.getByID()` fetches users would be located in

`users/test/Users/getByID/fetches-users.js`.



If classes have unique names, collapse directories when testing.

For instance, tests for `users/auth/AuthenticationMethod.js` can be located in `users/test/AuthenticationMethod/...`.



If classes do not have unique names, you can use directories inside `test/` to keep tests seperate.

For instance, the above tests could also be located inside `users/test/auth/AuthenticationMethod/...`.



Try to collapse directories as much as possible.

Only use sub-directories in `test` if collapsing directories severely impacts understanding the test organization.



=== HTTP Unit Tests Organization



HTTP unit tests are an interesting mix - they should be isolated to a single "method", but you may need to access a

larger section of code to get the HTTP routing logic.



In general, HTTP routing logic should be basic wrappers around other functions.

For user registration, the logic might look like:



[source,js]

----

const Users = require("users/Users");

const express = require("express");

const bodyParser = require("body-parser");



class Server {

  constructor() {

    this.app = express();

    this.app.post("/register", bodyParser.json(), (req, res) => {

      const { email, password } = req.body;

      Users

        .register(email, password)

        .then(user => {

          req.redirect("/login");

        })

        .catch(err => {

          res.status(500);

          res.send("Internal Error");

        });

    });

  }

}

----



For this example, `Users.register()` should be already unit tested, so the HTTP logic just needs to attempt to submit a

form, and ensure that `Users.register()` is called with the correct information.



HTTP unit tests should be located in `/test/http///.js`.



The test referenced above should be located in `server/test/http/register/POST/pass-to-Users-reigster.js`.



=== Interface Integration Test Organization



Interface integration tests are very similar to unit tests, and are stored almost identically.

However, you should note what other modules are being used in the test.



Let's test `Project#getOwner()`, which calls `Users.getByID()`, which in turn accesses the database.



A unit test might be `projects/test/Project/getOwner/returns-user.js` should be run with `Users.getByID` mocked,

and confirm that `Users.getByID()` is called with the ID of the project's owner, and correctly returns the user that

`Users.getByID()` returns.



For interface tests, we will be testing that `Users.getByID` and `Project#getOwner` are correctly talking to each other.

`projects/test/Project/getOwner/relays-Users-getByID.js` would use un-mocked `Users` and `Project` method, but should

mock the contents of the database.



=== Integration Test Organization



Integration tests should test user flow through the application, with minimal mocking.



In general, integration tests should be located in `/test/integration//.js`.



For instance, a test confirming users can log in after registering would be located in

`users/test/integration/user-register-and-login/password-authentication.js`.



In general, integration tests should be confirming that the module works as a whole,

so integration tests can be lumped together.



However, integration tests that are isolated to a minor class that doesn't represent the rest of the module could be

located inside the test directory for that class - such as `users/test/AuthenticationMethod/integration/...`.



== What to Test



=== Unit Test Contents



Unit tests should be written for each method and function in the repository.



Include one test showing the basic usage:



.`users/test/Users/getByID/fetches-users.js`

[source,js]

----

const id = "dummy-user";

const name = "A Fake User";

// (pre-load database in `before()` hook)



test("finds user", t => {

  return Users

    .getByID(id)

    .then(user => {

      t.is(user.name, name);

    });

});

----



Next, include tests for missed branches, such as error handling,

adding additional tests for common uses that hit other branches.

It might help to examine line/branch/statement coverage reports.



Along with testing error cases, test that input validation correctly identifies improper input.



Finally, test any easily identifiable edge cases.

Look for bizarre input that should actually pass successfully, such as validating empty passwords,

as well as documenting any errors that might be thrown, such as validating the password for a missing user.



When responding to bug reports, normally new edge cases will be added as unit tests.



[%hardbreaks]

{do} Cover common cases, such as omitting optional arguments

{dont} Test mis-use of functions, like omitting required arguments or providing arguments in the wrong order.



=== HTTP Unit Test Contents



For HTTP routes that wrap unit-tested methods, testing can often be heavily mocked.



Generally, you should write one test that calls the underlying method,

and checks that the output is correctly returned.



Then evaluate any other branches.

Does the method respond with a {status-codel} and a custom body if an error is thrown in the test?



See any HTTP API documentation for the current module to see if there are other edge cases.



Try to mock all resources under the wrapped method.  For some tests, you may even be able to mock the method completely,

and only test the HTTP wrapper.



[%hardbreaks]

{do} Test HTTP parsing edge cases.

{dont} Test edge-cases already covered by the wrapped method.



=== Integration Interface Test Contents



When testing the interface between two modules, start with a basic usage that uses both modules.



[source,js]

----

const name = "my-sample-name";



test.beforeEach("start server", t => {

  t.context.server = new Server();

});



test.beforeEach("start client", t => {

  t.context.client = new Client();

});



test("client can login after registration", t => {

  const { client } = t.context;

  return client

    .register({ name })

    .then(() => client.login())

    .then(user => {

      t.is(user.name, name);

    });

});

----



{do} Mock all other resources besides the two modules being tested.



For most cases, integration interface tests are only ensuring:



- Both modules can start without issues

- The two modules don't conflict when running in the same environment (e.g. reserving the same port)

- A few methods are able to communicate without issues



[%hardbreaks]

{do} Test interface edge-cases, like handling error {status-codel}.

{do} Test flows that include communication between the modules.

{do} Test sending weird data between the modules, such as non-URL-safe parameters that will be used as the URL.

{dont} Test methods that don't communicate between modules.



In most cases, covering every possible communication isn't useful.

Full integration tests will cover most of the useful communication,

and unit tests will already be covering each method and making sure that each module follows the API spec.



Having duplicate assertions as unit tests for each module as well as on the interface between the two modules

will likely slow future development, as both assertions will have to be kept up to date with the code,

and each module might interface with multiple modules, providing a large number of interfaces to check.



However, fully testing the communication between two modules might make sense in two scenarios.

Mission-critical components may need the reliability,

and integration interface tests can ensure lasting stability for components not yet in production

and not tested in full integration tests.



=== Integration Test Contents



Integration tests should evaluate the resulting application, script, or server as a whole, with limited mocking.



However, the tests should still be run in {ava},

so you can use tools like {supertestl} to start servers without reserving network ports.



[%hardbreaks]

{do} Write integration tests from a user's perspective.

{do} Abstract setup of integration tests to focus test files on user-driven features.

//{dont} Think of internal edge-cases that the user wouldn't know about.



Write as many integration tests as you need to test the different scenarios that a user might encounter.



==== JSVerify for Integration Tests



You could setup {jsverifyl} to generate different flows that a user might take through your program,

to find different edge cases.



.{jsverify} Integration Test Example

[source,js]

----

const test = require("ava");

const jsc = require("jsverify");

jsc.ava = require("ava-verify");



// Emulates a user flow through the program:

const UserEnvironment = require("test/helpers/user-flow/UserEnvironment");



// Builds a random flow of user actions:

const UserFlow = require("test/helpers/user-flow/UserFlow");



// Builds register/login actions that are bundled:

const UserRegisterLogin = require("test/helpers/user-flow/UserRegisterLogin");



test.beforeEach("start new user environment", t => {

  t.context.env = new UserEnvironment();

});



jsc.ava({

  suite: "login after registration",

}, [

  UserFlow.build(),

  UserRegisterLogin.build(),

], (t, state, [ register, login ]) => {

  t.plan(3);

  const { env } = t.context.env;

  return env

    .all(state) // Get into a random user state (with or without registering or logging in)

    .then(env => env.do(register)) // then execute user registration

    .then(env => env.do(login)) // then execute user login using credentials from registration

    .then(env => {

      // make assertions about the result from user login

      t.is(env.code, 200, "should result in 200 status code");

      t.is(env.url, "/", "should navigate to homepage after login");

      t.true(env.headers.LOGGED_IN, "should have 'LOGGED_IN' header");

    });

});

----



=== Package Integration Test Contents



The package integration tests should make sure that the fully packaged application:



- Starts

- Does not depend on any development assets that existed during unit testing

- Has connections to any needed resources (such as those mocked during standard integration tests)



Standard integration tests should be preferred over package integration tests,

as the extra development assets loaded into the environment can make tests

more efficient to process, more succinct, and more debuggable when something goes wrong.



[%hardbreaks]

{do} Execute tests in a clean environment

{dont} Have any testing assets in the packaged environment (like `devDependencies` for {node} applications)



[%hardbreaks]

{do} Test a standard user flow through the application.

{do} Add edge cases to cover environmental changes from the standard integration tests.

{dont} Test user flows that can be covered as standard integration tests.



For a web-server that gets packaged into a Docker container, you could spin up the server in the container,

then use tools like {webdriverl} to point browser instances at the server.



// TODO: Include psudocode for package integration tests.



== Test File Structure



All test files should follow the same structure:



[source,js]

----

// require/imports



// before/after logic



// macros



// test contents

----



=== Requires/Imports



Unless you have good reason for it, put all the `require()` or `import` statements at the top of the file.



AVA currently comes shipped with {babell}, so you can use `import` statements with the current version of Node.



If your internal code and documentation are using `import`, you probably should use `import` statements in your testing

files.



For all other use cases, we recommend using `require()` while `import` isn't natively supported in Node to reduce the

amount of transcompilation.  Either way, use a consistent syntax across your repository.



Roughly order libraries from built-in to local source files.



[source,js]

----

// Libraries needed to modify subsequent imports:

const Promise = require("bluebird");

// Node built-in libraries:

const fs = Promise.promisifyAll(require("fs"));

const path = require("path");

// External libraries:

const reduce = require("lodash.reduce");

const express = require("express");

const request = require("supertest");

// Test setup:

const setup = {

  rethink: require("ava-rethinkdb"),

};



// Source files:

const Users = require("Users");

----



=== Before/After Logic



All test setup and teardown should be done in `before`/`after` blocks.

Setup and teardown methods can come from remote libraries or from local files.



[source,js]

----

// Remote libraries

test.before(setup.rethink.init());

test.after.always(setup.rethink.cleanup);



// Internal

test.beforeEach("create Users instance", t => {

  t.context.users = new Users();

});

----



{do} Always name `before`/`after` blocks.



.{dotitle} Break Tasks into Multiple Statements

[source,js]

----

test.beforeEach("create user", t => {

  t.context.user = new User("Bob");

});



test.beforeEach("add user to Users", t => {

  t.context.users.add(t.context.user);

});

----



.{dotitle} Put Setup inside `beforeEach`

[source,js]

----

test.beforeEach("create server", t => {

  t.context.app = express();

  app.use(myRouter);

});



test("runs", t => {

  return request(t.context.app)

    .get("/")

    .then(res => t.is(res.body, "Hello World"));

});

----



.{donttitle} Include setup inside `test`

[source,js]

----

test("runs", t => {

  let app = express();

  app.use(myRouter);

  request(app);

  return request(app)

    .get("/")

    .then(res => t.is(res.body, "Hello World"));

});

----



=== Test Macros



{do} Use {ava-macrosl}!



Use macros for any repeated code.



.Basic Macro Usage

[source,js]

----

const add = require("add");



function adds(t, a, b, c) {

  test.is(add(a, b), c);

}

adds.title = (title, a, b, c) => `add(${a}, ${b}) === ${c}`;



test(adds, 1, 2, 3);

test(adds, 10, 100, 110);

----



For the most part, test macros should be simple.



.{donttitle} Hide Assertions in Macros

[source,js]

----

function gt(a, b) { return a > b; }



function gtTrue(t, a, b) {

  if(a <= b) { return t.pass(); }

  t.true(gt(a, b));

}



test(gtTrue, 1, 2);

test(gtTrue, 2, 1);

----



// TODO: "However, some can be more complex if needed" and include something like:

// https://github.com/CodeLenny/modconf/blob/master/test/priorities/comparisons.js



.{dotitle} Use `beforeEach` for Setup

[source,js]

----

test.beforeEach("creates a", t => {

  t.context.a = Math.random();

});



function checksNum(t, fn) {

  t.true(fn(t.context.a));

}



test("typeof", checksNum, a => typeof a === "number");

test("isFinite", checksNum, isFinite);

----



.{dotitle} Throw Errors if Preconditions not Met

[source,js]

----

test.beforeEach("connect to database", t => {

  t.context.db = db.connect(/* ... */);

  if(!t.context.db.connected) {

    throw new Error("Could not connect to database.");

  }

});

----



=== Test Contents



The contents of the test should run code, and then use an assertion to check the output.



[%hardbreaks]

{do} Use built-in {ava-assertions}

{do} Use {ava-planl} (`t.plan(1);`)



Don't use multiple assertions when possible.

If you need to validate pre-conditions, test in `beforeEach` and throw an error.



Name tests according to the behavior they are checking.

// TODO: include examples of test names.



.{donttitle} Duplicate Test Titles

[source,js]

----

// returns '200' status code

test("returns '200'", t => {

  return request(app)

    .get("/")

    .then(res => {

      t.is(res.status, 200);

    })

});

----



=== Missing from Test Files



Some things are intentionally missing from the test files:



mocks::

Mocked classes and data structures should be stored as test helpers.

utility functions::

Functions that assist in constructing test data should either be run once per test (`before`/`beforeEach`)

or stored as a test helper.