{"id":15062111,"url":"https://github.com/sumanjs/suman","last_synced_at":"2025-04-10T09:54:49.642Z","repository":{"id":57374377,"uuid":"46830244","full_name":"sumanjs/suman","owner":"sumanjs","description":"🌇 🌆 🌉 Advanced, user-friendly, language-agnostic, super-high-performance test runner. http://sumanjs.org ","archived":false,"fork":false,"pushed_at":"2024-02-23T18:31:06.000Z","size":10903,"stargazers_count":62,"open_issues_count":58,"forks_count":8,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-03-24T08:48:24.463Z","etag":null,"topics":["ava","better-than-ava","better-than-mocha","better-than-tapjs","ci-cd","concurrent","developer-experience","docker","dx","high-performance-testing","mocha","mocha-reporter","mocha-tests","nodejs","parallel","performance-testing","suman","tape","testing","testing-tools"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sumanjs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-11-25T01:44:44.000Z","updated_at":"2024-08-05T21:32:51.000Z","dependencies_parsed_at":"2024-06-18T22:42:01.127Z","dependency_job_id":"f8917d54-a1c5-4bf3-aa49-94f67e044c94","html_url":"https://github.com/sumanjs/suman","commit_stats":{"total_commits":223,"total_committers":2,"mean_commits":111.5,"dds":0.0358744394618834,"last_synced_commit":"c7ad5c3e12b7283d284471f5cd0c396e9add7ca9"},"previous_names":["oresoftware/suman"],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sumanjs%2Fsuman","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sumanjs%2Fsuman/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sumanjs%2Fsuman/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sumanjs%2Fsuman/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sumanjs","download_url":"https://codeload.github.com/sumanjs/suman/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247744282,"owners_count":20988781,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ava","better-than-ava","better-than-mocha","better-than-tapjs","ci-cd","concurrent","developer-experience","docker","dx","high-performance-testing","mocha","mocha-reporter","mocha-tests","nodejs","parallel","performance-testing","suman","tape","testing","testing-tools"],"created_at":"2024-09-24T23:30:26.523Z","updated_at":"2025-04-10T09:54:49.617Z","avatar_url":"https://github.com/sumanjs.png","language":"JavaScript","readme":"\n\u003cimg align=\"right\" width=\"20%\" height=\"20%\" src=\"https://raw.githubusercontent.com/sumanjs/suman-docs/master/images/suman-hex.png\"\u003e\n\n[![npm version](https://badge.fury.io/js/suman.svg)](https://badge.fury.io/js/suman)\n\n[![Greenkeeper badge](https://badges.greenkeeper.io/sumanjs/suman.svg)](https://greenkeeper.io/)\n\n[![Build Status](https://travis-ci.org/sumanjs/suman.svg?branch=master)](https://travis-ci.org/sumanjs/suman)\n\n[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/1072/badge)](https://bestpractices.coreinfrastructure.org/projects/1072)\n\n[![Coverage Status](https://coveralls.io/repos/github/sumanjs/suman/badge.svg?branch=master)](https://coveralls.io/github/sumanjs/suman?branch=master)\n\n\n# Suman: universal test runner - run tests in the language of your choice.\n\nDesigned to run tests written in any language - because Suman runs tests as child processes - just write TAP to stdout\nvia the runtime of choice (Golang, Java, Python, Node.js, Bash, whatever). \n\n\u003cp\u003e\nSuman is intended for unit testing in the browser, for backend and system testing,\nfor end-to-end testing with Selenium, etc.\n\n\u003cp\u003e\nOriginally designed for TypeScript and Babel transpilation, Suman was generified to support\ncompiling from source to target for \u003ci\u003eany\u003c/i\u003e language. Suman is written in TypeScript, and has strong support for TS.\nAll in all, Suman is a batteries-included test runner which encourages testing best practices through its design. \nStarting with Suman is very easy, but you will find it has extremely powerful features, as your demands increase.\n\n\u003cp\u003e\n\u003cb\u003ePerhaps most importantly, Suman makes testing significantly faster for both CPU bound and I/O bound test suites.\u003c/b\u003e\n\u003cb\u003eThe speed benefits are generally most useful during test development\u003c/b\u003e\n\n\u003cp\u003e\nAs final points, Suman is designed to be universal and not just another JS framework - it's designed for the long-haul. \n\u003cp\u003e\nElegance and utility of reporting output is paramount to the Suman philosophy.\nReporters are the best place to contribute for people interested in creating beautiful test output.\nAs they say (visual) design is what really separates contenders in a world of surplus.\n\u003cp\u003e\n\n\n# Purpose of project\n\u003e  I wrote Suman because I found test runners and test harnesses in the Node.js ecosystem to be lacking.\n\u003e  I consider Mocha to be woeful, and AVA to be slow, under-featured and too tightly coupled with Babel.\n\u003e  If your team is interested in speeding up your testing cycles, Suman is the absolute right place to look for answers.\n\u003e  Suman is designed to be 'better all-around' than AVA, TapJS and Mocha, etc. Reading the issue tracker for Mocha\n\u003e  made it very apparent that Mocha was never going to be vastly improved. If you look at the Karma codebase, you\n\u003e  also find that it's not designed with Webpack, TypeScript or Babel in mind, etc.\n\u003e\n\n# The Suman Challenge\n\nIf you can find a problem that Mocha or AVA has, which Suman does not solve, I will find some reward for you. \n\u003cbr\u003e\nSuman was written so that it would suffer from none of the problems that existing test runners have.\n\n\n### \u0026#9658; Disclaimers: \n\u003e\n\u003e Suman supports Node versions \u003e= 6.0.0, since it makes heavy use of the Proxy class and Symbol primitives.\n\u003e\n\u003e Windows support is on the roadmap, but will not be ready anytime soon. Currently, MacOS and *nix support only.\n\u003e\n\u003e Browser will be supported, but not until ~Summer 2018.\n\u003e\n\n----------------------------------------------------------------------------------------\n\n# Expected official release date\n\nSuman is not officially released yet - expected release date ~October 2017.\nUntil then, expect bugs, missing docs, etc. Proceed at your own risk :D\n\n\n# \u0026#9658; Documentation \n\n The Suman docs [sumanjs.org](http://sumanjs.org \"Suman Docs\")  \n\n-----------------------------------------------------------------------------------------\n\n\n## Suman is made up of two independent pieces:\n\n1. A CLI that can run a test in any language, and is designed to handle transpilation/compilation for any language as well.\n2. A super powerful test harness that can be used with JavaScript/Node.js tests. This test harness is highly recommended\n and has strong support for TypeScript.\n\nThe CLI can be used to run tests in any language; on the other hand, \nthe test harness, only works with Node.js and browser based JS.\n\nYou do not need the suman CLI to run suman tests. \u003cbr\u003e\nYou do not need to run suman tests to use the CLI. \u003cbr\u003e\nThey are completely independent, while obviously being designed to work great together. \u003cbr\u003e\n\n\u003cp\u003e\n\n--------------------------------------------------------------------------------------------\n\n## What makes Suman better and different\n\n\u003cdiv\u003e Everything about Suman is designed for parallelization. \u003c/div\u003e\n\nSuman is not just better than test runners in Node.js and JS land - it improves on test runners written\nfor other languages as well. This is primarily because the Suman CLI is built so that you can have complete control \nover the parellization of your tests. You can select which tests should exclude each other and run in series,\nand which tests should run in parallel with each other. In the _long run_ however, all tests should run in parallel\nin their own container with their own resources. In the _long long run_, each container should run on its own hardware,\nall in the name of speed. Suman has an experimental feature called \"suman groups\" which will allow you to group your\ntests together and assign a group to a container. Eventually the dream is support the assignment of each container\nto its own hardware, using different cloud platforms.\n\n___\n\n# Top 5 reasons to use Suman, instead of Mocha or AVA\n\n1. Run tests in any language or executable, not just Node.js/JavaScript.\n\n2. =\u003e You can run Mocha tests and AVA tests using the Suman CLI, but not the other way around!\n\n3. Easily containerize any test process in development, using `suman --ctrz x.js`\n\n4. Imagine you're a senior developer and a more junior developer joins the team, \nand they write a new test that brings down the whole CI/CD pipeline. With Suman, tests don't directly \ninteract because they are run in separate processes; in Mocha, not so much. Avoid that fateful day.\nUsing Suman, it will be clear which test process is causing the fatal problem; but with a single process\nit will not necessarily be clear that it is the junior developer's test which caused the issue.\n\n5. Suman is much faster than AVA, because Suman does not require transpilation.\n\n\u003cp\u003e\n\n# Suman CLI features\n\n```console\n$ suman\u003e \n```\n\n\u003cp\u003e\n\u003cdiv\u003e ✓ \u003cb style=\"color:purple\"\u003e  tests can all run in parallel, in separate processes \u003c/b\u003e \u003c/div\u003e\n\u003cdiv\u003e ✓ \u003cb style=\"color:purple\"\u003e  agnostic \u003c/b\u003e =\u003e want to learn a new language? write a test script with language X, then run the test with Suman.\u003c/div\u003e\n\u003cdiv\u003e ✓ \u003cb style=\"color:purple\"\u003e  flexible, generic, robust \u003c/b\u003e =\u003e  CLI can run JS tests directly, or in a child process \u003c/div\u003e\n\u003cdiv\u003e ✓ \u003cb style=\"color:purple\"\u003e  flexible, generic, robust \u003c/b\u003e =\u003e  Composability =\u003e Suman tests can run other Suman tests (in child processes). Sumanception! \u003c/div\u003e\n\u003cdiv\u003e ✓ \u003cb style=\"color:purple\"\u003e  test isolation \u003c/b\u003e =\u003e  each test can run in its own process \u003c/div\u003e\n\u003cdiv\u003e ✓ \u003cb style=\"color:purple\"\u003e  test independence \u003c/b\u003e =\u003e  easily run only one test at a time (unlike other Node.js test runners...)\u003c/div\u003e\n\u003cdiv\u003e ✓ \u003cb style=\"color:purple\"\u003e  \"nodeable test scripts\" \u003c/b\u003e =\u003e run individual JS tests with the node.js executable\u003c/div\u003e\n\u003cdiv\u003e ✓ \u003cb style=\"color:purple\"\u003e  supports unit testing in the browser \u003c/b\u003e (tested on Chrome and Firefox)\u003c/div\u003e\n\u003cdiv\u003e ✓ execute tests written in \u003cb style=\"color:purple\"\u003eany\u003c/b\u003e language, use write TAP to stdout\u003c/div\u003e\n\u003cdiv\u003e ✓ synchronous \u003cb style=\"color:purple\"\u003e*and*\u003c/b\u003e asynchronous reporters (!) Your reporter can be async.\u003c/div\u003e\n\u003cdiv\u003e ✓ execute tests written in \u003cb style=\"color:purple\"\u003eany\u003c/b\u003e language, use write TAP to stdout\u003c/div\u003e\n\u003cdiv\u003e ✓ Complete control =\u003e You *can* run JS unit tests all in the same process for speed, as needed. \u003c/div\u003e\n\u003cdiv\u003e ✓ \u003cb style=\"color:purple\"\u003e  bash completion is available for the Suman CLI :D \u003c/b\u003e\n\u003c/p\u003e\n\n\n## Suman test harness features\n\n```javascript\nimport * as suman from 'suman';\n```\n\nThese are the features for creating tests in JavaScript/TypeScript.\n\n\u003cp\u003e\n\u003cdiv\u003e ✓ \u003cb style=\"color:purple\"\u003e  fully asynchronous \u003c/b\u003e =\u003e allows for easy, dynamic test case creation\n\u003cdiv\u003e ✓ \u003cb style=\"color:purple\"\u003e  agnostic \u003c/b\u003e =\u003e works with your favorite assertion library; Suman is also bundled with Chai, if you wish to use that.\n\u003cdiv\u003e ✓ \u003cb style=\"color:purple\"\u003e  no globals \u003c/b\u003e =\u003e  no global variables as part of test harness - therefore JS tests can be run with Node.js executable \u003c/div\u003e\n\u003cdiv\u003e ✓ \u003cb style=\"color:purple\"\u003e  flexible, generic, robust \u003c/b\u003e =\u003e  CLI can run JS tests directly, or in a child process \u003c/div\u003e\n\u003cdiv\u003e ✓ \u003cb style=\"color:purple\"\u003e  test isolation \u003c/b\u003e =\u003e  each test can run in its own process \u003c/div\u003e\n\u003cdiv\u003e ✓ \u003cb style=\"color:purple\"\u003e  declarative style \u003c/b\u003e =\u003e declare (sync and async) dependencies for each test, and only load those\u003c/div\u003e\n\u003cdiv\u003e ✓ \u003cb style=\"color:purple\"\u003e  supports unit testing in the browser \u003c/b\u003e (tested on Chrome and Firefox)\u003c/div\u003e\n\u003cdiv\u003e ✓ \u003cb style=\"color:purple\"\u003e  supports observables (RxJS5) \u003c/b\u003e \u003c/div\u003e\n\u003cdiv\u003e ✓ only \u003cb style=\"color:purple\"\u003e18mbs \u003c/b\u003e on filesystem as npm install -D\u003c/div\u003e\n\u003cdiv\u003e ✓ works with \u003cb style=\"color:purple\"\u003e Selenium \u003c/b\u003e (use selenium-webdriver, wd, or webdriver.io)\u003c/div\u003e\n\u003cdiv\u003e ✓ Built-in watch features =\u003e Watch files, and run tests on changes \u003c/div\u003e\n\u003cdiv\u003e ✓ Complete control =\u003e You *can* run unit tests all in the same process for speed, as needed. \u003c/div\u003e\n\u003c/p\u003e\n\n ---\n\n##  Suman is a singular test runner focused on Node.js and front-end JavaScript, but is both generic and robust so that it can run tests in any runtime or language\n\n\nSuman is written with Node.js, and is focused on testing Node.js code, \nbut can run tests written in any language, not just JavaScript. This is \nbecause it can run tests in child processes and collect results using TAP (Test Anything Protocol, via stdout), IPC, or websockets.\n\nSuman can run a test in any language which exposes a script with a hashbang or a binary entrypoint file (e.g. Golang or C).\nTo run Java tests, where Java does not compile to binary and where you cannot put a hashbang in a .class file,\nyou would need to call Java from a shell script.\n\nIt is designed for maximum test performance, via careful parallelization at\nevery juncture in the testing process/pipeline. \n\n-----\n\n\n# \u0026#9658; Installation\n\n\u003ci\u003e For command line tools:\u003c/i\u003e\n## ```$ npm install -g suman```\n\n=\u003e **Please** do *not* use sudo to install suman globally; if you need to use sudo, then something is probably wrong\n=\u003e See: https://docs.npmjs.com/getting-started/fixing-npm-permissions\n=\u003e To avoid any problems with permissions, Suman recommends usage of NVM\n\n\u003ci\u003e For test suites in your project:\u003c/i\u003e\n## ```$ cd \u003cproject-root\u003e \u0026\u0026 suman --init``` \n\n* To avoid global NPM modules see: \"Local installations only\"\n\n=\u003e to convert a Mocha test or whole directory(s) of Mocha tests to Suman tests use \u003cbr\u003e\n### ```$ suman --convert --src=\u003csrc-file/src-dir\u003e --dest=\u003cdest-dir\u003e```\n\n=\u003e to simply install Suman as dev-dependency in any project you can use ```$ npm install -D suman```, \u003cbr\u003e\nhowever ```$ suman --init``` is the much preferred way to initialized suman in a given project, because it will\nadd a directory to your project which you can move to a location of your choosing.\n\n\n### \u0026#9658; Local installations only =\u003e \n\nIf you wish to avoid global NPM module installations, we commend you, see: \n [http://sumanjs.org/tutorial-11-advanced-installation.html/](http://sumanjs.org/tutorial-11-advanced-installation.html \"Suman Docs: Advanced Installation\") \n\n\n## Example commands\n\n```bash \n\n suman test/**/*.py   # run python tests\n \n suman test/**/*.rb   # run ruby tests\n \n suman test/**/*.sh  test/**/*.go  # run bash and golang tests\n \n suman test/src/*.spec.js --concurrency=6 # run the matching tests, no more than 6 Node.js processes at a time.\n\n suman -w project   # run a set of tests when a project file changes\n\n suman -w tests   # when developing a test, run it upon changes\n \n \n```\n\n## The Suman Story\n\nThe human side of this endeavor. (A link to docs will be here soon.);\n\n\n### Suman uses several intelligent pieces of setup:\n\nGlobal installations of Suman simply look for local installations to run. So if you run the suman command installed via npm install -g suman, that CLI\nwill just invoke the local installation of suman in any given project - this prevents any potential \nconflicts if there is a difference between the global/local module versions. Excellent!\n\nIf you use NVM and switch between Node.js versions, you can use bash functions (provided by SumanJS) which will \ndenecessitate the need for any global installations of Suman at all.\n\nSuman is designed to interop well with the most common libraries in the ecosystem for handling asynchronous code.\n\n\n## Simple example\n\n```js\n\nimport * as suman from 'suman';\nconst {Test} = suman.init(module);\n\nTest.create('example', (baz, http, beforeEach, context, inject, foo, x, beforeAll) =\u003e {\n\n  // Suman uses simple old-school style JavaScript DI\n  // we have injected some core modules by name (http, assert, path)\n  // we have also injected a module from our own project, baz\n\n  inject('bar', () =\u003e {\n    return baz(foo).then(v =\u003e {\n      return v.filter(val =\u003e val.isGreen())\n    })\n  })\n\n  beforeAll(h =\u003e {\n    return x.anything().then(function(v){\n      h.assert(typeof v === 'boolean');\n      h.$inject.v = v;\n    });\n  });\n\n  beforeEach(t =\u003e {\n    t.data.v = (t.value.v * 2) + t.$inject.v;\n  })\n\n  context('foo', {mode: 'series'}, (bar, it) =\u003e {\n\n    it('a', {value: 5}, t =\u003e {\n      t.assert.equal(t.title,'a')\n      t.assert.equal(t.data.v,10)\n    })\n\n    it('b', t =\u003e {\n      t.assert.equal(t.title, 'b')\n    })\n\n    it('c', t =\u003e {\n      t.assert.equal(t.title, 'c')\n    })\n\n    context('nested child', {mode: 'parallel'}, (bar, it) =\u003e {\n\n      it('a', t =\u003e {\n        t.assert.equal(t.title, 'a')\n      })\n\n      it('b', t =\u003e {\n        t.assert.equal(t.title, 'b')\n      })\n\n      it('c', t =\u003e {\n        t.assert.equal(t.title, 'c')\n      })\n\n    })\n\n  })\n\n})\n\n```\n\n\n# \u0026#9658; Purpose\n\nThe purpose of the Suman library is to provide the most sophisticated test runner in the Node.js ecosystem, \nwith a better developer experience, better features, higher performance, improved debuggability, \nand more expressiveness than AVA, Mocha, and Tape. \nSuman is a first-rate library and we hope you take the time to compare its capabilities with AVA, Mocha, Tape, TapJS, etc.\nOne of the more exciting things about Suman is that it can run tests written in any language; all you have to do\nis output TAP (Test Anything Protocol).\n\nThe primary aims are:\n\n* Developer experience and test debuggability are above ALL else\n* Provide a beautiful and intuitive API\n* Solve all major and minor problems in the Mocha API, specifically\n* Borrow some of the best features from Mocha, AVA and Tape\n* Make tests run faster by leveraging concurrency provided by async I/O *and* separate Node.js processes\n* _Isolate_ tests by running them in separate processes, so they do not share memory nor interact directly\n* Make tests _independent_, so that you can easily run one test at a time (damn you Mocha!).\n* Make debugging your test files easier; this is achieved by allowing for the running of tests with the plain-old node executable,\nthis makes Suman tests \"node-able\"\n* Provide cleaner output, so that developer logging output is not necessarily mixed with test result =\u003e \nachieved by using child processes and ignoring stdout.\n* Add the missing features from Mocha, Tape and AVA, while simplifying portions of the Mocha API and doing\naway with (implicit) global variables.\n* Stick closely to the popular Mocha API, so that automatic conversion is possible from Mocha to Suman,\n and that transition is as seamless as possible for the developer - you do *not* have to learn a whole new API!\n* Allow for maximum dynamicism so that Suman can match all use cases of users.\n* Allow users to create Suman-runnable tests in any languages - this possibility is created by running tests in child processes.\nWriting tests in the form of shell scripts is something that almost every project could benefit from and now you can run these \ntests through your universal test runner - Suman!\n* Composability - Suman tests should be able to run suman tests as child processes, ad inifitem, without hiccups.\n\n\n# \u0026#9658; Test Framework Comparison\n\n## The Table of Goodness\n\n\n|         | Node-able                                                                 | Supports ES6/ES7  features            | Supports test isolation using  multiple Node.js processes | Concurrency within suites | Dependency Injection |\n|---------|---------------------------------------------------------------------------|---------------------------------------|-----------------------------------------------------------|---------------------------|----------------------|\n| Mocha   | No                                                                        | No                                    | No                                                        | No                        | No                   |\n| Jasmine | No                                                                        | No                                    | No                                                        | No                        | No                   |\n| Tape    | Yes                                                                       | No                                    | No                                                        | No                        | No                   |\n| AVA     | No, requires transpilation first                                          | Yes                                   | Yes                                                       | Yes                       | No                   |\n| Suman   | Yep, you can run any given test suite with the plain old node executable  | Yep, Suman will support all features  | Yep                                                       | Yep                       | Yep                  |\n\n\n## Le Matrix of Madness\n\n|         | Implicit globals | Forces you to use their assertion library madness  | Confusing JS contexts madness                                         | Developer debugging / console.log output mixed with test output madness                                   | no concurrency madness |\n|---------|------------------|----------------------------------------------------|-----------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------|------------------------|\n| Mocha   | Yes              | No                                                 | Yes                                                                   | Yes                                                                                                       | Yes                    |\n| Jasmine | Yes              | No                                                 | Yes                                                                   | Yes                                                                                                       | Yes                    |\n| Tape    | No               | Yes                                                | No                                                                    | Yes                                                                                                       | Yes                    |\n| AVA     | No               | Yes                                                | No                                                                    | ?                                                                                                         | No                     |\n| Suman   | Nope             | Nope, Suman is completely assertion-lib agnostic   | Nope, Suman greatly simplifies the context puzzle that Mocha provided | Nope, Suman runner uses silent option with child_process so your output doesn't mix with the test results | Nope                   |\n\n\n## The reasons why Mocha and its peers need a replacement are clear:\n\n* In Mocha, Tape and Jasmine test files were not run in separate processes (necessary for isolation, speed and independence of test results)\n* Using Mocha, Tape and Jasmine, not only was everything run in a single process, but all test cases and hooks were also run in series, which takes unnecessary amounts of time for tests utilizing async I/O\n* Mocha prescribed no solution to the problem of starting up the services necessary to do system/integration testing - using Mocha/Tape/AVA it is up to the developer to manually start those services,\nwhich makes automated testing much more difficult.\n* Single process test runners like Mocha face out-of-memory issues - https://github.com/mochajs/mocha/issues/2555, these issues are\nmuch much less likely to occur if tests are split into multiple processes\n* Mocha and Jasmine could not move forward with ES6/ES7 features due to certain software patterns used (globals and complex context binding)\n* a BIG ONE: clean reporting - at the command line, using Mocha and Jasmine, logging/debugging output by the developer would obfuscate the test results, nullifying any advantage of reporting tools. Suman has a simple\ntrick up its sleeve to allow for 100% clean reporting for any test or group of tests. \n* Mocha and Jasmine could not have certain useful serverside features, because they were also constained by running in the browser.\n* Mocha lacked real ability to do true dynamic testing (meaning, registering a dynamic number of it() test cases) =\u003e e.g., make a network call, get X values, create a test case for each.\n* Mocha had confusing and obfuscated context values (values for 'this'), which we have greatly simplified, allowing for the usage of arrow functions, etc \n* Mocha, Jasmine and Tape lack some other nice features that are implemented by Suman. For example, Tape's ability to pre-load modules using the command line \nbefore running tests is nowhere near as powerful or easy to use as the dependency injection ability of this library.\n* Using Mocha, Tape, Jasmine you could not easily pass data to tests, to reuse the same test code for different scenarios; Suman allows you to pass dynamic data\nto tests using dependency injection.\n* cannot call tests programmatically without wrapping Mocha test suite in a function, Suman allows you to call tests programmiatcally, without having\nto wrap tests in a function.\n* After writing a few Mocha tests, developers will find it's very difficult to properly run only one test - the \"only\" feature can help,\nbut there are so many bugs that can crop up because of this, especially if you have some global hooks that need to run before your \"only\" test needs\nto run. Suman solves this problem ridiculously well, because Suman was designed to solve this very problem from the ground up. \n\n\n## Suman Philosophy \n\n* \"Just works\" - no need for addons or plugins, unless you want to write a custom reporter\n* Fail-fast\n* Stick to Node core modules\n* Unlike AVA, you don't need to transpile with Babel if you don't want to: _as ES6 generators + Promises\n can give you the same coding patterns as ES7 async/await_\n* Provide a full-featured, non-dumbed-down API that's easy to get started with, and\nboth powerful and intuitive to use over the long-run.\n* Listen to what the community wants.\n* Leverage Javascript's strengths (ahem *closures*)\n* Don't be lazy.\n* As Suman is a command line application, we can utilize a more functional programming style\n* Details matter, and developer experience matters most.\n\n\n# \u0026#9658; Suman features in detail:\n\n* \u003cb\u003e basics \u003c/b\u003e\n    * =\u003e tdd/bdd interfaces \n    * =\u003e easy migration from Mocha (it's automated using the --convert option)\n    * =\u003e extremely powerful features, while aiming to be straightforward, clean, concise, consistent and accurate; \n    * =\u003e designed with ES6 and ES7 in mind, including Promises, generators and async/await \n\n\n* \u003cb\u003e Improved mechanics, syntax and semantics \u003c/b\u003e\n    * singular param (t) is used for all hooks and test cases, in the style of AVA\n    * Pass data from test cases directly to hooks using the t.value option of a test case\n    * Pass data from hooks directly to test cases using the t.data value\n    (neither are possible with Mocha, and are very much a missing feature)\n    * encapsulation and immutability are utilized much more effectively than with Mocha etc\n    * instead of a \"grep\" option like Mocha's, we have \"match\" because we are filtering input not output!\n    \n  \n* \u003cb\u003e Full-blown concurrency\u003c/b\u003e\n    *  your tests will run much faster, especially if they are doing lots of network I/O\n    *  test files are run in separate Node.js processes for speed, isolation and independence\n    *  test cases and hooks in any given suite can be run concurrently, when using asynchronous I/O, \n    using the \"parallel\" option in your code\n    *  capability to control maximum number of processes running at a time (cap it at 6 processes, or 18 processes or whatever)\n    *  capability to add constaints to prevent any given pair of tests from running at the same time, (if two different tests access the same\n    external resource for example, and you don't want the interaction to cause false negatives).\n       \n       \n* \u003cb\u003e Improved reporting \u003c/b\u003e\n    *  using the Suman test runner, you can prevent any logging output from mixing with test reports, by redirecting stdout from the child process(es).\n    *  Future effort: Suman includes a built-in web reporter that you can use to share test results with your team, using the Suman server\n    *  Future effort: Suman server provides ability to store past test results (backdata) and view test results chronologically with browser to look at trends\n    *  Future effort: testing backdata is stored in a local SQLite database \n     which will allow you to run real queries on your test results, and share results with your team.)\n    \n* \u003cb\u003e Automatic test execution and/or test transpilation \u003c/b\u003e\n    * Using ```suman --watch``` you can execute test files or transpile test files as you write them\n    * Suman watcher processes run your tests and pipe stdout/stderr to log files which you tail with a terminal or browser window\n    * Running tests on the fly is a major portion of the optimal Suman workflow, and makes it all the more fun.\n       \n\n* \u003cb\u003e Dynamicism \u003c/b\u003e\n    * If there's anything you want to do with a test runner, you can do it with Suman\n    * Test files themselves allow for all sorts of dynamic behavior, dynamic and asynchronous test case generation, etc\n    * You can call tests programmatically and use them as macros\n    * Suman tests can create child processes which call Suman tests, etc etc.\n    \n       \n* \u003cb\u003e Use suman.once.js to run hooks before the test runner starts \u003c/b\u003e\n    * these hooks can be used to start the services necessary for any given test file to run successfully; they only run once no matter how many tests are run, are only run\n    if tests declare so. They offer a lighter weight option than containers for starting up the necessary servers in your backend system.\n    *  Your suman.once.js file can augment the behavior of container build scripts etc, to help start up services necessary for testing to commence\n       \n       \n* \u003cb\u003e Very simple but powerful dependency injection (DI/IoC)\u003c/b\u003e\n   *  Inject dependencies sourced synchronously or asynchronously\n   *  Most useful for injecting values acquired _asynchronously_, such as successful network connections and database values\n   *  Inspired by familiar tools such as Angular and RequireJS\n   *  Inject any core/\"built-in\" Node.js module by name, with zero configuration\n   *  DI is used throughout the library, and relieves the burden on the developer to remember order of parameters\n   *  Inject network values, test dependencies and library dependencies\n    \n    \n* \u003cb\u003e Test runner tuning \u003c/b\u003e\n    *  Add contraints to prevent any given pair of tests from running at the same time\n    *  Cap the total number of processes running at the same time\n    *  Suman 'once' feature gives the developer the option to run checks to see if all necessary network components are live before running any given test\n\n    \n* \u003cb\u003e Easy migration from Mocha \u003c/b\u003e\n    *  Suman includes a command line option to convert whole directories or individual Mocha tests to Suman tests\n    *  before/after/beforeEach/afterEach hooks behave just like in Mocha\n    *  solid command line tools and better grep facilities than predecessors\n    *  skip/only also work like Mocha\n    \n    \n* \u003cb\u003e Optional but first-rate transpilation features \u003c/b\u003e\n    * Suman support for tranpilation is first-rate\n    * Suman allows you to use \"babel-register\" to transpile your sources on the fly or transpile a src directory to a target directory\n    * If you have less than ~20 tests, the recommended way to incorporate Babel in your testing is to simply transpile your entire \"test\" directory to \"test-target\"\n    * Using a \"test-target\" directory instead of \"babel-register\" allows for better debugging, performance and transparency in your testing system\n    * If you have more than 20 tests, Suman does not recommend transpilation, due to the performance penalty.\n    * For enterprise usage, we don't expect many teams to use transpilation features anyway; it's more likely individual developers who may\n    wish to use Babel to transpile tests for a library.\n\n\n* \u003cb\u003e Log files help you debug \u003c/b\u003e\n\n    * Using Suman tests, you have 3 options as to how you execute a test file: ```1. node x.js, 2. suman x.js and 3. suman --runner x.js```\n    * Using the Suman runner, your test file(s) all run in child processes. This can make debugging harder, but Suman sends all stderr from the child processes\n    to a single log file to help you debug.\n    * Normally, to debug a test, you would run the test with node or using suman without using the runner, but if you see a problem with a particular test that only\n    occurs when using the runner, then the log files will help.\n\n* \u003cb\u003e Freedom: Suman is not highly opinionated, but gives you powerful features\u003c/b\u003e\n    *  Suman prefers the standard core assert Node module (Suman has unopinionated assertions), but like Mocha you can use any assertion lib that throws errors\n    *  Callbacks, promises, async/await, generators and event-emitters/streams are supported in any test case or hook.\n\n\n## Suman design\n\n* no implicit globals in test suite files, which were avoided due to the problems they caused for Jasmine and Mocha.\n* Suman uses domains to isolate errors in asynchronous tests and hooks, and currently this is the only solution to this problem at the moment. \nLab, the test runner for Hapi servers, also uses domains for this same purpose, and using domains allows for much better coding patterns (avoiding globals in\nthe suman library.) Domains are facing deprecation, and Suman will replace domains with whichever suitable replacement is chosen by the Node.js core team,\nalthough after talking with many people near to core, domains will be supported for quite some time. As it stands, _Suman is a perfect use case for domains, \nas untrapped errors are supposed to be thrown in test code_ (assertions, etc), and the developer experience in this library will be better than any other test library because of the use of domains, as they basically\nguarantee that we can pin an error to a particular test case or hook, no matter where the error originated from.\n\n## Details matter\n\n    * we designed Suman with details in mind\n    * fewer logical branches and contingencies than Mocha WRT rules for hooks\n    * much better semantics, with new standard functions alongside Mocha's 'done' callback: 'ctn', 'pass', 'fail' and 'fatal' are new functions\n    each with a unique purpose and meaning, and done is still in Suman's API with the same meaning as Mocha!\n    * friendly error messages, that also get sent to suman-stderr.log for reference\n    * when debugging, (the debug flag is set) timeouts will automatically be set to 'infinity'\n\n\n## We can say with confidence that Suman is the most powerful test framework for serverside JavaScript on planet Earth\n =\u003e as it gives the developer total control and access to a very large set of features, \n with the explicit goal of being bug-free first, full-featured second.\n\n\n### More Suman Examples\n\n* See the documentation @ sumanjs.org\n* You can also read: https://medium.com/@the1mills/introducing-suman-a-node-js-testing-library-20fdae524cd\n\n### S.L.A.\n\nThe Service Level Agreement is that Suman will constantly be up-to-date with the newest features available via the node executable.\nWe will focus on what's in Node and not what's available in Babel or other transpilers. That being said, we will also work to ensure Babel features are also supported,\nbut we will primarily focus on making Suman completely bug-free when it comes to the latest version of Node, not the latest abilities of Babel or the like.\nBy the time any ES6/ES7/ES8 feature is available in Node, it will be supported by Suman. We want to emphasize the utility of the option of running things\nwith the plain old Node executable, as opposed to adding the complexity of transpilation.\n\n\n### Execution modes for a single test file\n\nYou can execute a test file with the plain ```node``` executable, with ```$ suman``` and with ```$ suman --runner```\n\nHere are the differences between the 3 options:\n\n|                         | $ node a.test.js  | $ suman a.test.js  | $ suman --runner a.test.js  |\n|-------------------------|-------------------|--------------------|-----------------------------|\n| command line options    | no                | yes                | yes                         |\n| runs multiple processes | no                | no                 | yes                         |\n| suppress stdout/stderr  | no                | no                 | yes                         |\n| easy to debug?          | easy              | medium             | hard\n\n\nIn order to run multiple files, you must use ```$ suman --runner```; the above table\nonly pertains to running a single test file (usually when developing a particular test.)\n\n\n### FAQ\n\nTBD\n\n\n## Important aside - How is Suman better than AVA?\n\n It should be abundantly clear why Suman is better than Mocha, but how is Suman better than AVA? \n (If it's not clear why Suman is better than Mocha then scroll back up).\n Suman borrows some excellent features from Mocha that AVA seems to ignore, including the ability\n to use nested describe blocks for more control and preventing the sharing of scope within tests. AVA basically\n co-opted Tape and added concurrency. Suman co-opted Mocha, added concurrency, better reporting, dependency injection and \n less confusing contexts for callbacks. Suman has more powerful facilities for asynchronous testing than AVA due to Mocha/Jasmine-style hooks\n and nested describes. Dependency injection also makes Suman extremely convenient and fun to use, compared to AVA.\n Suman is simply more powerful and richer in features than AVA. \n \n * AVA test are not \"node-able\" - you *cannot* run them with node directly; Suman tests are node-able, which makes \n debugging so much easier and intuitive! Note that Suman does make is easy for developers to debug child processes.,\n giving them built-in tools to do so.\n \n ![alt text](https://github.com/ORESoftware/suman-private/blob/dev/images/ava-prob.png)\n \n * AVA requires Babel transpilation, which adds unnecessary complexity for test environments, and is also much slower\n * AVA does not handle errors thrown in asynchronous code gracefully, Suman is much better in this regard.\n * AVA does not feature the nested describes of Mocha or Suman, which limits the expressiveness of the library\n tremendously!\n * AVA expects you to use its assertion library, whereas Suman will accept usage of any assertion library that\n you are already familiar with.\n * Furthermore, AVA does not prescribe solutions to common test problems -\n \n        * registering dynamic test cases (given a dynamic value acquired asynchronously)\n        * starting up necessary services before running tests\n        * injecting different dependencies into test files\n        \n \n Alternatively, with Suman:\n\n *Suman has nested describe blocks, which are imperative for non-trivial tests; a simple but common example is a before hook that you want\n to run only for a certain subset of tests in your test file. The before hook and tests would go in a nested describe block.\n *Babel transpilation is totally optional - you can achieve the async/await pattern with generators and promises alone, you don't need ES7 for this\n *Suman uses domains to correctly map runtime/assertion errors to test cases and hooks, \n which provides a much more reliable and well-designed piece of software because it can handle any error that gets thrown, not just assertion errors.\n \n\n### Contributing - Testing and pull requests\n \n (Please see contributing.md)\n \n\n\n![alt text](https://raw.githubusercontent.com/sumanjs/suman-docs/master/images/suman.png \"Suman Primary Logo\")\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsumanjs%2Fsuman","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsumanjs%2Fsuman","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsumanjs%2Fsuman/lists"}