{"id":24081750,"url":"https://github.com/jonathanconway/testmatic","last_synced_at":"2026-02-06T09:07:47.464Z","repository":{"id":183917338,"uuid":"643826665","full_name":"jonathanconway/testmatic","owner":"jonathanconway","description":"Supercharge your test cases","archived":false,"fork":false,"pushed_at":"2024-08-15T05:38:09.000Z","size":25153,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-24T07:16:36.178Z","etag":null,"topics":["documentation","end-to-end","flows","javascript","manual","qa","steps","testing","testing-tools","tests","typescript"],"latest_commit_sha":null,"homepage":"https://x.com/testmaticapp","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jonathanconway.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-22T08:35:33.000Z","updated_at":"2024-08-15T05:38:13.000Z","dependencies_parsed_at":"2024-05-30T11:00:36.016Z","dependency_job_id":"3192e894-ca6a-4b7d-844a-97a01f89d17d","html_url":"https://github.com/jonathanconway/testmatic","commit_stats":null,"previous_names":["jonathanconway/testmatic"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jonathanconway/testmatic","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonathanconway%2Ftestmatic","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonathanconway%2Ftestmatic/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonathanconway%2Ftestmatic/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonathanconway%2Ftestmatic/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jonathanconway","download_url":"https://codeload.github.com/jonathanconway/testmatic/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonathanconway%2Ftestmatic/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267464522,"owners_count":24091505,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-28T02:00:09.689Z","response_time":68,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["documentation","end-to-end","flows","javascript","manual","qa","steps","testing","testing-tools","tests","typescript"],"created_at":"2025-01-09T23:26:08.323Z","updated_at":"2026-02-06T09:07:47.382Z","avatar_url":"https://github.com/jonathanconway.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# testmatic\n\n\u003e ⚡️ Supercharge your test cases ⚙️\n\nTestmatic is a test-case manager for quickly writing and organising human-readable test cases.\n\n- ✅ simple lists of [**_steps_**](#tests-and-steps)\n- 🧪 are grouped into [**_tests_**](#tests-and-steps)\n- 🏷️ which can be organised by [**_tags_**](#tags)\n- 🏃 with results tracked in [**_runs_**](#runs)\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\n      \u003cimg src=\"/docs/ui-demo.gif\" alt=\"Screen recording of user creating tests in the Testmatic web user interface (UI)\" /\u003e\n    \u003c/td\u003e\n    \u003ctd\u003e\n      \u003cimg src=\"/docs/cli-demo.gif\" alt=\"Screen recording of user creating tests in the Testmatic command line interface (CLI)\" /\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\nEverything is stored in Markdown files, for easy viewing, editing, searching and version control. If pushed to a server, links to Markdown files can be shared within your org, via e.g. Wiki pages, Chat posts, email, etc.\n\nThe main benefit of using Testmatic is that it helps you to perform [manual testing](#benefits-of-manual-testing) in an organised, consistent, rigorous, systematic manner.\n\nManual testing can offer fast benefits when automated test coverage is limited, development time is constrained and rapid delivery is required while minimising defects.\n\nWhenever you are ready to add automation, Testmatic can help you there too: code generators automatically convert your test cases into unit test files.\n\n## Contents\n\n- [Getting started](#getting-started)\n- [Concepts](#concepts)\n- [UI](#ui)\n- [CLI](#cli)\n- [Advanced](#advanced)\n- [Background](#background)\n- [FAQ](#faq)\n- [Contributing](#contributing)\n- [Troubleshooting](#troubleshooting)\n- [Contact](#contact)\n\n\u003c!-- ------------------------------------------------------------------------------------------------- --\u003e\n\n## Getting started\n\n### Installing the CLI\n\nGet started by installing Testmatic via the CLI.\n\n#### What you'll need\n\n- [Node.js](https://nodejs.org/en/download/) version 16.14 or above.\n\n#### Install Testmatic\n\nInstall globally via NPM:\n\n```bash\nnpm install -g testmatic\n```\n\nYou can verify the installation by running:\n\n```bash\ntestmatic\n```\n\n### Installing the local UI\n\nGet started by installing Testmatic via the UI.\n\n#### What you'll need\n\n- [Node.js](https://nodejs.org/en/download/) version 16.14 or above.\n\n#### Install Testmatic local UI\n\nInstall globally via NPM:\n\n```bash\nnpm install -g testmatic-ui\n```\n\n#### Run the Testmatic local UI\n\nYou can start the UI from your command line.\n\n```bash\ntestmatic-ui\n```\n\nYou will want to first switch to the directory where you want to locate your Testmatic project.\n\nFor example, if you want to put your Testmatic project under `~/Sources/my-app`, run this:\n\n```bash\ncd ~/Sources/my-app\ntestmatic-ui\n```\n\nThis will set up your Testmatic project in the following folder: `~/Sources/my-app/.testmatic`.\n\n### Using the hosted UI\n\nIf you just want to play around with Testmatic or use it lightly, you can immediately get started with the hosted UI.\n\nThe hosted UI can be used immediately, without any local installation.\n\nSimply navigate to: http://testmatic.surge.sh and begin adding your tests.\n\nYou can find instructions for using the hosted UI at [UI guide](#ui-guide).\n\n\u003c!-- ------------------------------------------------------------------------------------------------- --\u003e\n\n## Concepts\n\n### Tests and steps\n\nAt the heart of Testmatic is **tests** which are simple numbered lists of **steps**.\n\nYou simply follow the the steps to test the system.\n\nHere's an example of a test:\n\n\u003e User can login\n\u003e\n\u003e 1. Go to the homepage\n\u003e 2. Click the sign in button\n\u003e 3. Enter the username and password of a registered user\n\u003e 4. Click the sign in submit button\n\u003e 5. Observe that you are now signed in\n\nNotice some steps are **_actions_**, such as:\n\n\u003e 2. Click the sign in button\n\nWhereas others are **_expectations_**, such as:\n\n\u003e 5. Observe that you are signed in\n\nThe **_actions_** tell us what to do but the **_expectations_** tell us how the system should behave or respond.\n\nIf the system does not behave as expected then either there is a fault in the system or our expectation is wrong.\n\nThe title and steps should be as short and succinct as possible but also clear and specific enough to easily follow and understand.\n\n### Tags\n\nAs you write many tests, you'll no doubt notice common recurring elements.\n\nThese elements can be referenced by **tags**, which are re-usable tokens that can be used to group and organise tests.\n\nTags are indicated in steps by surrounding a piece of text with round brackets like this: `(tag)`.\n\nFor example, the \"sign in\" button might be a recurring element. Here's how it would look if we made it tag:\n\n\u003e Reset password\n\u003e\n\u003e 1. Go to the website\n\u003e 2. Click the **(sign in button)** `\u003c----------`\n\u003e 3. Enter the username and incorrect password of a registered user\n\u003e 4. Click the submit button\n\u003e 5. Observe that an error appears indicating the incorrect password\n\u003e 6. Observe that a forgotten password link appears\n\u003e 7. Click the forgotten password link\n\u003e 8. Open your email, copy the code\n\u003e 9. Switch back to the website, paste in the code\n\u003e 10. Enter a new password and click submit\n\u003e 11. Click the **(sign in button)** again `\u003c----------`\n\u003e 12. Enter the username and new password\n\u003e 13. Observe that you are signed in\n\nTags can be the following:\n\n- Visual elements in the UI\n- Pages of a website\n- Screens of an app\n- Test user accounts\n- Flows, e.g. sign in flow\n- Anything else that might be common between different tests\n\nNote: You can also attach tags to a test itself. This is useful when you want a tag to apply to a whole test, not just one or more steps.\n\n### Runs\n\nAfter you have written a test you might follow the steps of that test any number of times, to ensure the system is behaving correctly.\n\nWhat if you want to keep track of the results? This is what **test runs** are for.\n\nEach test can have multiple runs. Each run has a date/time stamp indicating when it was performed as well as a result and a list of checked/unchecked test steps.\n\nYou can mark each run with a **result**:\n\n- _Passed_ - the system behaved correctly\n- _Failed_ - there was some problem so the whole test failed\n- _Mixed_ - some steps passed, some failed\n- _Unset_ - you haven't performed the test yet\n\nYou can also check off individual test steps as you go through the run. This could be useful when making a screen recording, to show which steps have been executed, or just to keep track of the steps.\n\nEach run has a date/time stamped folder containing the run file.\n\nYou can put screenshots or screen recording files into this folder as well. That way they will be easier to locate if/when you need them.\n\nRuns are a powerful feature for measuring and organising your test results. You can quickly see which tests have passed or failed and keep your screen recordings organised.\n\n### Impacts\n\nPart of the benefit of tags (discussed earlier) is their usefulness for grouping and locating tests. If two or more tests reference the same tag then you can locate these tests by filtering by that tag.\n\nThis gives you a way of finding **impacts** – which tests might be impacted by a particular tag.\n\nFor example, suppose you are making a visual change to the website header, which might impact the sign up button. You can search for all tests that reference the `(sign up button)` tag.\n\nYour search might turn up a list of tests like this:\n\n- Sign in as user\n- Sign in as admin\n- Reset password\n- Sign out\n\nNow you can run some or all of these tests and check how the Sign up button behaves under various scenarios. If you change introduced a bug in one of the tests, you'll have a chance to find the bug and fix it early rather than waiting for it to crop up in production.\n\n\u003c!-- ------------------------------------------------------------------------------------------------- --\u003e\n\n## UI guide\n\n### Starting a project in the UI\n\nThere is no need to explicitly create a new project in the UI. As soon as you open the UI, a new test project will be created if one does not already exist.\n![Screenshot of the Testmatic UI with an empty project](/docs/ui_guide/starting_a_project_in_the_ui/empty_project.png)\n\n#### Switching to a different project in the local UI\n\nIf you are running the Testmatic local UI, you can switch to a project in a different folder.\n\nSimply exit the running Testmatic local UI instance (Ctrl+C or Cmd+C), use `cd` to switch to a different folder, then re-run `testmatic-ui`.\n\nFor example:\n\n```bash\n~/Sources/project-one $ testmatic-ui\n\nRunning Testmatic UI...\n...\n\n{ Cmd+C }\n\n~/Sources/project-one $ cd ../project-two\n\n~/Sources/project-two $ testmatic-ui\n\nRunning Testmatic UI...\n...\n```\n\n### Creating a test in the UI\n\nTo create a new test, click the Add button in the Project explorer (top-left of the screen).\n\n![Screenshot of the Testmatic UI with an empty project and clicking the Add button](/docs/ui_guide/creating_a_test_in_the_ui/click_add.png)\n\nIt will now appear in the Project explorer.\n\n![Screenshot of the Testmatic UI showing createed test in Project explorer](/docs/ui_guide/creating_a_test_in_the_ui/view_created_test_in_project_explorer.png)\n\nYou can then enter a title and steps.\n\n![Screenshot of the Testmatic UI entering text into the title field](/docs/ui_guide/creating_a_test_in_the_ui/enter_title.png)\n\n![Screenshot of the Testmatic UI entering text into the steps fields](/docs/ui_guide/creating_a_test_in_the_ui/enter_steps.png)\n\n### Adding steps to a test in the UI\n\nOpen a test by selected it in the project explorer on the left.\n\nYou can then enter some steps.\n\n![Screenshot of the Testmatic UI entering text into the steps fields](/docs/ui_guide/creating_a_test_in_the_ui/enter_steps.png)\n\n### Adding tags in the UI\n\n\u003e ℹ️ **Tags** are re-usable tokens that can be used to group and organise tests.\n\u003e\n\u003e For more info, see [tags](#tags).\n\nYou can add tags to steps or to the test as a whole.\n\n#### Adding tags to steps\n\nFocus on the step textbox and type in an \"open backet\" - `(`. Enter your tag name. Then type in \"close bracket\" - `)`.\n\nYou'll notice that a suggestion box appears – feel free to use this to quickly insert pre-existing tags.\n\n- Keyboard - You can navigate with arrow keys and select a tag with the enter/return or tab key.\n- Mouse - You can scroll up/down and click one of the tags in the list\n\n### Finding tests by tag in the UI\n\nTags appear in the Project explorer under the Tags section.\n\nSimply click a tag to view its associated tests.\n\n\u003c!-- ### Advanced UI usage\n\n#### Using Runs in the CLI\n\n#### Using Links in the CLI\n\n#### Using Impacts in the CLI\n --\u003e\n\n\u003c!-- ------------------------------------------------------------------------------------------------- --\u003e\n\n## CLI guide\n\n### Starting a project in the CLI\n\nA Testmatic project consists of a `.testmatic` folder in your current working directory, containing sub-folders for `tests`, `tags` and `runs`.\n\nYou can generate the folders using the `init` command:\n\n```bash\ntestmatic init\n```\n\n### Creating a test in the CLI\n\nYou can create your first test by running the `test add` command and answering prompt questions, pressing Enter/Return when done.\n\n```bash\n$ testmatic test add\n\nPlease enter test title: Homepage loads\n\nThank you!\n\nNow, please enter your steps, one-by-one.\n(Empty line to finish)\n\n1. Navigate to homepage\n2. Observe homepage has loaded\n3.\n\n$\n```\n\nYour test \"Homepage loads\" should now have been created.\n\nYou can verify this by running the `test list` command:\n\n```bash\n$ testmatic test list\n\nTITLE                  NAME                   DOC\nShould load homepage   should_load_homepage   ./.testmatic/tests/should_load_homepage.md\n\n$\n```\n\n#### Adding tags in the CLI\n\nSuppose you want to categorise your new test as applicable only to guest users (users who have not yet logged in).\n\nYou can add a tag to the test by running the `test tag add {tag-title}` command.\n\n```bash\n$ testmatic test tag add \"should_load_homepage\" \"guest user\"\n\n$\n```\n\n\u003e **Note:** We're using the test name `should_load_homepage` here. It's just the title condensed into one word using underlines: `_`. This makes it easier to copy/paste when needed.\n\nYour new tag \"Guest user\" should now have been created and adeed to the \"Homepage loads\" test.\n\nYou can verify this by running the `test show` command:\n\n```bash\n$ testmatic test show should_load_homepage\n\nShould load homepage\n====================\n\nDoc: ./.testmatic/tests/should_load_homepage.md\n\nSteps\n-----\n    STEP\n1   Navigate to homepage\n2   Observe homepage has loaded\n\nLinks\n-----\n(No items)\n\nTags\n----\nNAME         TITLE        DOC\nguest_user   Guest user   ./.testmatic/tags/guest_user.md\n\nRuns\n----\n(No items)\n\n$\n```\n\nYou can see that your new tag was added.\n\nIf you explore your local `.testmatic` folder you'll see that the test and tag files were created:\n\n- .testmatic\n  - tests\n    - should_load_homepage.md ⚡️\n  - tags\n    - guest_user.md ⚡️\n  - runs\n\n#### Finding tests by tag in the CLI\n\nSuppose you added a few other tests involving various kinds of users. But now you want to filter by only tests involving a guest user. You can filter the list of tests by the \"Guest user\" tag using `test list` command with the `--tag` switch.\n\n```bash\ntestmatic test list --tag \"Guest user\"\n\nTITLE                  NAME                   DOC\nShould load homepage   should_load_homepage   ./.testmatic/tests/should_load_homepage.md\n```\n\n### Advanced CLI usage\n\n#### Using Runs in the CLI\n\nWhen you perform a test, you might want to record certain details:\n\n- Date/time you performed the test\n- Result of the test – success, failure, mixed\n- Text and links\n- Screen recordings (videos, images, etc.)\n- Outputs (JSON or CSV files, etc.)\n\nTestmatic has a Runs feature to help you with this.\n\nEach test can have one or more runs.\n\nEach run has:\n\n- **One dated folder** containing a Markdown file and any other files you wish to include (screen recordings, outputs, etc.)\n- **One dated Markdown file** containing the date/time, text and links (if any)\n\nTo create a run, simply run the `run add` command, providing the test name (or title) as the first parameter:\n\n```bash\ntestmatic run add should_load_homepage\n```\n\n\u003e **Note:** You can optionally provide a date-time stamp in the format: `YYYY-MM-DD_HH-MM`. For example: `2024-10-01_11:30` for October 1, 2024 at 11:30 AM.\n\nThe new run folder and Markdown file should now have been created.\n\nYou can verify this using the `run show` command:\n\n```bash\n$ testmatic run show should_load_homepage\n\nShould load homepage – 24/3/2024 2:44\n=====================================\n\nFiles\n-----\nFILE\n./.testmatic/runs/should_load_homepage/2024-04-24_02-44/2024-04-24_02-44.md\n```\n\nA new folder and Markdown file will have been added:\n\n- .testmatic\n  - runs\n    - should_load_homepage\n      - 2024-04-24_02-44 ⚡️\n        - 2024-04-24_02-44.md ⚡️\n\nYou can open that folder in Finder (on Mac) using the `run open` command:\n\n```bash\ntestmatic run open should_load_homepage\n```\n\n\u003e **Note:** How does Testmatic know which run to show / open? It uses the latest by default. But if you prefer to target an older run, can provide an additional argument of the date/time stamp of the run to the `run show` or `run open` command. See [run show](#run-show), [run open](#run-open) for details.\n\n#### Using Links in the CLI\n\nYou might want to attach links to tests or tags.\n\nFor example:\n\n- Links to documentation\n- Links to web pages under test\n- Links to screenshots or screen recordings\n- Links to mock accounts\n\nYou can add/remove links manually by editing the test, tag or run Markdown files.\n\nTestmatic also includes `test link add` and `tag link add` commands, allowing you to quickly add links to tests or tags respectively without leaving the command line.\n\n```bash\n$ testmatic test link add should_load_homepage http://website.com/\n\n$ testmatic test show should_load_homepage\n\nShould load homepage\n====================\n\nDoc: ./.testmatic/tests/should_load_homepage.md\n\nSteps\n-----\n    STEP\n1   Navigate to homepage\n2   Observe homepage has loaded\n\nLinks\n-----\nNAME                  URL\nhttp://website.com/   http://website.com/\n```\n\n```bash\n$ testmatic tag link add guest_user http://test-accounts.com/guests\n\n$ testmatic tag show guest_user\n\nGuest user\n==========\n\nDoc: ./.testmatic/tags/guest_user.md\n\nLinks\n-----\nNAME                              URL\nhttp://test-accounts.com/guests   http://test-accounts.com/guests\n\nTests\n-----\nTITLE                  NAME                   DOC\nShould load homepage   should_load_homepage   ./.testmatic/tests/should_load_homepage.md\n```\n\n#### Using Impacts in the CLI\n\nYou might want to quickly find out, for a specific test or tag, which other tests or tags are related.\n\nFor example, some tests involving guest users might also involve other kinds of users or other systems, e.g. email notifications. So an error that causes failure of one test might cause failures of other related tests.\n\nTestmatic has an Impacts feature which shows you a graph of impacts of a given test or tag. You can use the `test impacts` command to view a test's impacts or the `tag impacts` command for a tag's impacts.\n\n```bash\n$ testmatic test impact should_load_homepage\n\nTest: Should load homepage - Impacts\n====================================\n\n--\u003e Homepage screen (homepage_screen) [tag]\n  --\u003e Footer (footer) [tag]\n    --\u003e Should show copyright info (should_show_copyright_info) [test]\n  --\u003e Header (header) [tag]\n    --\u003e Should show log out button (should_show_log_out_button) [test]\n--\u003e Homepage url (homepage_url) [tag]\n  --\u003e Should indicate logged in status after log in (should_indicate_logged_in_status_after_log_in) [test]\n    --\u003e Header login link (header_login_link) [tag]\n    --\u003e Login form (login_form) [tag]\n    --\u003e Valid existing user login credentials (valid_existing_user_login_credentials) [tag]\n    --\u003e Header logged in status (header_logged_in_status) [tag]\n```\n\n```bash\n$ testmatic tag impacts homepage_url\n\nTag: Homepage url - Impacts\n===========================\n\n--\u003e Should indicate logged in status after log in (should_indicate_logged_in_status_after_log_in) [test]\n--\u003e Should load homepage (should_load_homepage) [test]\n  --\u003e Homepage screen (homepage_screen) [tag]\n    --\u003e Footer (footer) [tag]\n      --\u003e Should show copyright info (should_show_copyright_info) [test]\n    --\u003e Header (header) [tag]\n      --\u003e Should show log out button (should_show_log_out_button) [test]\n```\n\n### Reference\n\n\u003c!-- insert cli-reference start --\u003e\n\n#### init\n\nUsage: `init`\n\nCreate a new project in the current working directory\n\n#### project create\n\nUsage: `project create`\n\nCreate a new project in the current working directory (same as `testmatic init`)\n\n#### test list\n\nUsage: `test list [options]`\n\nList tests in the current project\n\nOptions:\n\n\u003ctable\u003e\n\u003cthead\u003e\n  \u003ctr\u003e\n    \u003cth\u003eSyntax\u003c/th\u003e\n    \u003cth\u003eDescription\u003c/th\u003e\n  \u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n  \u003ctr\u003e\n\u003ctd\u003e\n\u003ccode\u003e-t\u003c/code\u003e,\u003cbr /\u003e\u003ccode\u003e --tag \u0026lt;value\u0026gt;\u003c/code\u003e\n\u003c/td\u003e\n\u003ctd\u003e\nFilter by tag\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n#### test add\n\nUsage: `test add [options]`\n\nAdd a new test to the project\n\nOptions:\n\n\u003ctable\u003e\n\u003cthead\u003e\n  \u003ctr\u003e\n    \u003cth\u003eSyntax\u003c/th\u003e\n    \u003cth\u003eDescription\u003c/th\u003e\n  \u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n  \u003ctr\u003e\n\u003ctd\u003e\n\u003ccode\u003e-t\u003c/code\u003e,\u003cbr /\u003e\u003ccode\u003e --title \u0026lt;value\u0026gt;\u003c/code\u003e\n\u003c/td\u003e\n\u003ctd\u003e\nTitle of the test.\nAlso used to generate an underscored filename used to refer to the test in short-hand.\nTitles must be unique.\nTitles should briefly summarise the test steps.\n\nRequired - must be provided, either via prompt or command line.\n\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\n\u003ccode\u003e-d\u003c/code\u003e,\u003cbr /\u003e\u003ccode\u003e --description \u0026lt;value\u0026gt;\u003c/code\u003e\n\u003c/td\u003e\n\u003ctd\u003e\nDescription of the test.\nLonger than the title, provides a more detailed summary of the test.\n\nTests can also include tags, enclosed in round brackets: (, ).\nFor further information, see 'testmatic tag help'.\n\nOptional.\n\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\n\u003ccode\u003e-s\u003c/code\u003e,\u003cbr /\u003e\u003ccode\u003e --steps [steps...]\u003c/code\u003e\n\u003c/td\u003e\n\u003ctd\u003e\nList of steps of the test.\n\nAdd each step in quotes separated by a space, e.g.: \"step one\" \"step two\"\nSteps will be in the order that they are provided.\n\nRequired - at least one step must be provided, either via prompt or command line.\n\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\n\u003ccode\u003e-l\u003c/code\u003e,\u003cbr /\u003e\u003ccode\u003e --links [links...]\u003c/code\u003e\n\u003c/td\u003e\n\u003ctd\u003e\nList of links to associate with the test.\nFor example, a deep link to the web page being tested or relevant documentation.\n\nAdd each link href in quotes separated by a space.\nE.g.: \"http://product.com/login\" \"http://wiki.com/login-flow\".\n\nLinks can be prefixed with text separated by a pipe \"|\".\nE.g. \"Login page|http://product.com/login\" \"Login flow docs|http://wiki.com/login-flow\"\n\nOptional.\n\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n#### test delete\n\nUsage: `test delete \u003ctestNameOrTitle\u003e`\n\nDelete a test\n\n#### test show\n\nUsage: `test show \u003ctestNameOrTitle\u003e`\n\nShow the full details of a test\n\n#### test link add\n\nUsage: `test link add [options] \u003ctestNameOrTitle\u003e \u003clinkHrefOrTitle\u003e`\n\nAdd a new link to a test\n\nOptions:\n\n\u003ctable\u003e\n\u003cthead\u003e\n  \u003ctr\u003e\n    \u003cth\u003eSyntax\u003c/th\u003e\n    \u003cth\u003eDescription\u003c/th\u003e\n  \u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n  \u003ctr\u003e\n\u003ctd\u003e\n\u003ccode\u003e-t\u003c/code\u003e,\u003cbr /\u003e\u003ccode\u003e --title \u0026lt;value\u0026gt;\u003c/code\u003e\n\u003c/td\u003e\n\u003ctd\u003e\nTitle of the new link.\n\nOptional.\n\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n#### test link delete\n\nUsage: `test link delete \u003ctestNameOrTitle\u003e \u003clinkHrefOrTitle\u003e`\n\nDelete a link from a test\n\n#### test link open\n\nUsage: `test link open \u003ctestNameOrTitle\u003e \u003clinkHrefOrTitle\u003e`\n\nOpen a test link in the browser\n\n#### tag list\n\nUsage: `tag list`\n\nList tags in the current project\n\n#### tag add\n\nUsage: `tag add [options]`\n\nAdd a new tag to the project\n\nOptions:\n\n\u003ctable\u003e\n\u003cthead\u003e\n  \u003ctr\u003e\n    \u003cth\u003eSyntax\u003c/th\u003e\n    \u003cth\u003eDescription\u003c/th\u003e\n  \u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n  \u003ctr\u003e\n\u003ctd\u003e\n\u003ccode\u003e-t\u003c/code\u003e,\u003cbr /\u003e\u003ccode\u003e --title \u0026lt;value\u0026gt;\u003c/code\u003e\n\u003c/td\u003e\n\u003ctd\u003e\nTitle of the tag.\nAlso used to generate an underscored filename used to refer to the test in short-hand.\nTitles must be unique.\nTitles should briefly describe the tag.\n\nRequired - must be provided, either via prompt or command line.\n\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\n\u003ccode\u003e-y\u003c/code\u003e,\u003cbr /\u003e\u003ccode\u003e --type \u0026lt;value\u0026gt;\u003c/code\u003e\n\u003c/td\u003e\n\u003ctd\u003e\nType of the tag.\nUsed to categorise one or more similar tags.\nE.g. \"page\" for tags that refer to a page in an website.\n\nOptional.\n\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\n\u003ccode\u003e-d\u003c/code\u003e,\u003cbr /\u003e\u003ccode\u003e --description \u0026lt;value\u0026gt;\u003c/code\u003e\n\u003c/td\u003e\n\u003ctd\u003e\nDescription of the test.\nLonger than the title, provides a more detailed description of the tag.\n\nOptional.\n\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\n\u003ccode\u003e-l\u003c/code\u003e,\u003cbr /\u003e\u003ccode\u003e --links [links...]\u003c/code\u003e\n\u003c/td\u003e\n\u003ctd\u003e\nList of links to attach to the tag.\nFor example, a deep link to the web page being tested or relevant documentation.\n\nAdd each link href in quotes separated by a space.\nE.g.: \"http://product.com/login\" \"http://wiki.com/login-flow\".\n\nLinks can be prefixed with text separated by a pipe \"|\".\nE.g. \"Login page|http://product.com/login\" \"Login flow docs|http://wiki.com/login-flow\"\n\nOptional.\n\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n#### tag delete\n\nUsage: `tag delete \u003ctagNameOrTitle\u003e`\n\nDelete a tag\n\n#### tag show\n\nUsage: `tag show \u003ctagNameOrTitle\u003e`\n\nShow the full details of a tag\n\n#### tag link add\n\nUsage: `tag link add [options] \u003ctagNameOrTitle\u003e \u003ctagLinkHref\u003e`\n\nAdd a new link to a tag\n\nOptions:\n\n\u003ctable\u003e\n\u003cthead\u003e\n  \u003ctr\u003e\n    \u003cth\u003eSyntax\u003c/th\u003e\n    \u003cth\u003eDescription\u003c/th\u003e\n  \u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n  \u003ctr\u003e\n\u003ctd\u003e\n\u003ccode\u003e-t\u003c/code\u003e,\u003cbr /\u003e\u003ccode\u003e --title \u0026lt;value\u0026gt;\u003c/code\u003e\n\u003c/td\u003e\n\u003ctd\u003e\nTitle of the new link.\n\nOptional.\n\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n#### tag link delete\n\nUsage: `tag link delete \u003ctagNameOrTitle\u003e \u003clinkHrefOrTitle\u003e`\n\nDelete a link from a tag\n\n#### tag link open\n\nUsage: `tag link open \u003ctagNameOrTitle\u003e \u003clinkHrefOrTitle\u003e`\n\nOpen a tag link in the browser\n\n#### tag type\n\nUsage: `tag type \u003ctagNameOrTitle\u003e \u003ctagType\u003e`\n\nSet the type of a tag\n\n#### tag impacts\n\nUsage: `tag impacts \u003ctagNameOrTitle\u003e`\n\nList the tests and tags that are impacted by a tag\n\n#### run show\n\nUsage: `run show \u003ctestNameOrTitle\u003e [runDateTime]`\n\nShow the full details of a run\n\n#### run open\n\nUsage: `run open \u003ctestNameOrTitle\u003e [runDateTime]`\n\nOpen a run folder\n\n\u003c!-- insert cli-reference end --\u003e\n\n## Topics\n\n### What is manual testing?\n\nManual testing is using an application yourself to ensure it works well.\n\nIt can be as simple as opening the app or website and clicking around to check that everything works.\n\nIn more complicated cases, it can involve systematic, rigorous testing, such as going through the entire sign-up flow to ensure that it works correctly.\n\n### Who does manual testing?\n\nSimple manual testing, such as clicking around, is often done by developers/engineers as the build the product.\n\nMore advanced testing is often outsourced to QA departments, who use sophisticated test case tools.\n\nHowever, as products become more complex, an increasing burden of testing falls on developers/engineers and others on the team.\n\nTestmatic fills this middle-ground between casual/informal testing and professional QA.\n\n### Benefits of manual testing\n\nThere is much agreement that **unit testing** is beneficial, but what about manual testing?\n\nManual testing provides unique benefits:\n\n- Manual testing helps you discover and document **intended behaviour** – how the system is **expected** to behave.\n- Manual test cases can be written **immediately**, without setting up test frameworks, etc.\n- Manual testing can be done in **any environment** you have access to.\n- Manual testing **puts you in the end-user's shoes**, letting you experience the system as an end-user would.\n- Manual testing lets you **verify complex, lengthy workflows**, which might be too difficult to automate.\n\n### Situations where manual testing is useful\n\nThe benefits make manual testing are highly useful in certain kinds of situations, such as:\n\n- Large and complicated applications\n- Poorly documented applications\n- Time-constrained businesses, e.g. a fast-growing startup\n- Cost-constrained businesses who cannot afford QA\n\n### Drawbacks of automated testing\n\nMuch of the industry has focussed on the benefits of automated testing and some have downplayed manual testing, claiming it to be time-consuming, inefficient and error-prone.\n\nThese criticisms may be strong in theory, but in practice they can ignore some important issues with automated testing:\n\n- Automated test coverage can be limited.\n- Automating all important flows can be time-consuming.\n- Automating certain kinds of behaviour can be very difficult. E.g. full user interface testing including observing smoothness, performance, accessibility, etc. or long complex flows involving multiple systems, both internal and external to the organisation, can be very difficult to automate.\n- Automating testing may limit incidental or unplanned but desirable discoveries. These would more likely be picked in manual testing. E.g. testing a login flow, one might discover a small UI glitch with the password entry field.\n\nIn most situations, the optimal choice is probably some combination of automated and manual testing.\n\nAutomated tests are useful and widely applicable, but not the \"one ring to rule them all\".\n\n### Collaboration in manual testing\n\nTraditionally, manual testing was mainly performed by QA engineers, using sophisticated test case tools.\n\nHowever, as products become more complex, an increasing burden of testing falls on developers/engineers and others on the team.\n\n- Engineers perform manual testing to understand what they're building, why they're building it, how people will use it, etc.\n- Engineers perform manual testing to ensure their code actually works as expected\n- Product owners perform manual testing to understand how people use the product, identify issues and bottlenecks and ideate on future directions for the product\n- Designers perform manual testing to see how the system looks and feels and identify areas for improvement, such as making the experience more consistent and cohesive\n\n### Benefits of everyone doing a little testing\n\nWhile specialisation is good for productivity, a blinkered or siloed approach to **testing** may not be optimal.\n\n- More (and a variety of) eyes on a product are more likely to uncover errors.\n- Errors (or important ones at least) eventually need to be fixed and this will likely involve engineers. It will likely be easier for engineers to fix errors if they already have a good grasp on key test cases and are generally capable testers themselves.\n- Fixes may come in various forms, not necessarily engineering. E.g. a small error in pricing might be cheaper to fix by simply adjusting copy in an email to alert users, rather than consuming expensive engineer-hours.\n- Engineers who understand test cases can gain a more holistic understanding of the system they are working on. Rather than just focussing on structure / code, engineers can think of how the software is being used and tailor their efforts accordingly. E.g. prioritisation of work, code structure, data structure selection, time estimates, long-term goal-setting.\n\n### Benefits of using Testmatic\n\nManual testing can be achieved in various ways with various systems.\n\nSome key benefits of using Testmatic for your manual tests are:\n\n- Fast and easy to create and update via CLI or text editor\n- Easy to share with business and tech people alike\n- Organised, structured and repeatable, enabling rigor and consistency in the testing effort\n- Searchable by tag, allowing second-order impacts of failures to be identified, tested and fixed\n- Can be used to generate automated unit test code scaffolding if desired (see: [Roadmap](#roadmap))\n\n#### Testing steps are simple and easy for people to read\n\nGiven/When/Then syntax is cumbersome and requires a learning curve and is generally the domain of software engineers.\n\nIn contrast, simple lists of testing steps, much like the method in a cooking recipe, are easy to understand for a broad set of team members – e.g. agile managers, product owners, QA, designers, engineers.\n\n#### Testing steps support simple, fast and rigorous manual testing\n\nIt's easier and faster to manually test software when you are clear on what specific actions need to be taken and in what order.\n\nBy documenting and organising testing procedures, manual testing can be performed consistently, ensuring a rigorously tested product.\n\n#### Testmatic tests can be incrementally automated\n\nTestmatic focusses on helping you write and organise your testing steps first, generating empty placeholder functions without you having to immediately write code to automate them.\n\nIf and when you decide to add automation, it's easy to locate the places in which to write code, and you can automate step by step, rather than having to automate a whole test sequence all at once.\n\n#### Testmatic tests are easy to share\n\nYou can share your test steps with team members and stakeholders easily.\n\nYou can link to a whole test, or a list of related tests.\n\n#### Testmatic tests can be hosted in version control, with zero third-party dependencies or additional setup\n\nThis makes it easy to get started quickly - simply fork the testmatic project, begin generating and committing your tests, and push to your own repository.\n\nIf your organisation has a version control system, and you have permission to create a new repository, you already have everything you need to get started.\n\n#### Testmatic tests are version-controlled and implemented in code\n\nAs your testmatic instance is a forked Git repository by default, you reap all the benefits of version control - tracking the history of changes, branching, ability to revert changes, etc.\n\nAs tests, steps and tags are stored internally as Typescript code, you can easily make modifications - large or small - using the standard tools of your IDE. For example, in VS Code, you can rename a token and automatically have it update all usages, by renaming the file with [Update imports on file move](https://code.visualstudio.com/docs/typescript/typescript-refactoring#_update-imports-on-file-move) and [Rename symbol](https://code.visualstudio.com/docs/editor/refactoring#_rename-symbol).\n\n#### Testmatic tags provide a powerful way to group related tests\n\nFor example, you can instantly retrieve a list of all tests for a particular screen, e.g. Login screen or Dashboard screen.\n\nOr you can instantly retrieve a list of tests that utilise a particular test account.\n\nThese lists can be conveniently linked from external repositories of information, such as a Solution design in a Wiki, a task tracking system or company chat.\n\nFor example, a wiki page for the Login screen could link to a testmatic doc listing all tests for that screen: http://github.com/myaccount/mytests/blob/main/docs/tags/login_screen.md.\n\n### FAQ\n\n#### Can I add screenshots or screencasts from my testing?\n\nYes!\n\nYou can add them to a run folder of your test or link to them from the run markdown file.\n\nSee the [Runs](#runs) section under [Advanced](#advanced).\n\n## Contributing\n\nYou can get started by **forking the testmatic repo**.\n\n### What you'll need\n\n- [Git](https://git-scm.com/)\n- [Node.js](https://nodejs.org/en/download/) version 16.14 or above:\n  - When installing Node.js, you are recommended to check all checkboxes related to dependencies.\n\n### Fork Testmatic\n\n1. On GitHub.com, navigate to the testmatic/testmatic repository.\n2. In the top-right corner of the page, click Fork.\n3. Under \"Owner,\" select the dropdown menu and click an owner for the forked repository.\n4. By default, forks are named the same as their upstream repositories. Optionally, to further distinguish your fork, in the \"Repository name\" field, type a name.\n5. Optionally, in the \"Description\" field, type a description of your fork.\n6. Optionally, select Copy the DEFAULT branch only.\n7. For many forking scenarios, such as contributing to open-source projects, you only need to copy the default branch. If you do not select this option, all branches will be copied into the new fork.\n8. Click Create fork.\n\nYou can then checkout to a local folder.\n\n```bash\ngit clone https://github.com/myname/mytestmatic\n```\n\nAnd install the dependencies using npm.\n\n```bash\nnpm install\n```\n\nCongratulations! You now have a working Testmatic project.\n\n### Roadmap\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003cth\u003eFuture\u003c/th\u003e\n    \u003ctd\u003eMore advanced querying, similarity search for tags, web-based UI, re-usability as a library, code generation framework.\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003cth\u003e28-May-2024\u003c/th\u003e\n    \u003ctd\u003eSimplification of core, leveraging of `commander` and `chalk` libraries, deferral of code generation, addition of runs and impacts feature.\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003cth\u003e13-Aug-2023\u003c/th\u003e\n    \u003ctd\u003eInitial release with core framework and basic support for tests, steps, tags, doc generation and querying of tests and steps.\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n## Troubleshooting\n\nNothing to write here so far. This section will be updated if and as needed.\n\n## Contact\n\nPlease feel free to contact Jonathan, the project maintainer.\n\n- Twitter - @conwy\n- Github - @jonathanconway\n- Email - jon@conwy.co\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonathanconway%2Ftestmatic","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjonathanconway%2Ftestmatic","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonathanconway%2Ftestmatic/lists"}