{"id":14979992,"url":"https://github.com/aws/universal-test-runner","last_synced_at":"2025-04-10T06:09:49.359Z","repository":{"id":65517203,"uuid":"506788887","full_name":"aws/universal-test-runner","owner":"aws","description":":runner: A universal test runner for any language and test framework.","archived":false,"fork":false,"pushed_at":"2024-09-30T13:13:51.000Z","size":1513,"stargazers_count":28,"open_issues_count":18,"forks_count":5,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-01T16:54:01.207Z","etag":null,"topics":["dotnet","gradle","jest","maven","pytest","test-runner","testing"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aws.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-06-23T20:55:12.000Z","updated_at":"2024-11-06T16:54:40.000Z","dependencies_parsed_at":"2024-05-02T14:47:14.980Z","dependency_job_id":"d12284c7-8284-4bc5-8a5d-1203c7998c73","html_url":"https://github.com/aws/universal-test-runner","commit_stats":{"total_commits":317,"total_committers":10,"mean_commits":31.7,"dds":"0.48895899053627756","last_synced_commit":"b28ac65a8e7239d69a7378af1bcc3c8183cab151"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aws%2Funiversal-test-runner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aws%2Funiversal-test-runner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aws%2Funiversal-test-runner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aws%2Funiversal-test-runner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aws","download_url":"https://codeload.github.com/aws/universal-test-runner/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248166925,"owners_count":21058481,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dotnet","gradle","jest","maven","pytest","test-runner","testing"],"created_at":"2024-09-24T14:01:04.591Z","updated_at":"2025-04-10T06:09:49.341Z","avatar_url":"https://github.com/aws.png","language":"TypeScript","readme":"# universal-test-runner\n\n[![npm version](https://img.shields.io/npm/v/@aws/universal-test-runner)](https://www.npmjs.com/package/@aws/universal-test-runner)\n[![npm downloads](https://img.shields.io/npm/dm/@aws/universal-test-runner)](https://npm-stat.com/charts.html?package=%40aws%2Funiversal-test-runner)\n[![build status](https://github.com/aws/universal-test-runner/actions/workflows/build.yml/badge.svg)](https://github.com/aws/universal-test-runner/actions/workflows/build.yml)\n[![codecov](https://codecov.io/github/aws/universal-test-runner/branch/main/graph/badge.svg?token=6TIM7H9HC5)](https://codecov.io/github/aws/universal-test-runner)\n\nA universal test runner for any language and test framework.\n\n**❗️ NB: universal-test-runner is currently working towards a 1.0.0 release.\nSee the [1.0.0 milestone](https://github.com/aws/universal-test-runner/milestone/1) \nfor all progress towards 1.0.0. ❗️**\n\n## 🌎 What is universal-test-runner?\n\nuniversal-test-runner is a command-line tool that uses the [Test Execution\nProtocol](./protocol/README.md) to run tests for any programming language and\nany test framework.\n\n**Installation**: Install globally using npm:\n\n```\n$ npm install -g @aws/universal-test-runner \n```\n\nNow the `run-tests` executable is available to be used.\n\n**Usage**: For example, to run a single test named \"test1\" in a\nproject using [jest](https://jestjs.io/), you can run the following:\n\n```\n$ export TEP_TESTS_TO_RUN=\"test1\" # set by IDE or CI/CD system\n\n$ run-tests jest\nPASS  ./index.test.js\n  Suite1\n    ✓ test1 (2 ms)\n    ○ skipped test2\n    ○ skipped test3\n\nTest Suites: 1 passed, 1 total\nTests:       2 skipped, 1 passed, 3 total\nSnapshots:   0 total\nTime:        0.288 s, estimated 1 s\nRan all test suites with tests matching \"test1\".\n```\n\nHow about running a test named \"test1\", but for a project using\n[pytest](https://pytest.org)? Easy -- we can use the same command!\n\n```\n$ export TEP_TESTS_TO_RUN=\"test1\" # set by IDE or CI/CD system\n\n$ run-tests pytest\n================== test session starts =====================\nplatform darwin -- Python 3.10.9, pytest-7.1.3, pluggy-1.0.0\ncollected 3 items / 2 deselected / 1 selected\n\ntest_example.py .\n```\n\nAs shown in these examples, the Test Execution Protocol is used to establish a\nunified interface for passing arguments to different test frameworks and\nrunners.  Check out [RFC 0001](./protocol/rfcs/0001/README.md) for the\nmotivation behind universal-test-runner and the Test Execution Protocol, and\nwhy having a common interface is so useful.\n\n## 🤔 When should I use universal-test-runner?\n\nYou should install universal-test-runner in the following cases:\n\n* Your IDE or CI/CD system tells you to, in order for it to support running tests according to the [Test Execution Protocol](./protocol/README.md)\n* You're developing an adapter for universal-test-runner, and you want to test your adapter\n* You're writing an IDE plugin or CI/CD integration that implements the Test Execution Protocol, and you need a protocol-aware runner to test your integration\n\n## 📈 Framework and build tool support\n\nFirst-party test adapter support is provided for the following frameworks/build tools:\n\n* jest: https://jestjs.io/\n* pytest: https://pytest.org\n* maven: https://maven.apache.org/\n* gradle: https://gradle.org/\n* dotnet: https://learn.microsoft.com/en-us/dotnet/core/tools/\n\nSee the [1.0.0 milestone](https://github.com/aws/universal-test-runner/milestone/1)\nfor all frameworks and build tools we plan to support for v1.0.0.\n\n## 📦 Packages in this monorepo\n\nThe only package you should install and depend on is\n[`@aws/universal-test-runner`](./packages/universal-test-runner),\nwhich follows [semantic versioning](https://semver.org/).\n\nThe other packages are either internal utilities or adapters that have unstable\nAPIs and won't necessarily follow semver. You should avoid depending on them\ndirectly.\n\n## 🏰 Architecture\n\nuniversal-test-runner is a \"Test Execution Protocol-aware\" runner that uses an\nadapter model to provide support for test frameworks.  The runner itself is not\naware of any frameworks, but delegates to the appropriate adapter in order to\nexecute tests for a specific framework.  For more details on the architecture,\nsee the following documentation:\n* [RFC 1](./protocol/rfcs/0001/README.md) for an explanation of the Test Execution Protocol\n* universal-test-runner [architecture documentation](./docs/architecture.md), for a description of the runner, the adapters, and how they interact\n\n## 🔋 Custom adapters\n\nIt's possible to write custom adapters and pass them to universal-test-runner,\nproviding support for new frameworks or custom testing setups. See the docs on\n[writing custom adapters](#-writing-adapters) for how to implement one.\n\nIf you write a custom adapter, please host it in its own GitHub repo and\npublish it to npm; then [open a pull request](https://github.com/aws/universal-test-runner/compare)\nto add it to our [list of known third-party adapters](./docs/third-party-adapters.md),\nso everyone can benefit. (Note that we won't be adding the source code of\nthird-party adapters directly to this repo.)\n\nExample of using a third-party adapter from npm:\n\n```\nnpm install -g my-awesome-adapter\nrun-tests my-awesome-adapter\n```\n\nIf you have a specific project setup that you don't think merits a generic\nthird-party adapter, you can pass an adapter from a local file:\n\n```\nrun-tests ./my-local-adapter.js\n```\n\nIf universal-test-runner doesn't suit your needs exactly, you can use it as an\nexample of how to write your own Test Execution Protocol-aware runner. See the\n[writing custom runners](#-custom-runners) and the \n[Test Execution Protocol](./protocol/README.md) docs for more info.\n\n## 👩‍💻 Writing adapters\n\nTest adapters are responsible for executing tests as specified by the Test\nExecution Protocol, and reporting the status of the test execution back to\nuniversal-test-runner. The runner will do all the work of parsing the protocol\nenvironment variables, and then invoke the `executeTests` function exposed by\nthe adapter.\n\n* The `executeTests` function must accept an input object of type [`AdapterInput`](./packages/universal-test-runner-types/src/index.ts)\n* The `executeTests` function must return an ouput object of type [`AdapterOutput`](./packages/universal-test-runner-types/src/index.ts) or `Promise\u003cAdapterOutput\u003e`.\n  * If the adapter executes the tests successfully, and the test run passes, `executeTests` should return an exitCode of `0` (or a promise resolved with an exitCode of `0`).\n  * If the adapter executes the tests successfully, and the test run fails, `executesTests` should return a non-zero exitCode (or a promise resolved with a non-zero exitCode).\n  * If the adapter cannot execute the tests due to an unrecoverable error, `executeTests` should throw an error (or return a rejected promise).\n\nTwo simple adapters are shown below, with the details of test execution omitted:\n\n```javascript\n// adapter.js\nexport function executeTests({ testNamesToRun }) {\n  const pass = doTestExecution(testNamesToRun)\n  return { exitCode: pass ? 0 : 1 }\n}\n```\n\n```javascript\n// adapter.js\nexport function executeTests({ testNamesToRun }) {\n  return new Promise((resolve, reject) =\u003e {\n    doTestExecution(testNamesToRun, (err, pass) =\u003e {\n      if (err) {\n        return reject(err)\n      }\n      resolve({ exitCode: pass ? 0 : 1 });\n    })\n  })\n}\n```\n\nThe adapter is passed to the runner as follows:\n\n```\nrun-tests ./adapter.js\n```\n\nAdapters can also accept a second argument called `context` of type\n[`RunnerContext`](./packages/universal-test-runner-types/src/index.ts):\n- `context.cwd`: prefer this value over using `process.cwd()` in your adapter. This allows the runner to execute the adapter in a different working directory from where the runner is executed, if needed, while still allowing the adapter to produce any artifacts (like reports) in the correct location.\n- `context.extraArgs`: Any unparsed, tokenized values passed to the runner after the end-of-argument marker `--`. Allows adapters to accommodate arbitrary flags being passed through the runner to the adapter.\n  - For example, you could pass a custom jest config to the jest adapter by running `run-tests jest -- --config ./path/to/config.js`\n- `context.logLevel`: The log level passed by the user to the runner when invoked from the command line, of type [`LogLevel`](./packages/universal-test-runner-types/src/index.ts). If adapters log anything when being executed, they should set the log level of their logger according to this value, where level rank from highest to lowest is `error`, `warn`, `info`, `debug`. Logs should only be written if their level rank is at least the rank of the specified log level, e.g. if the log level is `warn`, only logs of rank `error` and `warn` should be written.\n\nHere's an abridged example of an adapter using context:\n\n```javascript\nconst path = require('path')\n\nexport function executeTests({ testNamesToRun }, { cwd, extraArgs, logLevel }) {\n  // Use the log level specified by the runner\n  logger.setLogLevel(context.logLevel)\n\n  logger.info('Running tests...')\n\n  // Pass unparsed args to the underlying framework, if the adapter needs to support arbitrary user flags\n  const [pass, report] = doTestExecution(extraArgs, testNamesToRun)\n\n  logger.info('Running writing report...')\n\n  // If cwd is needed, prefer context.cwd over process.cwd()\n  report.write(path.join(cwd, 'reports', 'junit.xml'))\n\n  if (!pass) {\n    logger.error('Tests failed!')\n  }\n\n  return { exitCode: pass ? 0 : 1 }\n}\n```\n\n### Publishing adapters to npm\n\nStructure your adapter as described above, and make sure the `main` field in\nyour package.json points to a file that exports an `executeTests` function.\n\n## 🏃‍♀️ Custom runners\n\nTODO\n\n## Node.js support\n\nAll Active and Maintenance LTS versions of Node.js versions are supported.\nPlease see the Node.js [release schedule](https://github.com/nodejs/release#release-schedule)\nfor full details on the LTS process. We'll support the newest Current versions\nscheduled for LTS one month after their first release, and maintain support for\nold Maintenance versions for at least one month after they go end-of-life.\n\nFor example, as of writing, Node.js 14 and 16 are Maintenance LTS releases, and\nNode.js 18 is Active LTS, so all three are supported. Once Node.js 14 goes\nend-of-life in May 2023, universal-test-runner will support it until June 2023,\nafter which it will be removed from CI builds. Node.js 20 will become Current\nin April 2023, and LTS in October 2023, so universal-test-runner will support\nit as early as June 2023.\n\n(In the case that universal-test-runner's dependencies don't permit extended\nMaintenance support or early Current support, these one-month paddings may not\nbe possible, e.g. Jest drops support for Node.js 14 as soon as it goes\nend-of-life, or can't run on Node.js 20 until it goes LTS.)\n\n## 🔀 Contributing\n\nPlease see the [contributing guide](./CONTRIBUTING.md) for all the logistical\ndetails. Read the existing issues and pull requests before starting to make any\ncode changes; large changes that haven't been discussed will be rejected\noutright in favour of small, incremental changes, so please open an issue early\nto discuss anything that may be non-trivial before development starts.\n\nAll changes to the Test Execution Protocol must follow the [RFC process](./protocol/README.md).\n\n### Local development setup\n\n[Fork](https://github.com/aws/universal-test-runner/fork) the repository, and then clone your fork:\n\n```\ngit clone https://github.com/\u003cUSERNAME\u003e/universal-test-runner\ncd universal-test-runner\n```\n\nMake sure you're using the correct Node.js version (install nvm [here](https://github.com/nvm-sh/nvm) if needed):\n\n```\nnvm use\n```\n\n(Note that npm@8 or greater is required since this project uses \n[npm workspaces](https://docs.npmjs.com/cli/v9/using-npm/workspaces?v=true).\nNode.js 16 and up by default ship with versions of npm that support \nworkspaces.)\n\nInstall dependencies and build code:\n\n```\nnpm install\nnpm run compile\n```\n\nRun tests\n\n```\nnpm test\n```\n\nRun integration tests (you may have to install some of the frameworks and build tools manually to get these to pass locally):\n\n```\nnpm run test:integ\n```\n\nMake your changes and commit them. This project follows the \n[conventional commits](https://www.conventionalcommits.org/en/v1.0.0/#specification)\nspecification in order to support automatic [semantic versioning](https://semver.org/) \nand changelog generation, so a commit message hook will verify that you've\nformatted your commit message correctly. To run the pre-commit hook, you'll\nhave to install [TruffleHog](https://github.com/trufflesecurity/trufflehog).\nPush the changes to your fork, and open a pull request.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faws%2Funiversal-test-runner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faws%2Funiversal-test-runner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faws%2Funiversal-test-runner/lists"}