Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/ctrf-io/newman-reporter-ctrf-json
A Newman JSON test results reporter that follows the CTRF schema
https://github.com/ctrf-io/newman-reporter-ctrf-json
newman newman-reporter postman test-automation test-reporting
Last synced: 2 months ago
JSON representation
A Newman JSON test results reporter that follows the CTRF schema
- Host: GitHub
- URL: https://github.com/ctrf-io/newman-reporter-ctrf-json
- Owner: ctrf-io
- License: mit
- Created: 2024-01-30T18:46:33.000Z (12 months ago)
- Default Branch: main
- Last Pushed: 2024-08-07T20:05:52.000Z (6 months ago)
- Last Synced: 2024-10-30T03:38:06.535Z (3 months ago)
- Topics: newman, newman-reporter, postman, test-automation, test-reporting
- Language: TypeScript
- Homepage: https://ctrf.io
- Size: 262 KB
- Stars: 3
- Watchers: 1
- Forks: 0
- Open Issues: 6
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
# Newman Postman JSON test results report
> Save Newman Postman test results as a JSON file
A Postman newman JSON test reporter to create test reports that follow the CTRF standard.
[Common Test Report Format](https://ctrf.io) ensures the generation of uniform JSON test reports, independent of programming languages or test framework in use.
## Help us grow CTRF
⭐ **If you find this project useful, please consider following the [CTRF organisation](https://github.com/ctrf-io) and giving this repository a star** ⭐
**It means a lot to us and helps us grow this open source library.**
## Features
- Generate JSON test reports that are [CTRF](https://ctrf.io) compliant
- Straightforward integration with newman
```json
{
"results": {
"tool": {
"name": "newman"
},
"summary": {
"tests": 1,
"passed": 1,
"failed": 0,
"pending": 0,
"skipped": 0,
"other": 0,
"start": 1706828654274,
"stop": 1706828655782
},
"tests": [
{
"name": "ctrf should generate the same report with any tool",
"status": "passed",
"duration": 100
}
],
"environment": {
"appName": "MyApp",
"buildName": "MyBuild",
"buildNumber": "1"
}
}
}
```
## What is CTRF?
CTRF is a universal JSON test report schema that addresses the lack of a standardized format for JSON test reports.
**Consistency Across Tools:** Different testing tools and frameworks often produce reports in varied formats. CTRF ensures a uniform structure, making it easier to understand and compare reports, regardless of the testing tool used.
**Language and Framework Agnostic:** It provides a universal reporting schema that works seamlessly with any programming language and testing framework.
**Facilitates Better Analysis:** With a standardized format, programatically analyzing test outcomes across multiple platforms becomes more straightforward.
## Installation
```bash
npm install newman-reporter-ctrf-json
```
Run your tests with the reporter argument via the cli:
```bash
newman run ./postman_collection.json -r cli,ctrf-json
```
or programmatically:
```js
const newman = require('newman') // require newman in your project
// call newman.run to pass `options` object and wait for callback
newman.run(
{
collection: require('./sample-collection.json'),
reporters: ['cli', 'ctrf-json'],
},
function (err) {
if (err) {
throw err
}
console.log('collection run complete!')
}
)
```
You'll find a JSON file named `ctrf-report.json` in the `ctrf` directory.
## Reporter Options
The reporter supports several configuration options, you can pass these via the command line:
```bash
newman run ./postman_collection.json -r cli,ctrf-json \
--reporter-ctrf-json-output-file custom-name.json \
--reporter-ctrf-json-output-dir custom-directory \
--reporter-ctrf-json-test-type api \
--reporter-ctrf-json-minimal false \
--reporter-ctrf-json-app-name MyApp \
--reporter-ctrf-json-app-version 1.0.0 \
--reporter-ctrf-json-os-platform linux \
--reporter-ctrf-json-os-release 18.04 \
--reporter-ctrf-json-os-version 5.4.0 \
--reporter-ctrf-json-build-name MyApp \
--reporter-ctrf-json-build-number 100 \
--reporter-ctrf-json-build-url https://ctrf.io \
--reporter-ctrf-json-repository-name ctrf \
--reporter-ctrf-json-repository-url https://github.com/ctrf-io/newman-reporter-ctrf-json \
--reporter-ctrf-json-branch-name main \
--reporter-ctrf-json-test-environment staging
```
or programmatically:
```js
const newman = require('newman') // require newman in your project
// call newman.run to pass `options` object and wait for callback
newman.run(
{
collection: require('./sample-collection.json'),
reporters: ['cli', 'ctrf-json'],
reporter: {
'ctrf-json': {
outputFile: 'api_report_ctrf.json',
outputDir: 'api_reports',
minimal: true,
testType: 'api',
appName: 'MyApp',
appVersion: '1.0.0',
osPlatform: 'linux',
osRelease: '18.04',
osVersion: '5.4.0',
buildName: 'MyApp',
buildNumber: '100',
buildUrl: 'https://ctrf.io',
repositoryName: 'ctrf',
repositoryUrl: 'https://github.com/ctrf-io/newman-reporter-ctrf-json',
branchName: 'main',
testEnvironment: 'staging',
},
},
},
function (err) {
if (err) {
throw err
}
console.log('collection run complete!')
}
)
```
## Test Object Properties
The test object in the report includes the following [CTRF properties](https://ctrf.io/docs/schema/test):
| Name | Type | Required | Details |
| ---------- | ------ | -------- | ----------------------------------------------------------------------------------- |
| `name` | String | Required | The name of the test. |
| `status` | String | Required | The outcome of the test. One of: `passed`, `failed`, `skipped`, `pending`, `other`. |
| `duration` | Number | Required | The time taken for the test execution, in milliseconds. |
| `message` | String | Optional | The failure message if the test failed. |
| `trace` | String | Optional | The stack trace captured if the test failed. |
| `suite` | String | Optional | The suite or group to which the test belongs. |
| `type` | String | Optional | The type of test (e.g., `api`, `contract`). |
## Support Us
If you find this project useful, consider giving it a GitHub star ⭐ It means a lot to us.