{"id":21471118,"url":"https://github.com/im-open/process-postman-test-results","last_synced_at":"2025-07-01T15:05:38.341Z","repository":{"id":38425707,"uuid":"392003466","full_name":"im-open/process-postman-test-results","owner":"im-open","description":"GitHub action that creates a status check or pull request comment based a json results file generated by the newman json reporter","archived":false,"fork":false,"pushed_at":"2024-10-31T20:08:47.000Z","size":579,"stargazers_count":1,"open_issues_count":0,"forks_count":4,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-11-11T22:19:08.262Z","etag":null,"topics":["continuous-integration","postman-tests","pr-comment","status-check","swat-team","test","testing"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/im-open.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2021-08-02T15:35:35.000Z","updated_at":"2024-03-27T15:57:20.000Z","dependencies_parsed_at":"2024-02-29T00:29:26.575Z","dependency_job_id":"ffb1289a-4ba0-4bf1-bc04-4a02108c1d0a","html_url":"https://github.com/im-open/process-postman-test-results","commit_stats":{"total_commits":28,"total_committers":6,"mean_commits":4.666666666666667,"dds":0.3214285714285714,"last_synced_commit":"514e046613ce7a36bee9afa9d9c5b5da5d9c4507"},"previous_names":[],"tags_count":22,"template":false,"template_full_name":"im-open/javascript-action-template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/im-open%2Fprocess-postman-test-results","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/im-open%2Fprocess-postman-test-results/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/im-open%2Fprocess-postman-test-results/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/im-open%2Fprocess-postman-test-results/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/im-open","download_url":"https://codeload.github.com/im-open/process-postman-test-results/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226027910,"owners_count":17562134,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["continuous-integration","postman-tests","pr-comment","status-check","swat-team","test","testing"],"created_at":"2024-11-23T09:31:34.789Z","updated_at":"2024-11-23T09:32:02.716Z","avatar_url":"https://github.com/im-open.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# process-postman-test-results\n\nThis action works in conjunction with another step that runs Postman tests. That step should include a json reporter like the following npm script:\n`\"postman\": \"newman run postman_collection.json -r cli,json -reporter-json-export postman-results.json\"`. \n\nThis action takes the json file and creates a Status Check or PR Comment depending on the flags set with the test outcome. This action does not execute the tests itself, it relies on a previous step to run the Postman tests. It can only process one result file.\n\n## Index \u003c!-- omit in toc --\u003e\n\n- [process-postman-test-results](#process-postman-test-results)\n - [Failures](#failures)\n - [Limitations](#limitations)\n - [Action Outputs](#action-outputs)\n - [Pull Request Comment](#pull-request-comment)\n - [Pull Request Status Check](#pull-request-status-check)\n - [Workflow Run](#workflow-run)\n - [Failed Test Details](#failed-test-details)\n - [Inputs](#inputs)\n - [Outputs](#outputs)\n - [Usage Examples](#usage-examples)\n - [npm setup](#npm-setup)\n - [Workflow](#workflow)\n - [Contributing](#contributing)\n - [Incrementing the Version](#incrementing-the-version)\n - [Source Code Changes](#source-code-changes)\n - [Recompiling Manually](#recompiling-manually)\n - [Updating the README.md](#updating-the-readmemd)\n - [Code of Conduct](#code-of-conduct)\n - [License](#license)\n\n## Failures\n\nThe test status \u0026 action's conclusion can be viewed in multiple places:\n\n- In the body of a PR comment this action generates\n- Next to the name of one of the status checks under the `Checks` section of a PR\n- Next to the name of one of the status checks under the `Jobs` section of the workflow run\n- In the body of a status check listed on the workflow run\n\nIf the test results contain failures, the status check's conclusion will be set to `failure`. If the status check is required and its conclusion is `failure` the PR cannot be merged. If this required status check behavior is not desired, the `ignore-test-failures` input can be set and the conclusion will be marked as `neutral` if test failures are detected. The status badge that is shown in the comment or status check body will still indicate it was a `failure` though.\n\n## Limitations\n\nGitHub does have a size limitation of 65535 characters for a Status Check body or a PR Comment. This action would fail if the test results exceeded the GitHub [limit]. To mitigate this size issue only details for failed tests are included in the output in addition to a badge, duration info and outcome info. If the comment still exceeds that size, it will be truncated with a note to see the remaining output in the log.\n\nIf you have multiple workflows triggered by the same `pull_request` or `push` event, GitHub creates one checksuite for that commit. The checksuite gets assigned to one of the workflows randomly and all status checks for that commit are reported to that checksuite. That means if there are multiple workflows with the same trigger, your status checks may show on a different workflow run than the run that created them.\n\n## Action Outputs\n\n### Pull Request Comment\n\nThis is shown on the pull request when the `create-pr-comment` is set to `true` and there is a PR associated with the commit.\n\u003ckbd\u003e\u003cimg src=\"./docs/pr_comment.png\"\u003e\u003c/img\u003e\u003c/kbd\u003e\n\n### Pull Request Status Check\n\nThis is shown on the pull request when the `create-status-check` is set to `true` and there is a PR associated with the commit.\n\u003ckbd\u003e\u003cimg src=\"./docs/pr_status_check.png\"\u003e\u003c/img\u003e\u003c/kbd\u003e\n\n### Workflow Run\n\nThis is shown on the workflow run when the `create-status-check` is set to `true`.\n\u003ckbd\u003e\u003cimg src=\"./docs/workflow_status_check.png\"\u003e\u003c/img\u003e\u003c/kbd\u003e\n\n### Failed Test Details\n\nFor failed test runs you can expand each failure to view more details.\n\u003ckbd\u003e\u003cimg src=\"./docs/failed_tests.png\"\u003e\u003c/img\u003e\u003c/kbd\u003e\n\n## Inputs\n\n| Parameter | Is Required | Default | Description |\n|--------------------------------|-------------|--------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `github-token` | true | N/A | Used for the GitHub Checks API. Value is generally: `secrets.GITHUB_TOKEN`. |\n| `results-file` | true | N/A | The json test results file generated by the postman/newman json reporter. |\n| `report-name` | false | `Postman Test Results` | The desired name of the report that is shown on the PR Comment and inside the Status Check. |\n| `create-status-check` | false | `true` | Flag indicating whether a status check with postman test results should be generated. |\n| `ignore-test-failures` | false | `false` | If there are test failures, the check's conclusion is set to `neutral` so it will not block pull requests.\u003cbr/\u003e\u003cbr/\u003e*Only applicable when `create-status-check` is true.* |\n| `create-pr-comment` | false | `true` | This flag determines whether a new comment is created or if the action updates an existing comment (*if one is found*).\u003cbr/\u003e\u003cbr/\u003e*Only applicable when `create-pr-comment` is true.* |\n| `update-comment-if-one-exists` | false | `true` | When `create-pr-comment` is true, this flag determines whether a new comment is created or if the action updates an existing comment if one is found which is the default behavior. |\n| `comment-identifier` | false | `${{ env.GITHUB-JOB }}_${{ env.GITHUB-ACTION }}` | A unique identifier which will be added to the generated markdown as a comment (*it will not be visible in the PR comment*).\u003cbr/\u003e\u003cbr/\u003eThis identifier enables creating then updating separate results comments on the PR if more than one instance of this action is included in a single job. This can be helpful when there are multiple test projects that run separately but are part of the same job.\u003cbr/\u003e\u003cbr/\u003e*Only applicable when `create-pr-comment` is true.* |\n| `timezone` | false | `UTC` | IANA time zone name (e.g. America/Denver) to display dates in. |\n\n## Outputs\n\n| Output | Description |\n|--------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `test-outcome` | Test outcome based on presence of failing tests: *Failed,Passed*\u003cbr/\u003eIf exceptions are thrown or if it exits early because of argument errors, this is set to Failed. |\n| `test-results-truncated` | Flag indicating whether test results were truncated due to markdown exceeding character limit of 65535. |\n| `test-results-file-path` | File path for the file that contains the pre-truncated test results in markdown format. This is the same output that is posted in the PR comment. |\n| `status-check-id` | The ID of the Status Check that was created. This is only set if `create-status-check` is `true` and a status check was created successfully. |\n| `pr-comment-id` | The ID of the PR comment that was created. This is only set if `create-pr-comment` is `true` and a PR was created successfully. |\n\n## Usage Examples\n\n### npm setup\n\n1. Some npm scripts have also been added to the project\n\n ```json\n \"scripts\": {\n \"postman\": \"newman run postman_collection.json -r cli,json -reporter-json-export postman-results.json\"\n }\n ```\n\n### Workflow\n\n```yml\nenv:\n PACKAGE_JSON_DIR: 'src' # The directory containing package.json\n POSTMAN_NPM_SCRIPT_NAME: 'postman' # The name of the postman script in package.json\n POSTMAN_RESULTS_NAME: 'postman-results.json' # The name of the results file set in npm script in package.json\n\njobs:\n run-postman:\n runs-on: ubuntu-20.04\n\n defaults:\n run:\n shell: bash\n working-directory: ${{ env.PACKAGE_JSON_DIR }}\n\n steps:\n - uses: actions/checkout@v4\n\n - name: Restore npm packages\n run: npm install\n\n - name: Run Postman Tests\n continue-on-error: true\n run: npm run ${{ env.POSTMAN_NPM_SCRIPT_NAME }}\n\n - name: Create Status check based on postman results\n id: process-postman\n # You may also reference the major or major.minor version\n uses: im-open/process-postman-test-results@v2.3.0\n with:\n github-token: ${{ secrets.GITHUB_TOKEN }}\n results-file: ${{env.PACKAGE_JSON_DIR }}/${{ env.POSTMAN_RESULTS_NAME }}\n report-name: 'Postman ${{ github.run_number }}' # Default: Postman Test Results\n create-status-check: true # Default: true\n create-pr-comment: false # Default: true\n update-comment-if-one-exists: false # Default: true\n ignore-test-failures: true # Default: false\n timezone: 'america/denver' # Default: UTC\n \n - name: Fail if there were errors in the postman tests\n if: steps.process-postman.outputs.test-outcome == 'Failed'\n run: |\n echo \"There were postman failures.\"\n exit 1\n```\n\n## Contributing\n\nWhen creating PRs, please review the following guidelines:\n\n- [ ] The action code does not contain sensitive information.\n- [ ] At least one of the commit messages contains the appropriate `+semver:` keywords listed under [Incrementing the Version] for major and minor increments.\n- [ ] The action has been recompiled. See [Recompiling Manually] for details.\n- [ ] The README.md has been updated with the latest version of the action. See [Updating the README.md] for details.\n\n### Incrementing the Version\n\nThis repo uses [git-version-lite] in its workflows to examine commit messages to determine whether to perform a major, minor or patch increment on merge if [source code] changes have been made. The following table provides the fragment that should be included in a commit message to active different increment strategies.\n\n| Increment Type | Commit Message Fragment |\n|----------------|---------------------------------------------|\n| major | +semver:breaking |\n| major | +semver:major |\n| minor | +semver:feature |\n| minor | +semver:minor |\n| patch | *default increment type, no comment needed* |\n\n### Source Code Changes\n\nThe files and directories that are considered source code are listed in the `files-with-code` and `dirs-with-code` arguments in both the [build-and-review-pr] and [increment-version-on-merge] workflows. \n\nIf a PR contains source code changes, the README.md should be updated with the latest action version and the action should be recompiled. The [build-and-review-pr] workflow will ensure these steps are performed when they are required. The workflow will provide instructions for completing these steps if the PR Author does not initially complete them.\n\nIf a PR consists solely of non-source code changes like changes to the `README.md` or workflows under `./.github/workflows`, version updates and recompiles do not need to be performed.\n\n### Recompiling Manually\n\nThis command utilizes [esbuild] to bundle the action and its dependencies into a single file located in the `dist` folder. If changes are made to the action's [source code], the action must be recompiled by running the following command:\n\n```sh\n# Installs dependencies and bundles the code\nnpm run build\n```\n\n### Updating the README.md\n\nIf changes are made to the action's [source code], the [usage examples] section of this file should be updated with the next version of the action. Each instance of this action should be updated. This helps users know what the latest tag is without having to navigate to the Tags page of the repository. See [Incrementing the Version] for details on how to determine what the next version will be or consult the first workflow run for the PR which will also calculate the next version.\n\n## Code of Conduct\n\nThis project has adopted the [im-open's Code of Conduct](https://github.com/im-open/.github/blob/main/CODE_OF_CONDUCT.md).\n\n## License\n\nCopyright \u0026copy; 2024, Extend Health, LLC. Code released under the [MIT license](LICENSE).\n\n\u003c!-- Links --\u003e\n[Incrementing the Version]: #incrementing-the-version\n[Recompiling Manually]: #recompiling-manually\n[Updating the README.md]: #updating-the-readmemd\n[source code]: #source-code-changes\n[usage examples]: #usage-examples\n[build-and-review-pr]: ./.github/workflows/build-and-review-pr.yml\n[increment-version-on-merge]: ./.github/workflows/increment-version-on-merge.yml\n[esbuild]: https://esbuild.github.io/getting-started/#bundling-for-node\n[git-version-lite]: https://github.com/im-open/git-version-lite\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fim-open%2Fprocess-postman-test-results","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fim-open%2Fprocess-postman-test-results","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fim-open%2Fprocess-postman-test-results/lists"}