{"id":15091475,"url":"https://github.com/allenheltondev/postman-contract-test-generator","last_synced_at":"2025-10-03T23:05:42.284Z","repository":{"id":38190631,"uuid":"302622150","full_name":"allenheltondev/postman-contract-test-generator","owner":"allenheltondev","description":"Postman collection and environment that will take an Open API Spec, validate component adherence, generate contract tests, and execute them.","archived":false,"fork":false,"pushed_at":"2024-01-12T10:13:06.000Z","size":309,"stargazers_count":120,"open_issues_count":11,"forks_count":34,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-08-23T02:57:23.557Z","etag":null,"topics":["oas","postman","schema-test"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/allenheltondev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-10-09T11:32:34.000Z","updated_at":"2025-04-17T19:01:19.000Z","dependencies_parsed_at":"2024-09-25T10:41:21.357Z","dependency_job_id":null,"html_url":"https://github.com/allenheltondev/postman-contract-test-generator","commit_stats":{"total_commits":43,"total_committers":9,"mean_commits":4.777777777777778,"dds":"0.39534883720930236","last_synced_commit":"d2c445fd7052bbf93b80ae62c21b064552aedb25"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/allenheltondev/postman-contract-test-generator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allenheltondev%2Fpostman-contract-test-generator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allenheltondev%2Fpostman-contract-test-generator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allenheltondev%2Fpostman-contract-test-generator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allenheltondev%2Fpostman-contract-test-generator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/allenheltondev","download_url":"https://codeload.github.com/allenheltondev/postman-contract-test-generator/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allenheltondev%2Fpostman-contract-test-generator/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278239978,"owners_count":25954098,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-03T02:00:06.070Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["oas","postman","schema-test"],"created_at":"2024-09-25T10:41:17.488Z","updated_at":"2025-10-03T23:05:42.246Z","avatar_url":"https://github.com/allenheltondev.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Postman Contract Test Generator\nWhen building APIs, a common need is to validate the shape of the requests and responses. You want to verify the implementation of the API matches the definition document. This is typically done through *contract tests*, which exercise required fields, optional fields, and things like query and path parameters.\n\n[Postman](https://www.postman.com/) has the ability to group requests together and associate them to your API as a contract test. The intention of this feature is to build a [collection](https://www.postman.com/collection/) manually to add coverage to your API.\n\n# Objective\nTo build an automated way to provide an exhaustive set of tests providing close to 100% coverage for contract tests. The **contract test generator** should not need to be maintained, it should be able to dynamically create all tests with every run based on the Open API Specification (OAS) document defining the API.\n\nThe generator tests the **implementation** of a given API -- not the definition. It should use the definition document as a ruleset and compare the responses of the generated requests to it.\n\n# What It Does\n\n## Execution\nEvery run, the generator will perform the following actions:\n\n* Load the OAS for the provided API/workspace\n* Validate the definition is in the proper format\n    * All parameters, schemas, and responses have `description` and `example` provided for every field\n    * `Servers` are defined\n    * User-defined governance settings configured in the *enviroment*\n* Build schema test array\n* Loop through every test in the array and submit the request to the API\n* Validate status code and response body schema against OAS\n\n## Schema Test Definition\nFor every path defined in the Open API Spec, a `json` object will be created in the following format to define the schema test:\n\n```\n{\n  \"path\": \"\", // Combines url from server and path\n  \"parameters\": [ ], // All parameters defined at the path\n  \"method\": \"\", // Path method\n  \"allowedRole\": \"\", // If using role based apps, grabs the first allowable role\n  \"responses\": [ // All expected responses for the path\n    {\n      \"statusCode\": 200, // Status code defined in responses array\n      \"$ref\": \"#/components/responses/Ok\" // Grabs either referenced response or inline defined schema\n    }\n  ],\n  \"success\": true, // Boolean for if this test is expected to be a success\n  \"description\": \"Has all required fields\", // Generated description for what is being tested\n  \"body\": { }, // Generated body in JSON\n  \"name\": \"\" // Generated request name with method, path, description, and success\n}\n```\n\n## Test Generation\nThe generator will build request bodies for each endpoint using the **example** provided for every property. If there is no example provided for an individual property, it will not be included in the request. Examples will be used for all parameters and schema properties.\n\nFor accurate tests, it is advised *to use real values from your test environment in your examples* in order to get valid responses back during test execution. This means using known, hardcoded or seeded values in the system to define your API in the OAS.\n\nThe request body will be generated with the required fields only. The generator will take the request body with all required fields and create an array of mutations against it. For each required field in the request body, two mutations will be created:\n\n* Omit the required field\n* Leave the required field blank\n\nOnce all the mutations have been created, the generator proceeds to execute the tests.\n\n### Example\nLet's take an example API that maintains cars. The car schema as defined in the OAS would be:\n\n```yaml\ncomponents:\n  schemas:\n    Car:\n      type: object\n      required:\n        - make\n        - model\n        - year\n      properties:\n        make:\n          type: string\n          example: Nissan\n        model:\n          type: string\n          example: Pathfinder\n        year:\n          type: number\n          example: 2015\n        color:\n          type: string\n          example: red\n```\n\nThe generator will create the following request body for the schema:\n\n```json\n{\n  \"make\": \"Nissan\",\n  \"model\": \"Pathfinder\",\n  \"year\": 2015\n}\n```\n\nIt will also make mutations for every property:\n```json\n{\n  \"model\": \"Pathfinder\",\n  \"year\": 2015\n}\n```\nand\n\n```json\n{\n  \"make\": \"\",\n  \"model\": \"Pathfinder\",\n  \"year\": 2015\n}\n```\n\nThe generator is building these tests to verify the implementation of the API matches what is defined in the OAS. It will check to see if a valid http status code (400) is given if a missing required field is submitted.\n\n\n# What It Does NOT Do\nThe generator will **not** build a collection to be run at a later date. It builds and executes the tests at runtime, allowing for the test to dynamically update as changes are made to the Open API Spec.\n\n# Requirements\nThe Open API Spec tested by the contract test generator is required to be in a specific format. Below are the minimum requirements for the tests to execute:\n\n* The `Servers` object is defined and has at least one server with a description\n* Every schema property must have an example defined\n* The following fields in the provided environment are configured\n    * `env-apiKey` - Integration API Key for Postman\n    * `env-workspaceId` - Identifier for the workspace that contains the API to be tested\n    * `env-requireParamExample` - Must be set to true (the collection will still run, but this will show you where any problems lie)\n    * `env-server` - Matches the description of the server to be tested in the `Servers` object\n* Request bodies are defined in the `#/components/schemas` section of the OAS and are referred to by using `$ref`\n\n# Authentication\nAuthentication is the only piece of the generator that needs to be handled by the consumer. It is set up assuming all requests that execute against your API use the same authentication method.\n\n## Configuration\nIf your API uses authentication, you will be required to configure it on the collection itself.\n\n1. Right click the collection in your Postman workspace and select **Edit**.\n2. Click on the **Authorization** tab and configure the type of Authentication your API requires.\n3. If using OAuth2.0, you may [reference my blog post](https://www.readysetcloud.io/blog/allen.helton/how-to-automate-oauth2-token-renewal-in-postman-864420d381a0/) on how to automate the token renewal.\n4. Click **Update** to save your changes\n\n**NOTE-** If your API uses a standard api key header like `x-api-key` this step is unnecessary. You may just add it in the `#/components/parameters` section and it will be included automatically in each request.\n```yaml\ncomponents:\n  parameters:\n    ApiKey:\n      name: x-api-key\n      in: header\n      example: 982345jsdw0971ls09812354\n      schema:\n        type: string\n```\n\n## Setup\nIn this repo there are two files you need to import into your [Postman](https://postman.com/) workspace:\n* Contract Test Generator *collection*\n* Contract Test Generator *Environment*\n\nIf you are unsure how to import these into Postman, please [refer to this guide](https://kb.datamotion.com/?ht_kb=postman-instructions-for-exporting-and-importing).\n\n# CI Pipeline / Automation\nTo run this as part of a CI pipeline, [Newman](https://learning.postman.com/docs/running-collections/using-newman-cli/command-line-integration-with-newman) a command line interface from Postman can be used to execute the collection.\n\nThe collection runs as a single command with overridden environment variables. As many of the [environment variables](#Environment) can be overridden as you'd like, just keep adding the `--env-var` flag to the command to override variables. \n\nBelow is an example of a completely genericized Newman execution. This command will pull from variables in the CI environment (all variables with a `$` come from CI) and substitute them in the command.\n\n```\nnewman run https://api.getpostman.com/collections/$POSTMAN_TEST_GENERATION_COLLECTION_ID?apikey=$POSTMAN_API_KEY --environment https://api.getpostman.com/environments/$POSTMAN_TEST_GENERATION_ENVIRONMENT_ID?apikey=$POSTMAN_API_KEY --env-var \"env-workspaceId=$POSTMAN_WORKSPACE_ID\" --env-var \"env-server=$POSTMAN_TEST_ENVIRONMENT\"\n```\n\nThis command offers maximum portability, offering the user the ability to import the collection and environment into their Postman workspace one time and reuse it for all APIs they own by simply replacing environment variables.\n\nNewman will return the appropriate exit code if any assertions fail, which will automatically cause your build pipeline to fail.\n\n# Extensions\nIf you wish to use extensions with your Open API Spec to assist with the test generation, see details below for supported extensions:\n\n## x-amazon-apigateway-integration\nIf you use AWS API Gateway and practice [API-first development](https://www.readysetcloud.io/blog/allen.helton/api-first-development-with-postman/), chances are you use the [x-amazon-apigateway-integration extension](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-swagger-extensions-integration.html). This extension allows you to identify how each endpoint proxies to an AWS service. For API-first development, you are able to to mock out the proxy and response for endpoints that are not implemented yet. \n\nWhen using this extension, you can add the following snippet to tell API Gateway to mock out the response.\n```yaml\nx-amazon-apigateway-integration:\n  responses:\n    200:\n      statusCode: 200\n  passthroughBehavior: when_no_match\n  requestTemplates:\n    application/json: |\n      {\n        'statusCode': 200\n      }\n  type: mock\n```\n\nThe generator will skip over any endpoint methods that have `type: mock` defined in this extension. No tests will be run for mocked endpoints.\n\n## x-postman-variables\nInstead of relying completely on seeded, or known, data in the system for the generated tests, this extension allows you to use objects created in the generated tests in subsequent tests. For example, if you have an endpoint that creates a `book` object by doing a **POST** to `/books`, this extension will allow you to save the returned id as a collection variable and use it in other API calls, like doing a **GET** on `/books/{bookId}`.\n\n**Saving a variable**\n\nIn the `responses` section of an endpoint method you can add the following snippet to save a collection variable.\n```yaml\nresponses:\n  201:\n    $ref: `#/components/responses/Created`\n    x-postman-variables:\n      - type: save\n        name: bookId\n        path: .id\n```\n\n`type` - Must be *save* in order to save the value to a collection variable\n`name` - What to name the collection variable\n`path` - Json-path to the property you want. Currently does not support arrays. It must start with a '.'\n\nThe extension is an array, so you can add as many saves as you'd like. \n\n**Consuming a variable**\n\nThe extension must be added to a parameter for consumption. This could be a header, query param, or path param. It works with both inline and ref parameters.\n```yaml\nparameters:\n  bookId:\n      name: bookId\n      in: path\n      description: Unique identifier for the book\n      required: true\n      schema:\n        type: string\n        example: 23SovYJfRZ5Wt7jpZEPHVo\n      x-postman-variables:\n        - type: load\n          name: bookId\n```\n\nWhenever this parameter is used, the test generator will load the value from the collection variable. If the collection variable does not exist or has no value, it will fall back to the provided example.\n\n# Environment\nBelow is a list of environment variables currently consumed by the generator:\n\n* `env-apiKey` - Your Postman API key used to access the Postman API\n* `env-minApiCount` - The minimum amount of APIs required in a provided workspace\n* `env-maxApiCount` - The maximum amount of APIs required in a provided workspace\n* `env-workspaceId` - The identifier of the workspace you wish to generate tests for\n* `env-requireParamDescription` - Flag denoting if assertions should be run that require a parameter description - **BOOLEAN**\n* `env-requireParamExample` - Flag denoting if assertions should be run that require a parameter example - **BOOLEAN**\n* `env-paramDescriptionMinLength` - The minimum length a description should be. This is only used if `env-requireParamDescription` is true\n* `env-paramDesciptionMaxLength` - The maximum length a description should be. This is only used if `env-requireParamDescription` is true\n* `env-securityExtensionName` - If using a role-based API, the name of the extension you use to denote allowed roles on an endpoint\n* `env-roleHeaderName` - If using a role-based API, the name of the header where you supply the user's assumed role\n* `env-server` - The description of which `server` element to use. This is for the base url of your API. [See OAS Documentation](https://swagger.io/docs/specification/api-host-and-base-path/)\n* `env-runComponentTests` - Flag denoting if assertions should be run validating schema adherence - **BOOLEAN**\n* `env-runContractTests` - Flag denoting if contract tests should be generated and run - **BOOLEAN**\n* `env-schemaPropertyExceptions` - The names of any properties that should not be validated in the component tests - **ARRAY OF STRINGS**\n* `env-jsonToYaml` - NPM Package that converts yaml to json. DO NOT EDIT!\n\n# Contact\nYou may contact me by any of the social media channels below:\n\n[![Twitter][1.1]][1] [![GitHub][2.1]][2] [![LinkedIn][3.1]][3] [![Ready, Set, Cloud!][4.1]][4]\n\n[1.1]: http://i.imgur.com/tXSoThF.png\n[2.1]: http://i.imgur.com/0o48UoR.png\n[3.1]: http://i.imgur.com/lGwB1Hk.png\n[4.1]: https://readysetcloud.s3.amazonaws.com/logo.png\n\n[1]: http://www.twitter.com/allenheltondev\n[2]: http://www.github.com/allenheltondev\n[3]: https://www.linkedin.com/in/allen-helton-85aa9650/\n[4]: https://readysetcloud.io\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallenheltondev%2Fpostman-contract-test-generator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fallenheltondev%2Fpostman-contract-test-generator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallenheltondev%2Fpostman-contract-test-generator/lists"}