{"id":23820994,"url":"https://github.com/truesparrowsystems/openapi-test-suite","last_synced_at":"2025-07-01T04:07:46.682Z","repository":{"id":37824569,"uuid":"492695454","full_name":"TrueSparrowSystems/openapi-test-suite","owner":"TrueSparrowSystems","description":"REST API complete test suite using openapi.json","archived":false,"fork":false,"pushed_at":"2023-01-30T13:52:55.000Z","size":117,"stargazers_count":23,"open_issues_count":0,"forks_count":1,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-06-17T06:05:09.133Z","etag":null,"topics":["openapi"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TrueSparrowSystems.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-05-16T05:29:36.000Z","updated_at":"2023-09-30T10:23:04.000Z","dependencies_parsed_at":"2022-06-22T19:15:01.371Z","dependency_job_id":null,"html_url":"https://github.com/TrueSparrowSystems/openapi-test-suite","commit_stats":null,"previous_names":["plg-works/openapi-test-suite"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/TrueSparrowSystems/openapi-test-suite","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TrueSparrowSystems%2Fopenapi-test-suite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TrueSparrowSystems%2Fopenapi-test-suite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TrueSparrowSystems%2Fopenapi-test-suite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TrueSparrowSystems%2Fopenapi-test-suite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TrueSparrowSystems","download_url":"https://codeload.github.com/TrueSparrowSystems/openapi-test-suite/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TrueSparrowSystems%2Fopenapi-test-suite/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262893645,"owners_count":23380711,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["openapi"],"created_at":"2025-01-02T08:19:40.091Z","updated_at":"2025-07-01T04:07:46.662Z","avatar_url":"https://github.com/TrueSparrowSystems.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Openapi Test Suite\n![npm version](https://img.shields.io/npm/v/@truesparrow/openapi-test-suite.svg?style=flat)\n\n## Objective\nThis package aims to solve the following two problems:\n1. Maintenance is a big problem to solve in any test suite. As the APIs evolve at a fast pace, maintenance becomes more difficult.\n2. Usually when negative test cases are written, all combinations of parameters are not covered. Writing all the combinations manually is difficult and gets neglected.\n\nWe solve point 1, by consuming the openapi.json file to generate test cases in DRY manner.\nOpenapi.json is auto-generated using a generator script, as described in this [blog](https://truesparrow.com/blog/dry-api-docs-and-validations/).\n\nTo solve point 2, we use the cartesian product of possible (correct as well as incorrect) sets of values for each parameter to get to the set of all negative test cases.\n\n## Approach\n- For each route, we figure out the possible set of values (correct as well as incorrect) for each parameter. Now, we take the \n   cartesian product of these sets, to get all possible combinations of parameters. Correctness of the values is based \n   on the parameter type and parameter constraints given in openapi.json\n- For all the incorrect combinations, we fire API calls and check whether the API response has the correct param level error.\n- In each route, every optional parameter is tested with its correct, incorrect as well as null value.\n- For cookie protected routes, we can include a Security Scheme object as a third argument while initializing the package. This object will be automatically consumed\n   based on the specific security requirements of a route. If the cookies provided in Security Scheme object cannot be validated then http error 401 Unauthorized is expected.\n\n## Install\nFor installing the npm package, run following command in terminal:\n```\n    npm install @truesparrow/openapi-test-suite\n```\n\n## Initialize\n\n```node\n    const openapiObj = require('@truesparrow/openapi-test-suite');\n    \n    const ApiTestSuite = require('@truesparrow/openapi-test-suite');\n    const serverIndex = 0; // Index of the server (to be hit) from the servers block of openapi.json\n    const securityScheme = require('./examples/securityScheme.json');\n    const apiSuiteObj = new ApiTestSuite(openApiObj, serverIndex, securityScheme);\n```\n\n## Run Test Suite\n```node\n    // To run the test suite, use following command\n    apiSuiteObj.runTest();\n\n    // After the test suite run is over, it is recommended to call cleanup because the openApiObj and serverIndex are in-memory cached.\n    apiSuiteObj.cleanup();\n```\n\n### Examples\n\n#### Case 1: Expected success true but received false.\nIn the following, all params were passed correctly, but the API response received `success` as false.\n```\n    GET /api/admin/web/login/phone/otp \n    params: {\"country_code\":1,\"raw_phone_number\":\"9878654325\"} \n    Expected response success: true \n    Incorrect Parameters: {} \n    Response HTTP code: 200 \n    API response: {\n                    \"success\": false,\n                    \"err\": {\n                      \"code\": \"BAD_REQUEST\",\n                      \"msg\": \"Something went wrong.\",\n                      \"error_data\": [\n                        {\n                          \"parameter\": \"phone_number\",\n                          \"msg\": \"Invalid phone number.\"\n                        }\n                      ],\n                      \"internal_id\": \"INVALID_PHONE_NUMBER\"\n                    }\n                  } \n    Response validation error: {\"kind\":\"errorRespForCorrectCase\"}\n```\n\n#### Case 2: Expected success false but received true.\nIn the following, `raw_phone_number` was passed incorrectly, but the API response received `success` as true.\n```\n    GET /api/admin/web/login/phone/otp \n    params: {\"country_code\":91,\"raw_phone_number\":null} \n    Expected response success: false \n    Incorrect Parameters: {\"raw_phone_number\":\"Value cannot be null\"}\n    Response HTTP code: 200 \n    API response: {\n                    \"success\": true,\n                    \"data\": {\n                      \"otp_detail\": {\n                        \"id\": 69,\n                        \"sms_identifier\": \"69:dfdc1b36-3fa8-4b9c-8c09-a61f334dfd9f\",\n                        \"uts\": 1653291961\n                      }\n                    }\n                  } \n```\n\n#### Case 3: Incorrect parameter passed but not obtained back as an error\nIn the following, `raw_phone_number` was passed incorrectly, but the API response `error_data` does not convey the same.\n```\n    GET /api/admin/web/login/phone/otp \n    params: {\"country_code\":\"959bb2\",\"raw_phone_number\":null} \n    Expected response success: false \n    incorrectParamsMap: {\"country_code\":\"Value cannot be string\",\"raw_phone_number\":\"Value cannot be null\"} \n    Response HTTP code: 200 \n    API response: {\n                    \"success\": false,\n                    \"err\": {\n                      \"code\": \"BAD_REQUEST\",\n                      \"msg\": \"Something went wrong.\",\n                      \"error_data\": [\n                        {\n                          \"parameter\": \"country_code\",\n                          \"msg\": \"Invalid parameter country_code.  Please ensure the input is well formed.\"\n                        },\n                        {\n                          \"parameter\": \"phone_number\",\n                          \"msg\": \"Invalid phone number.\"\n                        }\n                      ],\n                      \"internal_id\": \"v_ap_rd_1\"\n                    }\n                  } \n    Response validation error: {\"kind\":\"parameterErrorNotObtained\",\"parameter\":\"phone_number\"} \n```\n\n#### Case 4: Correct parameter passed but got API error.\nIn the following, all params were passed correctly, but got the api error \"Something went wrong\".\n```\n    GET /api/admin/web/login/phone/otp \n    params: {\"country_code\":91,\"raw_phone_number\":\"9876543239\"} \n    Expected response success: true \n    Incorrect Parameters: {} \n    Response HTTP code: 500 \n    API response: {\n                    \"success\": false,\n                    \"err\": {\n                      \"code\": \"INTERNAL_SERVER_ERROR\",\n                      \"msg\": \"Something went wrong.\",\n                      \"error_data\": [],\n                      \"internal_id\": \"PHONE_NUMBER_DOES_NOT_EXIST\"\n                    }\n                  } \n    Response validation error: {\"kind\":\"errorRespForCorrectCase\"} \n```\n\n#### Case 5: Correct parameter passed but error in schema validation.\nIn the following, all params were passed correctly, but got `respEntityTypeMismatch` error.\n```\n   GET /api/admin/web/login/phone/otp \n   params: {\"country_code\":91,\"raw_phone_number\":\"9876543233\"} \n   Expected response success: true \n   Incorrect Parameters: {} \n   Response HTTP code: 200 \n   API response: {\n                   \"success\": true,\n                   \"data\": {\n                     \"otp_detail\": {\n                       \"id\": \"abc\",\n                       \"sms_identifier\": \"72:83c91ffc-9741-492f-8fbf-f61d6e9b9ba8\",\n                       \"uts\": 1653293902\n                     }\n                   }\n                 } \n   Response validation error: {\"kind\":\"respEntityTypeMismatch\",\"debugLevel\":\"response.data.otp_detail.id\",\"schemaType\":\"string\"} \n```\n\n#### Case 6: Incorrect cookie value and correct parameters passed.\nIn the following, incorrect cookie value was passed, so the expected response http code was 401; but the API response received `success` as true.\n```\n   GET /api/consumer/v1/user/current \n   params: {\"country_code\":91,\"raw_phone_number\":\"9876543233\"} \n   Expected response success: false \n   Incorrect Parameters: {} \n   Response HTTP code: 200 \n   API response: {\n                   \"success\": true,\n                   \"data\": {\n                           \"current_user\": {\n                               \"id\": 100000,\n                               \"first_name\": \"ishaphone\",\n                               \"last_name\": \"lastn\",\n                               \"basic_user_detail_id\": 100000,\n                               \"status\": \"ACTIVE\",\n                               \"uts\": 1651651178\n                           }\n                    }\n                 }\n   Response validation error: {\"kind\":\"mandatoryCookieValidationFailed\"}\n   Request headers: {\"Cookie\":\"aulc=25de238247286fa7fc7b9cbd071a45eda87f8690dd524cbf71c104d1d7e8252ef2dc4a698a389368dceb673238ccc47a342b611fdd00e3522183c25d6fb866faf368a7f9ec716e9df4b446d3541cd\"} \n```\n\n#### Case 7: Correct cookie value and incorrect parameters passed.\nIn the following, correct cookie value was passed with incorrect combination of parameters, but got http code 401.\n```\n    GET /api/consumer/v1/user/current \n       params: {\"country_code\":91,\"raw_phone_number\":null}\n       Expected response success: false \n       Incorrect Parameters: {\"raw_phone_number\":\"Value cannot be null\"} \n       Response HTTP code: 401 \n       API response: {\n                       \"success\": false,\n                       \"err\": {\n                         \"code\": \"UNAUTHORIZED\",\n                         \"msg\": \"Access denied due to invalid credentials.\",\n                         \"error_data\": [],\n                         \"internal_id\": \"l_ch_vclcr_2\"\n                       }\n                     }\n       Response validation error: {\"kind\":\"unauthorizedApiRequestForValidCookie\"}\n       Request headers: {\"Cookie\":\"aulc=25de238247286fa7fc7b9cbd071a45eda87f8690dd524cbf71c104d1d7e8252ef2dc4a698a389368dceb673238ccc47a342b611fdd00e3522183c25d6fb866faf368a7f9ec716e9df4b446d3541cd\"} \n```\n\n## Schema Validator\nUsing schema validator, the data can be recursively cross validated against its schema.\n\n### Examples\nGet schema validator class using apiSuiteObj.\n```\n    const schemaValidator = apiSuiteObj.SchemaValidator;\n```\nThen call validateObjectBySchema(data,dataSchema,debugLevel) method of schemaValidator class with following arguments:\n1) data = Data to be validated (dataType : object).\n2) dataSchema = Schema against which the given data should be validated. (dataType : object).\n3) debugLevel = String value which helps in debugging (dataType : string).\n\nNote: validateObjectBySchema function throws error only in case of incorrect data.\n#### Case 1: Validate object data.\n```\n    const data = require('./examples/dataToBeValidated_1.json');\n    const dataSchema = require('./examples/dataSchema_1.json');\n    const correctData = data.correctData;\n    const incorrectData = data.incorrectData;\n```\nWith correct data\n```\nnew schemaValidator().validateObjectBySchema(correctData,dataSchema,'Response');\n```\nWith incorrect data\n```\nnew schemaValidator().validateObjectBySchema(incorrectData,dataSchema,'Response');\n```\nLogs:\n```\n{\n  kind: 'respEntityTypeMismatch',\n  debugLevel: 'VALUE(Response).status',\n  schemaType: 'string'\n}\n```\nHere kind indicates the type of validation error, debugLevel indicates the level at which the validation failed and schemaType indicates expected schema.\n#### Case 2: Validate array of object data.\n```\n    const data = require('./examples/dataToBeValidated_2.json');\n    const dataSchema = require('./examples/dataSchema_2.json');\n    const correctData = data.correctData;\n    const incorrectData = data.incorrectData;\n```\n```\nnew schemaValidator().validateObjectBySchema(incorrectData,dataSchema,'Response');\n```\nLogs:\n```\n{\n  kind: 'respEntityTypeMismatch',\n  debugLevel: 'Response.[0].name',\n  schemaType: 'string'\n}\n```\n#### Case 3: Validate array of integer data.\n```\n    const data = require('./examples/dataToBeValidated_3.json');\n    const dataSchema = require('./examples/dataSchema_3.json');\n    \n    const correctData = data.correctData;\n    const incorrectData = data.incorrectData;\n```\n```\nnew schemaValidator().validateObjectBySchema(incorrectData,dataSchema,'Response');\n```\nLogs:\n```\n{\n  kind: 'respEntityTypeMismatch',\n  debugLevel: 'Response',\n  schemaType: 'array'\n}\n```\n#### Case 4: Validate string data.\n```\n    const data = require('./examples/dataToBeValidated_4.json');\n    const dataSchema = require('./examples/dataSchema_4.json');\n    \n    const correctData = data.correctData;\n    const incorrectData = data.incorrectData;\n```\n```\nnew schemaValidator().validateObjectBySchema(incorrectData,dataSchema,'Response');\n```\nLogs:\n```\n{\n  kind: 'respEntityTypeMismatch',\n  debugLevel: 'Response',\n  schemaType: 'string'\n}\n```\n## SQL \u0026 Executable Commands Injection.\nUsing Start security test you can test your API parameters' vulnerabilities against SQL and Command Injection attacks.\n\nManual monitoring of API logs is needed to check if any resources are affected by SQL or Commands injection queries.\n\nExecute security test class using apiSuiteObj.\n```\n    apiSuiteObj.runSecurityTest();\n```\n### Example\nAPI request will be executed for injection queries against each parameter.\n\n```\nGET /api/consumer/v1/signup/phone/otp \nRequest headers: {} \nparams: {\"country_code\":\"OR 1=1#\",\"raw_phone_number\":\"1111111111\"}\n```\n\n\n## Future Scope\n- Error schema in component section and it's use.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftruesparrowsystems%2Fopenapi-test-suite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftruesparrowsystems%2Fopenapi-test-suite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftruesparrowsystems%2Fopenapi-test-suite/lists"}