{"id":25625263,"url":"https://github.com/dreamquality/swagger-coverage-cli","last_synced_at":"2026-02-09T08:37:05.700Z","repository":{"id":273226354,"uuid":"919049933","full_name":"dreamquality/swagger-coverage-cli","owner":"dreamquality","description":"A Node.js CLI tool to measure test coverage of Swagger/OpenAPI specs using Postman collections.","archived":false,"fork":false,"pushed_at":"2025-01-23T14:31:19.000Z","size":426,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-22T19:05:57.914Z","etag":null,"topics":["openapi","postman-reporter","reporter","reporting","swagger"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/swagger-coverage-cli","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dreamquality.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-19T15:29:22.000Z","updated_at":"2025-01-23T14:31:23.000Z","dependencies_parsed_at":"2025-01-19T16:29:38.205Z","dependency_job_id":"ce6689c1-7c91-4c94-aca0-9fc51f39d4b1","html_url":"https://github.com/dreamquality/swagger-coverage-cli","commit_stats":null,"previous_names":["dreamquality/swagger-coverage-cli"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dreamquality%2Fswagger-coverage-cli","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dreamquality%2Fswagger-coverage-cli/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dreamquality%2Fswagger-coverage-cli/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dreamquality%2Fswagger-coverage-cli/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dreamquality","download_url":"https://codeload.github.com/dreamquality/swagger-coverage-cli/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248867526,"owners_count":21174748,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["openapi","postman-reporter","reporter","reporting","swagger"],"created_at":"2025-02-22T14:56:46.964Z","updated_at":"2026-02-09T08:37:05.694Z","avatar_url":"https://github.com/dreamquality.png","language":"JavaScript","readme":"![Postman](https://img.shields.io/badge/Postman-FF6C37?logo=postman\u0026logoColor=white) ![OpenAPI](https://img.shields.io/badge/OpenAPI-5392CE?logo=openapi\u0026logoColor=white) ![Swagger](https://img.shields.io/badge/Swagger-85EA2D?logo=swagger\u0026logoColor=white) ![gRPC](https://img.shields.io/badge/gRPC-4caf50?logo=grpc\u0026logoColor=white) ![GraphQL](https://img.shields.io/badge/GraphQL-e91e63?logo=graphql\u0026logoColor=white) ![Tests](https://github.com/dreamquality/swagger-coverage-cli/actions/workflows/test.yml/badge.svg) ![CSV](https://img.shields.io/badge/CSV-1DA598?logo=csv\u0026logoColor=white) ![npm](https://img.shields.io/npm/v/swagger-coverage-cli?color=blue\u0026label=npm\u0026logo=npm)\n \n\n# Swagger Coverage CLI\n\n\u003e **A comprehensive command-line utility to analyze API test coverage across **multiple protocols**: OpenAPI/Swagger (REST), gRPC Protocol Buffers, and GraphQL schemas. Generates unified HTML reports with protocol-specific insights.**\nCheck out the [Example!](https://dreamquality.github.io/swagger-coverage-cli)**\n\n## Table of Contents\n\n1. [Introduction](#introduction)\n2. [Multi-Protocol Support](#multi-protocol-support)\n3. [Features](#features)\n4. [How It Works (Diagram)](#how-it-works-diagram)\n5. [Installation \u0026 Requirements](#installation--requirements)\n6. [Getting Started](#getting-started)\n\n   - [1. Prepare Your Files](#1-prepare-your-files)\n   - [2. Run the CLI](#2-run-the-cli)\n   - [3. Check the Coverage Report](#3-check-the-coverage-report)\n\n7. [Protocol-Specific Usage](#protocol-specific-usage)\n8. [Detailed Matching Logic](#detailed-matching-logic)\n9. [Smart Endpoint Mapping](#smart-endpoint-mapping)\n10. [Supported File Formats](#supported-file-formats)\n\n- [Using CSV for Documentation](#using-csv-for-documentation)\n\n11. [Contributing](#contributing)\n12. [License](#license)\n\n---\n\n## Introduction\n\n**swagger-coverage-cli** is a comprehensive tool that helps you **measure API test coverage across multiple protocols**. It analyzes how much of your documented APIs are actually covered by your Postman tests. The tool supports:\n\n### 🚀 Supported API Protocols\n\n- **📋 REST APIs**: OpenAPI/Swagger specifications (v2/v3) in JSON or YAML format\n- **⚡ gRPC APIs**: Protocol Buffer (`.proto`) files with service definitions\n- **🔀 GraphQL APIs**: GraphQL schema (`.graphql`, `.gql`) files with queries, mutations, and subscriptions\n- **📊 CSV APIs**: Custom CSV format for flexible API documentation\n\n### 🎯 Input Sources\n\n1. **API Specifications**: Single or multiple API files in supported formats\n2. **Test Collections**: Postman collections (JSON) with requests and test scripts\n3. **Execution Reports**: Newman run reports (JSON) with actual test execution results\n\nThe tool supports processing **multiple API specifications in a single run**, making it ideal for organizations managing microservices with diverse protocols. It **calculates unified coverage percentages** and produces **detailed HTML reports** with protocol-specific insights.\n\n---\n\n## Multi-Protocol Support\n\n**swagger-coverage-cli** provides comprehensive support for modern API ecosystems with multiple protocols, enabling unified coverage analysis across your entire technology stack.\n\n### 🌐 Universal CLI Interface\n\n```bash\n# Single protocol APIs\nswagger-coverage-cli api.yaml collection.json           # OpenAPI/REST\nswagger-coverage-cli service.proto collection.json     # gRPC\nswagger-coverage-cli schema.graphql collection.json    # GraphQL\nswagger-coverage-cli api-docs.csv collection.json      # CSV\n\n# Mixed protocol APIs (Enterprise-ready)\nswagger-coverage-cli \"api.yaml,service.proto,schema.graphql\" collection.json\n\n# All existing options work across protocols\nswagger-coverage-cli \"api.yaml,service.proto\" collection.json --verbose --strict-body\n```\n\n### 🎯 Protocol-Specific Features\n\n#### 📋 REST/OpenAPI Support\n- **OpenAPI v2/v3**: Full specification support\n- **Smart Path Matching**: Handles parameter variations (`/users/{id}` vs `/users/{userId}`)\n- **Status Code Intelligence**: Prioritizes 2xx → 4xx → 5xx responses\n- **Request Body Validation**: JSON schema validation with strict mode\n\n#### ⚡ gRPC Support  \n- **Protocol Buffer Parsing**: Automatic `.proto` file analysis\n- **Service Discovery**: Extracts all services, methods, and message types\n- **Path Generation**: Maps to HTTP/2 paths (`/package.service/method`)\n- **Content-Type Validation**: Supports `application/grpc` and variants\n\n#### 🔀 GraphQL Support\n- **Schema Analysis**: Parses `.graphql` and `.gql` files\n- **Operation Extraction**: Identifies queries, mutations, and subscriptions\n- **Type System**: Full support for arguments, unions, and interfaces\n- **Endpoint Unification**: Maps all operations to `/graphql` endpoint\n\n### 📊 Unified Reporting\n\n- **Protocol Column**: Color-coded identification (🟢 gRPC, 🔴 GraphQL, 🔵 REST)\n- **Mixed Statistics**: Combined coverage metrics across all protocols\n- **Individual Breakdown**: Per-API and per-protocol insights\n- **Smart Search**: Protocol-aware filtering and search functionality\n\n---\n\n## Features\n\n- **🌐 Multi-Protocol Support**: Native support for REST (OpenAPI/Swagger), gRPC (Protocol Buffers), and GraphQL schemas\n- **🔄 Mixed API Analysis**: Process multiple API specifications with different protocols in a single run\n- **🎯 Protocol-Aware Matching**: Intelligent request matching tailored to each API protocol's characteristics\n- **📊 Unified Reporting**: Generate consolidated HTML reports with protocol-specific insights and color coding\n- **⚡ Easy to Use**: Simple CLI interface works across all supported protocols with consistent syntax\n- **🔍 Multiple Input Types**: Supports Postman collections and Newman run reports for maximum flexibility\n- **🤖 Auto-Detection**: Automatically detects API file types and Newman report formats\n- **🏗️ Enterprise Ready**: Perfect for microservices architectures using diverse API protocols\n- **🎨 Smart Endpoint Mapping**: Intelligent endpoint matching with status code prioritization and enhanced path matching\n- **🔒 Strict Matching (Optional)**: Enforce strict checks for query parameters, request bodies, and more\n- **🛡️ Flexible Validation**: Skip spec validation with `--disable-spec-validation` for legacy APIs or specs with reference issues\n- **📈 Enhanced HTML Reports**: Generates interactive `coverage-report.html` with protocol identification\n- **🧩 Extensible**: Modular code structure allows customization of matching logic and protocol support\n- **📋 CSV Support**: Flexible API documentation format for teams preferring spreadsheet-based docs\n- **✅ Unit Tested**: Comprehensive Jest test suite covering all protocols and edge cases\n\n---\n\n## How It Works (Diagram)\n\n```mermaid\nflowchart LR\n    A[OpenAPI/Swagger Spec] --\u003e B[Load \u0026 Parse Spec]\n    C[Postman Collection] --\u003e D[Load \u0026 Parse Collection]\n    B --\u003e E[Match Operations]\n    D --\u003e E\n    E --\u003e F[Calculate Coverage]\n    F --\u003e G[Generate HTML \u0026 Console Report]\n```\n\n1. Load \u0026 Parse Spec/CSV: The CLI reads your Swagger (YAML or JSON) or CSV file and extracts all documented operations (method, path, status codes, parameters, requestBody details, etc.).\n2. Load \u0026 Parse Collection: The CLI reads your Postman collection (JSON) to gather all requests, their URLs, methods, test scripts, and any explicitly tested status codes.\n3. *Match Operations: The tool compares each documented operation with each Postman request to see if it’s covered (based on path patterns, method, status codes tested, and optional strict checks on parameters and body).\n4. Calculate Coverage: It counts how many documented API operations are matched with Postman tests vs. how many total operations are defined in your spec or CSV.\n5. Generate Report: Prints a summary to the console and creates an HTML file (by default coverage-report.html) with matched and unmatched endpoints.\n\n---\n\n## Multi-API Support\n\n**swagger-coverage-cli** now supports processing multiple Swagger/OpenAPI specifications in a single run, making it perfect for organizations managing multiple APIs or microservices.\n\n### Usage\n\nTo analyze multiple APIs, provide comma-separated file paths:\n\n```bash\nswagger-coverage-cli \"api1.yaml,api2.yaml,api3.json\" collection.json\n```\n\n### Features\n\n- **Unified Coverage Report**: Get a single report showing coverage across all APIs\n- **API Identification**: Each operation in the report is tagged with its source API\n- **Individual API Breakdown**: The report header shows both the combined API list and individual API names\n- **Separate Operation Tracking**: Operations with the same path/method from different APIs are tracked separately\n- **Backwards Compatibility**: Single API mode works exactly as before\n\n### Example Output\n\nWhen processing multiple APIs, the console output will show:\n\n```\n=== Swagger Coverage Report ===\nAPIs analyzed: User API, Product API, Order API\nTotal operations in spec(s): 24\nMatched operations in Postman: 18\nCoverage: 75.00%\n\nUnmatched Spec operations:\n - [User API] [GET] /users/{id}/profile (statusCode=404)\n - [Product API] [POST] /products (statusCode=400)\n - [Order API] [DELETE] /orders/{id} (statusCode=204)\n```\n\nThe HTML report will include:\n- **API column** in the operations table to identify the source API\n- **Combined API information** in the report header\n- **Individual API breakdown** for detailed analysis\n\n---\n\n## Installation \u0026 Requirements\n\n### Prerequisites\n\n- **Node.js** version 12+ (16+ recommended).\n- **NPM** (or Yarn) installed.\n\n### Installation\n\n1. **Clone** or **download** this repository.\n2. **Install dependencies** by running:\n\n```bash\nnpm install -g swagger-coverage-cli\n\n\n\n```\n\n---\n\n## Getting Started\n\n### 1. Prepare Your Files\n\nYou will need:\n\n1. **OpenAPI/Swagger** specification file (e.g., `openapi.yaml` or `swagger.json`) OR CSV documentation file following the specified format.\n2. **Postman** collection file (JSON format), which you can export from the Postman app, **OR** a **Newman run report** (JSON format) generated by running your collection with Newman.\n\n\u003e **Note**: \n\u003e - If using a Postman collection, make sure it includes actual test scripts that assert or check specific status codes (e.g., `pm.response.to.have.status(200)`).\n\u003e - **Folder-level tests** are fully supported: tests defined at the folder level in your Postman collection will be automatically applied to all requests within that folder, ensuring comprehensive coverage calculation.\n\u003e - If using a Newman report, the tool will extract actual response codes and test results from the execution data.\n\n### 2. Run the CLI\n\nUse the following command:\n\n```bash\nnpm swagger-coverage-cli \u003cswaggerFile\u003e \u003cpostmanCollectionOrNewmanReport\u003e [options]\n\n```\n\n**For Multiple APIs:**\n\n```bash\nnpm swagger-coverage-cli \"api1.yaml,api2.yaml,api3.json\" collection.json [options]\n\n```\n\n**Examples:**\n\n**With Postman Collection:**\n```bash\nnpm swagger-coverage-cli openapi.yaml collection.json --verbose --strict-query --strict-body\n\n```\n\n**With Newman Report:**\n```bash\nnpm swagger-coverage-cli openapi.yaml newman-report.json --newman --verbose\n\n```\n\n**Multiple APIs with Newman Report:**\n```bash\nnpm swagger-coverage-cli \"users-api.yaml,products-api.yaml\" newman-report.json --newman --output multi-api-report.html\n\n```\n\n**Options**:\n\n- `--verbose`: Display additional logs (helpful for debugging).\n- `--newman`: Treat input file as Newman run report instead of Postman collection.\n- `--strict-query`: Enforce strict checks on query parameters (e.g., required params, `enum`, `pattern`, etc.).\n- `--strict-body`: Verify that `application/json` request bodies in the spec match raw JSON bodies in Postman requests.\n- `--disable-spec-validation`: Disable OpenAPI/Swagger spec validation (useful for specs with validation or reference issues).\n- `--output \u003cfile\u003e`: Customize the name of the HTML report file (default is `coverage-report.html`).\n\n### Run via NPM Script\n\n```bash\nnpm swagger-coverage-cli -- \u003cswaggerFile\u003e \u003cpostmanCollectionOrNewmanReport\u003e [options]\n\n\n```\n\n### 3. Check the Coverage Report\n\nAfter execution, you will see:\n\n1. **Console Output**:\n\n```bash\n=== Swagger Coverage Report ===\nTotal operations in spec: 12\nMatched operations in Postman: 9\nCoverage: 75.00%\n\nUnmatched operations:\n- [DELETE] /items/{id} (statusCode=204)\n- [PUT] /items/{id} (statusCode=400)\n...\n\n\n\n```\n\n2. **HTML Report**:\n   - A file named `coverage-report.html` (or the name you provided with `--output`) is generated.\n   - Open it in your browser to see a table of matched/unmatched operations with color highlights.\n\n![main](https://github.com/dreamquality/swagger-coverage-cli/blob/main/assets/main.png?raw=true)\n![table](https://github.com/dreamquality/swagger-coverage-cli/blob/main/assets/table.png?raw=true)\n\n---\n\n## Protocol-Specific Usage\n\n### 🌐 OpenAPI/REST APIs\n\nUse standard OpenAPI/Swagger files in YAML or JSON format:\n\n```bash\n# Single OpenAPI specification\nswagger-coverage-cli api-spec.yaml collection.json\n\n# Multiple REST APIs\nswagger-coverage-cli \"api-v1.yaml,api-v2.yaml,legacy.json\" collection.json\n\n# With strict validation\nswagger-coverage-cli openapi.yaml collection.json --strict-query --strict-body\n```\n\n**Supported OpenAPI features:**\n- Path parameters (`/users/{id}`, `/users/{userId}`)\n- Query parameters with schema validation\n- Request body validation (JSON, form-data, etc.)\n- Multiple response status codes per operation\n- OpenAPI v2 and v3 specifications\n\n### ⚡ gRPC APIs\n\nAnalyze Protocol Buffer service definitions:\n\n```bash\n# Single gRPC service\nswagger-coverage-cli user-service.proto collection.json\n\n# Multiple gRPC services\nswagger-coverage-cli \"user.proto,order.proto,payment.proto\" collection.json\n\n# Mixed with OpenAPI\nswagger-coverage-cli \"rest-api.yaml,grpc-service.proto\" collection.json\n```\n\n**gRPC-specific features:**\n- Service and method extraction from `.proto` files\n- HTTP/2 path mapping (`/package.service/method`)\n- Content-type validation (`application/grpc`, `application/grpc+proto`)\n- Nested package support (`company.api.v1.UserService`)\n\n**Example Postman request for gRPC:**\n```json\n{\n  \"method\": \"POST\",\n  \"url\": \"{{grpcUrl}}/user.v1.UserService/GetUser\",\n  \"header\": [\n    { \"key\": \"Content-Type\", \"value\": \"application/grpc\" }\n  ],\n  \"body\": {\n    \"mode\": \"raw\",\n    \"raw\": \"{\\\"user_id\\\": \\\"123\\\"}\"\n  }\n}\n```\n\n### 🔀 GraphQL APIs\n\nAnalyze GraphQL schema definitions:\n\n```bash\n# Single GraphQL API\nswagger-coverage-cli schema.graphql collection.json\n\n# Multiple GraphQL schemas  \nswagger-coverage-cli \"user-schema.gql,product-schema.graphql\" collection.json\n\n# Full stack coverage\nswagger-coverage-cli \"api.yaml,service.proto,schema.graphql\" collection.json\n```\n\n**GraphQL-specific features:**\n- Query, mutation, and subscription extraction\n- Argument analysis with type information\n- Union and interface type support\n- Nested type relationship mapping\n\n**Example Postman request for GraphQL:**\n```json\n{\n  \"method\": \"POST\", \n  \"url\": \"{{apiUrl}}/graphql\",\n  \"header\": [\n    { \"key\": \"Content-Type\", \"value\": \"application/json\" }\n  ],\n  \"body\": {\n    \"mode\": \"raw\",\n    \"raw\": \"{\\\"query\\\": \\\"query GetUser($id: ID!) { user(id: $id) { id name email } }\\\", \\\"variables\\\": {\\\"id\\\": \\\"123\\\"}}\"\n  }\n}\n```\n\n### 📊 CSV Documentation\n\nUse CSV format for flexible API documentation:\n\n```bash\n# CSV-based API documentation\nswagger-coverage-cli api-docs.csv collection.json\n\n# Mixed with other formats\nswagger-coverage-cli \"api.yaml,docs.csv,service.proto\" collection.json\n```\n\n**CSV format columns:**\n- `method`: HTTP method (GET, POST, etc.)\n- `path`: API endpoint path\n- `statusCode`: Expected response status code\n- `description`: Operation description\n- `tags`: Comma-separated tags for grouping\n\n### 🏗️ Enterprise Scenarios\n\n**Microservices Architecture:**\n```bash\n# Complete microservices stack\nswagger-coverage-cli \"gateway.yaml,user-service.proto,analytics.graphql,docs.csv\" tests.json\n\n# Per-team analysis\nswagger-coverage-cli \"team-a-api.yaml,team-b-service.proto\" team-tests.json\n```\n\n**CI/CD Integration:**\n```bash\n# Production coverage check\nswagger-coverage-cli \"$(find apis -name '*.yaml' -o -name '*.proto' -o -name '*.graphql' | tr '\\n' ',')\" collection.json --output coverage-$(date +%Y%m%d).html\n```\n\n---\n\n## Coverage Calculation Formulas\n\n**swagger-coverage-cli** uses precise mathematical formulas to calculate API test coverage. Understanding these formulas helps you interpret coverage reports and set appropriate coverage targets.\n\n### Basic Coverage Formula\n\nThe core coverage calculation is based on the ratio of matched operations to total operations in your API specification:\n\n```\nCoverage (%) = (Matched Operations / Total Operations) × 100\n```\n\nWhere:\n- **Total Operations** = Number of unique operation-status code combinations in your API specification\n- **Matched Operations** = Number of operations that have corresponding tests in your Postman collection or Newman report\n\n### Operation Matching Criteria\n\nAn operation is considered **matched** when ALL of the following criteria are satisfied:\n\n1. **HTTP Method Match**: `specOperation.method === postmanRequest.method`\n2. **Path Pattern Match**: Postman URL matches the Swagger path pattern (with parameter substitution)\n3. **Status Code Verification**: At least one expected status code from the spec is tested\n4. **Query Parameters** (if `--strict-query` enabled): Required parameters are present and valid\n5. **Request Body** (if `--strict-body` enabled): JSON body structure matches specification\n\n### Coverage Types\n\n#### Standard Coverage\n```\nStandard Coverage = Matched Operations / Total Spec Operations × 100\n```\n\n#### Postman Collection Coverage\nWhen using Postman collections, coverage is calculated based on:\n- Static analysis of test scripts\n- Expected status codes extracted from `pm.response.to.have.status(code)` assertions\n- Request structure validation\n\n#### Newman Report Coverage (Recommended)\nWhen using Newman reports, coverage calculation includes:\n- **Actual execution data** from real test runs\n- **Response codes** from executed requests\n- **Assertion results** (passed/failed tests)\n\n```\nNewman Coverage = (Executed Operations with Passing Tests / Total Spec Operations) × 100\n```\n\n### Multi-API Coverage\n\nFor multiple API specifications, coverage is calculated as:\n\n```\nCombined Coverage = (Sum of Matched Operations across all APIs / Sum of Total Operations across all APIs) × 100\n```\n\nEach API operation is identified by: `{API_Name}:{Method}:{Path}:{StatusCode}`\n\n### Tag-Based Coverage\n\nCoverage can also be calculated per tag/group:\n\n```\nTag Coverage = (Matched Operations in Tag / Total Operations in Tag) × 100\n```\n\n### Examples\n\n#### Example 1: Single API\n- API Specification: 20 operations (GET /users/200, POST /users/201, GET /users/404, etc.)\n- Postman Tests: 15 operations covered\n- **Coverage**: 15/20 × 100 = **75%**\n\n#### Example 2: Newman vs Postman Collection\n- API Specification: 18 operations\n- Newman Report: 8 operations with actual execution data\n- Postman Collection: 2 operations with static test scripts\n- **Newman Coverage**: 8/18 × 100 = **44.44%**\n- **Postman Coverage**: 2/18 × 100 = **11.11%**\n\n#### Example 3: Multi-API Portfolio\n- User API: 10 operations (7 matched) = 70% coverage\n- Product API: 15 operations (12 matched) = 80% coverage\n- Order API: 8 operations (5 matched) = 62.5% coverage\n- **Combined Coverage**: (7+12+5)/(10+15+8) × 100 = 24/33 × 100 = **72.73%**\n\n### Coverage Quality Metrics\n\nBeyond basic percentage, consider these quality indicators:\n\n1. **Status Code Coverage**: Percentage of documented status codes that are tested\n2. **Method Distribution**: Coverage across different HTTP methods (GET, POST, PUT, DELETE)\n3. **Critical Path Coverage**: Coverage of business-critical API endpoints\n4. **Error Case Coverage**: Percentage of error scenarios (4xx, 5xx) that are tested\n\n### Recommendations\n\n- **Minimum Coverage**: Aim for 80%+ coverage for production APIs\n- **Critical Endpoints**: Ensure 100% coverage for authentication, payment, and data modification endpoints\n- **Error Testing**: Include at least 50% of documented error cases\n- **Use Newman Reports**: For more accurate coverage analysis in CI/CD pipelines\n\n---\n\n## Handling Specs with Validation Issues\n\nSometimes API specifications may have validation errors or broken references, especially in legacy systems or during development. The `--disable-spec-validation` flag allows you to analyze coverage even when specs have issues.\n\n### When to Use\n\nUse `--disable-spec-validation` when:\n- Working with legacy APIs that have incomplete specifications\n- Specs contain broken `$ref` references that can't be resolved\n- External references aren't available in your CI/CD environment\n- You need quick coverage analysis without fixing all spec issues first\n- API specifications are still in development\n\n### Usage\n\n```bash\n# Standard usage (validation enabled - will fail on invalid specs)\nswagger-coverage-cli api.yaml collection.json\n\n# Disable validation for specs with issues\nswagger-coverage-cli api.yaml collection.json --disable-spec-validation\n\n# Works with all other flags\nswagger-coverage-cli api.yaml collection.json --disable-spec-validation --verbose --strict-body\n```\n\n### Example\n\n**Without the flag (validation enabled):**\n```bash\n$ swagger-coverage-cli broken-spec.yaml collection.json\nError: Token \"NonExistentSchema\" does not exist.\n```\n\n**With the flag (validation disabled):**\n```bash\n$ swagger-coverage-cli broken-spec.yaml collection.json --disable-spec-validation\n=== Swagger Coverage Report ===\nTotal operations in spec(s): 12\nMatched operations in Postman/Newman: 9\nCoverage: 75.00%\n\nHTML report saved to: coverage-report.html\n```\n\n### Important Notes\n\n- When validation is disabled, the tool parses the spec without validating references\n- Coverage can still be calculated for the operations that are defined\n- The tool won't catch structural issues in the spec\n- Default behavior (with validation enabled) ensures spec quality\n\n---\n\n## Detailed Matching Logic\n\n**swagger-coverage-cli** tries to match each **operation** from the spec with a **request** in Postman. An operation is considered **covered** if:\n\n1. **HTTP Method** matches exactly (`GET`, `POST`, `PUT`, `DELETE`, etc.).\n2. **Path**:\n\n   - The path pattern from Swagger (e.g., `/users/{id}`) is converted to a regex (like `^/users/[^/]+$`).\n   - The Postman request URL (minus any base URL placeholders like `{{baseUrl}}`) must match that regex.\n\n3. **Status Code**:\n\n   - If the spec operation has a specific status code (e.g., `200`, `404`), the CLI checks the Postman test scripts to see if that status code is asserted (e.g., `pm.response.to.have.status(200)`).\n\n4. **Query Parameters** (only if `--strict-query` is enabled):\n\n   - If the spec says a query parameter is required and has certain constraints (e.g., `enum`, `pattern`, `type`), the tool verifies that the Postman request includes that parameter and meets the constraints.\n\n5. **Request Body** (only if `--strict-body` is enabled):\n\n   - If the spec says `requestBody` includes `application/json`, the CLI checks if the Postman request body is raw JSON and can be parsed without errors.\n\nIf all criteria are satisfied, the operation is **matched** (covered). Otherwise, it’s reported as **unmatched**.\n\n## Smart Endpoint Mapping\n\n**Smart endpoint mapping** is an advanced feature that significantly improves coverage accuracy by using intelligent algorithms to match endpoints. It is **enabled by default** in all operations.\n\n### Key Benefits\n\n- Enhanced path matching for better parameter recognition\n- **Status Code Prioritization**: Prioritizes successful (2xx) status codes over error codes\n- **Enhanced Path Matching**: Better handling of parameter variations and naming conventions\n- **Confidence Scoring**: Assigns quality scores to matches (0.0-1.0)\n- **Multi-API Support**: Works seamlessly with microservices and complex architectures\n\n### Quick Start\n\n```bash\n# Smart mapping is enabled by default\nswagger-coverage-cli api-spec.yaml collection.json --verbose\n\n# Output shows smart mapping statistics:\n# Smart mapping: 6 primary matches, 3 secondary matches\n# Coverage: 50.00%\n```\n\n### Example Use Cases\n\n**Status Code Intelligence:**\n```yaml\n# API defines multiple status codes\nGET /users:\n  responses:\n    '200': { description: Success }\n    '400': { description: Bad Request }\n    '500': { description: Server Error }\n\n# Postman only tests success case\npm.test(\"Status code is 200\", function () {\n    pm.response.to.have.status(200);\n});\n\n# Smart mapping result:\n# ✅ Primary Match: GET /users (200) - Matched\n# ❌ Secondary: GET /users (400, 500) - Unmatched but deprioritized\n```\n\n**Enhanced Path Matching:**\n```yaml\n# API Spec: /users/{userId}/profile\n# Postman:   /users/123/profile\n# Result:    ✅ Intelligent parameter matching (confidence: 1.0)\n```\n\n### Complete Documentation\n\nFor comprehensive examples, use cases, and implementation details, see:\n**📖 [Smart Mapping Examples \u0026 Use Cases](docs/smart-mapping-examples.md)**\n\nThis document covers:\n- 25+ detailed examples across 8 categories\n- Real-world API scenarios (CRUD, microservices, versioning)\n- Edge cases and error handling\n- Performance testing and best practices\n- CLI integration examples\n\n---\n---\n\n## Supported File Formats\n\n**Swagger/OpenAPI/.csv**:\n\n- **JSON** or **YAML**\n- **OpenAPI v2 (Swagger 2.0)** or **OpenAPI v3.x**\n- **CSV**: API documentation can be provided in CSV format following the specified structure.\n\n**Postman**:\n\n- **Postman collection** in **JSON** format (v2.1 or higher recommended).\n   - Typically exported via the Postman app: *File → Export → Collection*.\n\n**Newman Reports**:\n\n- **Newman run report** in **JSON** format.\n   - Generated by running: `newman run collection.json --reporters json --reporter-json-export newman-report.json`\n\n### Postman Collection vs Newman Report\n\nThe tool supports two types of input for test data:\n\n#### Postman Collection\n- **What it is**: The collection definition containing requests and test scripts\n- **Pros**: \n  - Contains all test logic and assertions\n  - Can extract expected status codes from test scripts\n  - **Supports folder-level tests**: Tests defined at the folder level are automatically applied to all requests within that folder and its subfolders\n- **Cons**: \n  - No actual execution data\n  - Relies on parsing test scripts to understand expected outcomes\n\n#### Newman Report  \n- **What it is**: Actual execution results from running a Postman collection with Newman\n- **Pros**: \n  - Contains real execution data with actual response codes\n  - Includes assertion results (passed/failed)\n  - More accurate representation of what was actually tested\n  - Includes response times and execution metadata\n- **Cons**: \n  - Requires an additional step to generate the report\n\n**Recommendation**: Use Newman reports when possible for more accurate coverage analysis, especially in CI/CD pipelines where collections are actually executed.\n\n### Folder-Level Tests in Postman Collections\n\n**swagger-coverage-cli** fully supports **folder-level tests** in Postman collections. Tests defined at the folder level are automatically applied to all requests within that folder and its subfolders, making it easy to apply common test assertions across multiple endpoints.\n\n#### How It Works\n\nWhen you define tests at the folder level in your Postman collection:\n1. The tests are extracted from the folder's event scripts\n2. Status codes and assertions are identified from the test scripts\n3. These tests are automatically combined with any request-level tests\n4. All requests within the folder (and nested folders) inherit these tests\n\n#### Example\n\nConsider this Postman collection structure:\n\n```json\n{\n  \"name\": \"Users API\",\n  \"item\": [\n    {\n      \"name\": \"Get User\",\n      \"request\": { \"method\": \"GET\", \"url\": \"/users/1\" }\n    },\n    {\n      \"name\": \"Create User\", \n      \"request\": { \"method\": \"POST\", \"url\": \"/users\" }\n    }\n  ],\n  \"event\": [\n    {\n      \"listen\": \"test\",\n      \"script\": {\n        \"exec\": [\n          \"pm.test('Status code is 200 or 201', function () {\",\n          \"    pm.expect(pm.response.code).to.be.oneOf([200, 201]);\",\n          \"});\"\n        ]\n      }\n    }\n  ]\n}\n```\n\nIn this example:\n- The folder \"Users API\" has a test that checks for status codes 200 or 201\n- **Both** \"Get User\" and \"Create User\" requests will inherit these status codes\n- This means both endpoints will be counted as testing status codes 200 and 201\n\n#### Benefits\n\n- **Reduced duplication**: Define common tests once at the folder level\n- **Better coverage**: Ensures all requests within a folder test common scenarios\n- **Easier maintenance**: Update folder-level tests to affect all child requests\n- **Nested support**: Folder-level tests are inherited through multiple levels of nesting\n\n#### Supported Test Patterns\n\nFolder-level tests support the same patterns as request-level tests:\n- `pm.response.to.have.status(200)`\n- `pm.expect(pm.response.code).to.eql(201)`\n- `pm.expect(pm.response.code).to.be.oneOf([200, 201, 204])`\n- And other common Postman test assertions\n\n#### Edge Cases Handled\n\nThe folder-level test feature handles various edge cases:\n- **Empty folders**: Folders with no requests are handled gracefully\n- **Deep nesting**: Supports unlimited levels of nested folders with test inheritance at each level\n- **Missing events**: Requests or folders without event properties work correctly\n- **Mixed patterns**: Multiple test patterns in the same folder or request are combined\n- **Null/undefined events**: Folders with null or undefined event properties are handled safely\n- **Non-test events**: Only \"test\" events are processed; other events (like \"prerequest\") are ignored\n\n### Using CSV for Documentation\n\nIn addition to traditional OpenAPI/Swagger specifications, **swagger-coverage-cli** supports API documentation provided in a **CSV** format. This allows for a more flexible and easily editable documentation process, especially for teams that prefer spreadsheet-based documentation.\n\n#### CSV Structure\n\nYour CSV file should adhere to the following structure to ensure compatibility with **swagger-coverage-cli**:\n\n1. **Header Row**: The first row must contain the following columns in **exactly** this order:\n\n```sh\nMETHOD,URI,NAME,STATUS CODE,BODY,TAGS\n\n```\n\n2. **Data Rows**: Each subsequent row represents an API endpoint response with data corresponding to the headers.\n\n#### Column Descriptions\n\n- **METHOD**\n\n   - **Type:** String\n   - **Description:** The HTTP method used for the API endpoint (e.g., `GET`, `POST`, `PUT`, `DELETE`).\n\n- **URI**\n\n   - **Type:** String\n   - **Description:** The endpoint's URI path (e.g., `/api/products`).\n\n- **NAME**\n\n   - **Type:** String\n   - **Description:** A descriptive name for the API endpoint action (e.g., `getProducts`).\n\n- **STATUS CODE**\n\n   - **Type:** Integer\n   - **Description:** The HTTP status code returned by the API (e.g., `200`, `400`).\n\n- **BODY**\n\n   - **Type:** String (JSON format)\n   - **Description:** The response body returned by the API in JSON format. Ensure that JSON strings are properly escaped.\n\n- **TAGS**\n\n   - **Type:** String\n   - **Description:** Tags associated with the endpoint for categorization and filtering (e.g., `products`).\n\n#### Example CSV\n\nBelow is an example of how your `documentation.csv` should be structured:\n\n```csv\nMETHOD,URI,NAME,STATUS CODE,BODY,TAGS\nGET,/api/products,getProducts,200,\"{\\\"products\\\":\\\"updated\\\"}\",\"products\"\nGET,/api/products,getProducts,400,\"{\\\"error\\\":\\\"Invalid query parameters\\\"}\",\"products\"\n```\n---\n\n## Contributing\n\nContributions are welcome! Please:\n\n1. **Fork** this repo\n2. **Create a branch** for your feature or fix (`git checkout -b feature/something`)\n3. **Commit** your changes\n4. **Open a Pull Request**\n\nFeel free to add tests in the [`test/`](./test) folder to cover any new logic.\n\n---\n\n## License\n\nThis project is licensed under the [MIT License](./LICENSE).\n\nHappy testing! Feel free to open issues or submit pull requests for any improvements.\n\n\n```\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdreamquality%2Fswagger-coverage-cli","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdreamquality%2Fswagger-coverage-cli","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdreamquality%2Fswagger-coverage-cli/lists"}