{"id":48291006,"url":"https://github.com/mustafaautomation/ai-testgen","last_synced_at":"2026-04-04T23:06:07.509Z","repository":{"id":343072875,"uuid":"1165016787","full_name":"mustafaautomation/ai-testgen","owner":"mustafaautomation","description":"AI-powered test case generator — transform PRDs, OpenAPI specs, and user stories into Playwright tests, API tests, and Gherkin features","archived":false,"fork":false,"pushed_at":"2026-03-08T19:43:41.000Z","size":148,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-08T22:24:16.491Z","etag":null,"topics":["ai","anthropic","cli","code-generation","gherkin","llm","openai","openapi","playwright","qa-automation","test-generation","typescript"],"latest_commit_sha":null,"homepage":"https://quvantic.com","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mustafaautomation.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-23T18:28:09.000Z","updated_at":"2026-03-08T19:42:39.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/mustafaautomation/ai-testgen","commit_stats":null,"previous_names":["mustafaautomation/ai-testgen"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/mustafaautomation/ai-testgen","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mustafaautomation%2Fai-testgen","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mustafaautomation%2Fai-testgen/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mustafaautomation%2Fai-testgen/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mustafaautomation%2Fai-testgen/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mustafaautomation","download_url":"https://codeload.github.com/mustafaautomation/ai-testgen/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mustafaautomation%2Fai-testgen/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31418288,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-04T20:09:54.854Z","status":"ssl_error","status_checked_at":"2026-04-04T20:09:44.350Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","anthropic","cli","code-generation","gherkin","llm","openai","openapi","playwright","qa-automation","test-generation","typescript"],"created_at":"2026-04-04T23:06:07.429Z","updated_at":"2026-04-04T23:06:07.496Z","avatar_url":"https://github.com/mustafaautomation.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AI TestGen\n\n[![CI](https://github.com/mustafaautomation/ai-testgen/actions/workflows/ci.yml/badge.svg)](https://github.com/mustafaautomation/ai-testgen/actions)\n[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)\n[![Node.js](https://img.shields.io/badge/Node.js-18+-339933.svg?logo=node.js\u0026logoColor=white)](https://nodejs.org)\n[![TypeScript](https://img.shields.io/badge/TypeScript-strict-3178c6.svg?logo=typescript\u0026logoColor=white)](https://www.typescriptlang.org)\n[![Docker](https://img.shields.io/badge/Docker-Ready-2496ED.svg?logo=docker\u0026logoColor=white)](Dockerfile)\n\nAI-powered test case generator that transforms PRDs, OpenAPI specs, and user stories into Playwright tests, API tests, Gherkin features, or Markdown test plans. Provider-agnostic — works with OpenAI, Anthropic, or any custom LLM endpoint.\n\n---\n\n## Table of Contents\n\n- [Why?](#why)\n- [Demo](#demo)\n- [Quick Start](#quick-start)\n- [Architecture](#architecture)\n- [CLI Commands](#cli-commands)\n- [Input Format Detection](#input-format-detection)\n- [Configuration](#configuration)\n- [Caching](#caching)\n- [Streaming](#streaming)\n- [Error Handling](#error-handling)\n- [Programmatic API](#programmatic-api)\n- [Provider Support](#provider-support)\n- [CI/CD Integration](#cicd-integration)\n- [Project Structure](#project-structure)\n- [Development](#development)\n\n---\n\n## Why?\n\n42% of testers struggle to write automation scripts. AI TestGen bridges the gap by generating structured, runnable test cases from natural-language requirements — no prompt engineering required.\n\n- **3 input formats**: PRD/Markdown, OpenAPI (YAML/JSON), User Stories (As a.../Given/When/Then)\n- **4 output formats**: Playwright tests, API tests, Gherkin features, Markdown test plans\n- **Provider-agnostic**: OpenAI, Anthropic, or custom LLM endpoints\n- **TypeScript validation**: Validates generated code via `tsc --noEmit`\n- **Configurable**: Test style (descriptive/concise/BDD), include negative/boundary cases\n\n---\n\n## Demo\n\n```\n$ npm test\n\n Test Files  13 passed (13)\n      Tests  94 passed (94)\n   Duration  1.44s\n```\n\n\u003e **94 unit tests** covering parsers, templates, validation, config, generator, CLI, providers, retry logic, streaming, and caching.\n\n---\n\n## Quick Start\n\n```bash\nnpm install ai-testgen\n\n# Copy and fill in your API keys\ncp .env.example .env\n\n# Initialize config\nnpx ai-testgen init\n\n# Generate Playwright tests from a PRD\nnpx ai-testgen generate requirements.md --format playwright\n\n# Generate API tests from OpenAPI spec\nnpx ai-testgen from-spec api-spec.yaml\n\n# Generate Gherkin features from user stories\nnpx ai-testgen generate stories.txt --format gherkin\n```\n\n---\n\n## Architecture\n\n```\n┌─────────────────────────────────────────────────────┐\n│                    CLI / API                         │\n├─────────────────────────────────────────────────────┤\n│                   Generator                         │\n│         (detect input → parse → prompt → output)    │\n├──────────┬──────────┬───────────────────────────────┤\n│  Parsers │Templates │        Outputs                │\n│  PRD     │Playwright│    Playwright (.spec.ts)       │\n│  OpenAPI │API       │    Gherkin (.feature)          │\n│  Stories │Gherkin   │    Markdown (.md)              │\n│          │Markdown  │                                │\n├──────────┴──────────┴───────────────────────────────┤\n│              Provider Layer (fetch-based)             │\n│         OpenAI  │  Anthropic  │  Custom HTTP         │\n└─────────────────────────────────────────────────────┘\n```\n\n---\n\n## CLI Commands\n\n### `generate \u003cfile\u003e`\n\nGenerate test cases from any supported input file.\n\n```bash\nnpx ai-testgen generate \u003cfile\u003e [options]\n\nOptions:\n  -f, --format \u003ctype\u003e   playwright, api, gherkin, markdown (default: playwright)\n  -o, --output \u003cdir\u003e    Output directory (default: ./generated-tests)\n  -c, --config \u003cpath\u003e   Path to config file\n  --style \u003ctype\u003e        descriptive, concise, bdd (default: descriptive)\n  --no-negative         Skip negative test cases\n  --no-boundary         Skip boundary test cases\n  --no-stream           Disable streaming output\n  --dry-run             Show parsed input and prompt without calling LLM\n  --model \u003cmodel\u003e       Override LLM model from command line\n  --no-cache            Bypass response cache\n  --clear-cache         Clear all cached responses\n  -v, --verbose         Enable debug logging\n```\n\n### `from-spec \u003cfile\u003e`\n\nGenerate API tests specifically from OpenAPI specs.\n\n### `plan \u003cfile\u003e`\n\nGenerate a Markdown test plan from any input.\n\n### `validate \u003cfile\u003e`\n\nValidate generated test files (TypeScript compilation, Gherkin syntax, Markdown structure).\n\n### `init`\n\nCreate default configuration file.\n\n```bash\nnpx ai-testgen init [options]\n\nOptions:\n  --force               Overwrite existing config\n```\n\n---\n\n## Input Format Detection\n\n| Format | Detection |\n|--------|-----------|\n| **OpenAPI** | `openapi` or `swagger` key in YAML/JSON |\n| **User Stories** | Contains \"As a\" or \"Given\" keywords |\n| **PRD** | Markdown with headers + requirement keywords |\n\n---\n\n## Configuration\n\nCopy `.env.example` to `.env` and fill in your API keys:\n\n```bash\ncp .env.example .env\n```\n\nThen create or edit `ai-testgen.config.json`:\n\n```json\n{\n  \"provider\": {\n    \"type\": \"openai\",\n    \"apiKey\": \"$OPENAI_API_KEY\",\n    \"model\": \"gpt-4o-mini\"\n  },\n  \"output\": {\n    \"format\": \"playwright\",\n    \"dir\": \"./generated-tests\",\n    \"style\": \"descriptive\"\n  },\n  \"options\": {\n    \"includeNegative\": true,\n    \"includeBoundary\": true,\n    \"maxTokens\": 4096,\n    \"temperature\": 0.2\n  }\n}\n```\n\n---\n\n## Caching\n\nAI TestGen caches LLM responses to avoid redundant API calls. Cache behavior:\n\n- **Key**: SHA-256 hash of input content + model + format + style + temperature\n- **Location**: `.ai-testgen/cache/` (configurable)\n- **TTL**: 24 hours default (configurable via `cache.ttlSeconds`)\n- **Bypass**: `--no-cache` flag\n- **Clear**: `--clear-cache` flag\n\nCache config in `ai-testgen.config.json`:\n\n```json\n{\n  \"cache\": {\n    \"dir\": \".ai-testgen/cache\",\n    \"ttlSeconds\": 86400,\n    \"enabled\": true\n  }\n}\n```\n\n---\n\n## Streaming\n\nWhen running in a TTY (interactive terminal), AI TestGen streams LLM tokens in real-time with a progress spinner. In non-TTY environments (CI pipelines), it falls back to plain log lines. Use `--no-stream` to disable streaming.\n\n---\n\n## Error Handling\n\n- **Retry with exponential backoff**: 3 retries with 1s → 2s → 4s delays on 429/5xx errors\n- **API key validation**: Clear error message if no key is configured\n- **Graceful errors**: File not found, invalid config, and API errors produce clean messages (no stack traces)\n\n---\n\n## Programmatic API\n\n```typescript\nimport { Generator, DEFAULT_CONFIG } from 'ai-testgen';\n\nconst generator = new Generator({\n  ...DEFAULT_CONFIG,\n  provider: {\n    type: 'openai',\n    apiKey: process.env.OPENAI_API_KEY,\n    model: 'gpt-4o-mini',\n  },\n});\n\nconst result = await generator.generate('./requirements.md', 'playwright');\nconsole.log(`Generated ${result.summary.totalTests} tests`);\n```\n\n---\n\n## Provider Support\n\n| Provider | Config Type | Default Model |\n|----------|------------|---------------|\n| OpenAI | `openai` | gpt-4o-mini |\n| Anthropic | `anthropic` | claude-sonnet-4-5-20250514 |\n| Custom | `custom` | (configurable) |\n\n---\n\n## CI/CD Integration\n\nThe included GitHub Actions workflow:\n\n1. Runs lint, format, type check on Node 18 \u0026 20\n2. Executes all 94 unit tests\n3. Builds the package to verify publishability\n\nAdd your API keys as repository secrets for generation:\n- `OPENAI_API_KEY`\n- `ANTHROPIC_API_KEY`\n\n---\n\n## Project Structure\n\n```\nai-testgen/\n├── .github/\n│   ├── workflows/ci.yml          # CI pipeline (Node 18/20 matrix)\n│   ├── dependabot.yml            # Automated dependency updates\n│   ├── CODEOWNERS                # Review ownership\n│   └── pull_request_template.md  # PR checklist\n├── src/\n│   ├── parsers/                  # Input format parsers\n│   │   ├── prd.parser.ts         # PRD/Markdown parser\n│   │   ├── openapi.parser.ts     # OpenAPI spec parser\n│   │   └── story.parser.ts       # User story parser\n│   ├── templates/                # LLM prompt templates\n│   │   ├── base.template.ts      # Abstract template\n│   │   ├── playwright.template.ts\n│   │   ├── api.template.ts\n│   │   ├── gherkin.template.ts\n│   │   └── markdown.template.ts\n│   ├── providers/                # LLM provider adapters\n│   │   ├── base.provider.ts      # Abstract base with timedCall\n│   │   ├── openai.provider.ts    # OpenAI chat completions\n│   │   ├── anthropic.provider.ts # Anthropic messages API\n│   │   └── custom.provider.ts    # Any HTTP-based LLM\n│   ├── outputs/                  # Output file writers\n│   │   ├── base.output.ts\n│   │   ├── playwright.output.ts\n│   │   ├── gherkin.output.ts\n│   │   └── markdown.output.ts\n│   ├── core/                     # Framework core\n│   │   ├── generator.ts          # Main generator orchestrator\n│   │   ├── config.ts             # Config loader + env resolution\n│   │   ├── validator.ts          # TS/Gherkin/MD validation\n│   │   └── types.ts              # Type definitions\n│   ├── utils/                    # Shared utilities\n│   │   ├── prompt.ts             # Code block extraction\n│   │   └── logger.ts             # Colored structured logging\n│   ├── cli.ts                    # Command-line interface\n│   └── index.ts                  # Public API exports\n├── tests/unit/                   # 94 unit tests\n├── CONTRIBUTING.md\n├── SECURITY.md\n├── Dockerfile\n└── .dockerignore\n```\n\n---\n\n## Development\n\n```bash\ngit clone https://github.com/mustafaautomation/ai-testgen.git\ncd ai-testgen\nnpm install\nnpm test              # Run unit tests\nnpm run typecheck     # Type checking\nnpm run lint          # ESLint\nnpm run format:check  # Prettier\nnpm run build         # Compile TypeScript\n```\n\n---\n\n## License\n\nMIT\n\n---\n\nBuilt by [Quvantic](https://quvantic.com)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmustafaautomation%2Fai-testgen","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmustafaautomation%2Fai-testgen","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmustafaautomation%2Fai-testgen/lists"}