{"id":51757392,"url":"https://github.com/sritajkumarpatel/testforge","last_synced_at":"2026-07-19T08:06:19.696Z","repository":{"id":370321213,"uuid":"1291449296","full_name":"sritajkumarpatel/testforge","owner":"sritajkumarpatel","description":"AI-powered test case generation for Azure DevOps. Feed it requirements documents, ADO work items, or plain text — three autonomous AI agents analyze, design, and produce ready-to-create test cases.","archived":false,"fork":false,"pushed_at":"2026-07-08T20:55:17.000Z","size":530,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-08T22:20:18.005Z","etag":null,"topics":["agile-testing","ai-testing","azure-devops","bdd","claude","gemini","gherkin","llm","ollama","openai","playwright","qa-automation","quality-engineering","software-testing","test-automation","test-case-generator","test-management"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sritajkumarpatel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-07-06T20:13:34.000Z","updated_at":"2026-07-08T20:55:24.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/sritajkumarpatel/testforge","commit_stats":null,"previous_names":["sritajkumarpatel/testforge"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/sritajkumarpatel/testforge","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sritajkumarpatel%2Ftestforge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sritajkumarpatel%2Ftestforge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sritajkumarpatel%2Ftestforge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sritajkumarpatel%2Ftestforge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sritajkumarpatel","download_url":"https://codeload.github.com/sritajkumarpatel/testforge/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sritajkumarpatel%2Ftestforge/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35646062,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-19T02:00:06.923Z","response_time":112,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agile-testing","ai-testing","azure-devops","bdd","claude","gemini","gherkin","llm","ollama","openai","playwright","qa-automation","quality-engineering","software-testing","test-automation","test-case-generator","test-management"],"created_at":"2026-07-19T08:06:18.330Z","updated_at":"2026-07-19T08:06:19.687Z","avatar_url":"https://github.com/sritajkumarpatel.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n# TestForge ⚒️\n\n### AI-Powered Test Case Generation for Azure DevOps\n\n**Requirements → ADO Test Cases. Three AI agents. One click.**\n\n[![CI](https://img.shields.io/github/actions/workflow/status/sritajkumarpatel/testforge/ci.yml?branch=main\u0026style=flat-square)](https://github.com/sritajkumarpatel/testforge/actions)\n[![npm](https://img.shields.io/npm/v/testforge?style=flat-square)](https://www.npmjs.com/package/testforge)\n[![GitHub stars](https://img.shields.io/github/stars/sritajkumarpatel/testforge?style=flat-square)](https://github.com/sritajkumarpatel/testforge/stargazers)\n[![GitHub release](https://img.shields.io/github/v/release/sritajkumarpatel/testforge?style=flat-square)](https://github.com/sritajkumarpatel/testforge/releases)\n[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg?style=flat-square)](LICENSE)\n[![Node](https://img.shields.io/badge/Node-%3E%3D18-339933?style=flat-square\u0026logo=node.js)](https://nodejs.org/)\n[![React](https://img.shields.io/badge/React-18-61DAFB?style=flat-square\u0026logo=react)](https://reactjs.org/)\n[![Vite](https://img.shields.io/badge/Vite-6-646CFF?style=flat-square\u0026logo=vite)](https://vitejs.dev/)\n\n[Quick Start](#quick-start) · [How It Works](#how-it-works) · [Features](#features) · [Configuration](#configuration) · [API Reference](#api-reference) · [Development \u0026 Testing](#development--testing)\n\n---\n\n\u003c/div\u003e\n\n## The Problem\n\n\u003e Manual test case creation is the #1 bottleneck in QA cycles. Writing 100+ test cases per sprint takes **days**, coverage is inconsistent, and Azure DevOps' UI makes bulk creation painful.\n\nTestForge solves this — feed it a requirements document, an ADO work item, or plain text, and three autonomous AI agents analyze, design, and produce ready-to-create ADO test cases in **under 3 minutes**.\n\n## The Solution\n\nYou don't write test cases — you **have a conversation** with three AI agents. Load your requirements, and watch each agent reason in real time:\n\n```\nYou: Load a requirements PDF\nAgent 1 (Requirements Analyst): \"I found 12 functional requirements with 2 ambiguity gaps...\"\nAgent 2 (Test Designer): \"Applying equivalence partitioning, boundary value analysis, and state transition...\"\nAgent 3 (Test Case Writer): \"Produced 48 ADO-ready test cases with preconditions and verification steps.\"\nYou: Click \"Create in Azure DevOps\" → 48 test cases created in 15 seconds.\n```\n\nNo manual test writing. No tedious copy-paste into ADO. Just your requirements + three AI agents + one click.\n\n---\n\n## How It Works\n\n```\nRequirements ──\u003e [Requirements Analyst] ──\u003e [Classifier] ──\u003e [UI / API / Mock Designers] ──\u003e [Test Case Writer] ──\u003e ADO Test Cases\n```\n\n| Agent                    | Role                                                       | Specialism                                                        | Prompt File                                                                                                                                                                                                                |\n| :----------------------- | :--------------------------------------------------------- | :---------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| **Requirements Analyst** | Parses raw input into structured functional requirements   | NLP extraction, ambiguity detection, gap analysis                 | [01-requirements-analyst.md](file:///Users/zenitsu/Documents/GitHub/testforge/agents/01-requirements-analyst.md)                                                                                                           |\n| **Classifier**           | Decides whether the requirement is UI, API, Mock, or mixed | Requirement-type routing                                          | [00-classifier.md](file:///Users/zenitsu/Documents/GitHub/testforge/agents/00-classifier.md)                                                                                                                               |\n| **UI Test Designer**     | Designs UI/UX scenarios                                    | Forms, navigation, validation, accessibility, responsive behavior | [02-ui-designer.md](file:///Users/zenitsu/Documents/GitHub/testforge/agents/02-ui-designer.md)                                                                                                                             |\n| **API Test Designer**    | Designs API scenarios                                      | HTTP contracts, auth, status codes, schema, rate limits           | [02-api-designer.md](file:///Users/zenitsu/Documents/GitHub/testforge/agents/02-api-designer.md)                                                                                                                           |\n| **Mock Designer**        | Provides mock/stub guidelines                              | External dependency simulation, contract testing                  | [02-mock-designer.md](file:///Users/zenitsu/Documents/GitHub/testforge/agents/02-mock-designer.md)                                                                                                                         |\n| **Test Case Writer**     | Produces a JSON array of ADO-ready test case objects       | Step-by-step actions, BDD/Gherkin formatting                      | [03-test-case-writer.md](file:///Users/zenitsu/Documents/GitHub/testforge/agents/03-test-case-writer.md), [03-test-case-writer-bdd.md](file:///Users/zenitsu/Documents/GitHub/testforge/agents/03-test-case-writer-bdd.md) |\n\nEach agent receives the full output of the previous agent, building context sequentially. Supports both **Regular (Standard steps)** and **BDD (Gherkin/Given-When-Then)** test formats.\n\n---\n\n## Features\n\n### Input \u0026 Pipeline\n\n- **3 Input Modes**: Upload files (PDF, Word, Markdown, plain text), fetch description from an ADO work item directly, or paste text.\n- **Streaming Pipeline**: Watch each agent reason in real time via Server-Sent Events (SSE).\n- **Regular + BDD Output**: Select BDD mode to generate test steps formatted in Given-When-Then Gherkin syntax.\n- **Run Log Export**: Download execution logs as `.txt` or `.json` for debugging or records.\n\n### Azure DevOps Integration\n\n- **One-Click ADO Creation (CDP)**: Connects to your browser session via Chrome/Edge DevTools Protocol (CDP) — no Personal Access Token (PAT) required. Uses existing active browser authentication context automatically.\n- **ADO PAT Support**: Alternatively, specify a PAT (`ADO_PAT`) to run bulk operations headlessly without browser interaction.\n- **Bulk Creation**: Test cases are pushed in parallel to Azure DevOps as rich work items, attaching steps, title, and technique tags.\n\n### Enterprise Security \u0026 Architecture\n\n- **API Protection**: Optional API authorization wrapper using `AUTH_TOKEN` bearer validation.\n- **Rate Limiting**: Built-in protection against API abuse using `express-rate-limit` for both general routes and LLM pipeline runs.\n- **State Persistence**: Form input, selected options, settings, and agent console logs survive tab refreshes and accidental navigations.\n\n---\n\n## Quick Start\n\n### Option 1: Docker (Recommended for Production)\n\n```bash\ngit clone https://github.com/sritajkumarpatel/testforge.git\ncd testforge\ncp .env.example .env\n# Edit .env with your LLM API key(s) and ADO settings\ndocker compose up --build\n```\n\nOpen **http://localhost:3010** in your browser.\n\n---\n\n### Option 2: AI Agent Setup\n\nOpen the repo with your AI coding agent (e.g., Claude, Antigravity, or Copilot):\n\n```bash\ngit clone https://github.com/sritajkumarpatel/testforge.git\ncd testforge\nopencode .\n# then say: \"Set up and run TestForge\"\n```\n\n---\n\n### Option 3: Manual Local Development\n\n1. **Clone the repository**:\n\n   ```bash\n   git clone https://github.com/sritajkumarpatel/testforge.git\n   cd testforge\n   ```\n\n2. **Install all dependencies** (installs both server and client workspaces):\n\n   ```bash\n   npm install\n   ```\n\n3. **Configure environment variables**:\n\n   ```bash\n   cp .env.example .env\n   ```\n\n   Open `.env` and set at least one LLM provider key (e.g., `OPENAI_API_KEY`, `CLAUDE_API_KEY`, or `GOOGLE_API_KEY`). For local inference, configure `OLLAMA_URL`.\n\n4. **Launch the application** (two terminal windows):\n\n   ```bash\n   # Terminal 1 — backend (port 3010, hot-reloads on save)\n   npm run dev:server\n\n   # Terminal 2 — frontend (port 5173, proxies /api to backend)\n   npm run dev:client\n   ```\n\n   Open **http://localhost:5173** in your browser.\n\n   **For production bundle:**\n\n   ```bash\n   # Build client and start integrated server\n   npm run build --workspace=client\n   npm start\n   ```\n\n   Open **http://localhost:3010** in your browser.\n\n---\n\n## Screenshots\n\nBelow is a conceptual layout of the TestForge dashboard:\n\n```\n┌────────────────────────────────────────────────────────────────────────┐\n│  ⚒️ TestForge                     [ Generator ]   [ History ]   [ Settings ]│\n├────────────────────────────────────────────────────────────────────────┤\n│ ┌───────────────────────────────┐ ┌──────────────────────────────────┐ │\n│ │ 1. Input Source               │ │ 3. Pipeline Output               │ │\n│ │ [ Text ] [ File ] [ ADO WI ]  │ │ ┌──────────────────────────────┐ │ │\n│ │ ┌───────────────────────────┐ │ │ │ Classifier: [API] / Mixed      │ │ │\n│ │ │ Paste requirement specs   │ │ │ │ Analyst: Gap found on Step 4   │ │ │\n│ │ │ ...                       │ │ │ │ Designer: Equivalence Partition│ │ │\n│ │ └───────────────────────────┘ │ │ └──────────────────────────────┘ │ │\n│ │ [ Run Pipeline ]              │ │ 4. Generated Test Cases          │ │\n│ ├───────────────────────────────┤ │ [ Create in ADO ]   [ Export ]   │ │\n│ │ 2. Active LLM Configuration   │ │ ┌──────────────────────────────┐ │ │\n│ │ Provider: Claude Sonnet 3.5   │ │ │ - TC1: Validate user signup  │ │ │\n│ │ Mode:     BDD (Gherkin)       │ │ │ - TC2: Edge-case rate limiting│ │ │\n│ └───────────────────────────────┘ └──────────────────────────────────┘ │\n└────────────────────────────────────────────────────────────────────────┘\n```\n\n---\n\n## Configuration\n\nTestForge is highly configurable via environment variables in the `.env` file. A central configuration module handles these defaults:\n\n### Server \u0026 System Configurations\n\n| Variable                  | Default           | Required For / Description                                                              |\n| :------------------------ | :---------------- | :-------------------------------------------------------------------------------------- |\n| `PORT`                    | `3010`            | Port the Express server listens on.                                                     |\n| `AUTH_TOKEN`              | —                 | Secure all `/api/*` endpoints. Client must supply `Authorization: Bearer \u003cAUTH_TOKEN\u003e`. |\n| `LOG_LEVEL`               | `info`            | Logger severity limit (`trace`, `debug`, `info`, `warn`, `error`, `fatal`, `silent`).   |\n| `MAX_JSON_BODY_SIZE`      | `2mb`             | Maximum JSON request payload size accepted.                                             |\n| `MAX_FILE_SIZE`           | `10485760` (10MB) | Max size in bytes allowed for document file uploads.                                    |\n| `LOGS_DIR`                | `logs`            | Folder directory relative to root to store persisted JSON audit run logs.               |\n| `PIPELINE_RATE_LIMIT_MAX` | `10`              | Rate limit: maximum orchestrated pipeline executions per minute per IP.                 |\n| `GENERAL_RATE_LIMIT_MAX`  | `60`              | Rate limit: maximum overall API requests per minute per IP.                             |\n\n### AI Pipeline Configuration\n\n| Variable                   | Default       | Description                                                       |\n| :------------------------- | :------------ | :---------------------------------------------------------------- |\n| `AGENT_TIMEOUT_MS`         | `60000` (60s) | Max milliseconds allowed for each agent's LLM generation request. |\n| `AGENT_MAX_RETRIES`        | `2`           | Maximum retry attempts if an LLM invocation fails.                |\n| `MAX_INPUT_LENGTH`         | `50000`       | Max character length allowed for raw text requirement input.      |\n| `MAX_SYSTEM_PROMPT_LENGTH` | `100000`      | Max characters permitted for combined system prompts.             |\n\n### Azure DevOps \u0026 Chrome (CDP) Configurations\n\n| Variable                | Default         | Description                                                                                       |\n| :---------------------- | :-------------- | :------------------------------------------------------------------------------------------------ |\n| `ADO_ORG`               | —               | Pre-fills Azure DevOps organization name in settings.                                             |\n| `ADO_PROJECT`           | —               | Pre-fills Azure DevOps project name in settings.                                                  |\n| `ADO_PAT`               | —               | Azure DevOps Personal Access Token (for programmatic `/api/ado/run-pat` test case push).          |\n| `CHROME_PATH`           | _Auto-detected_ | Custom filesystem path to the Chrome/Edge browser binary executable.                              |\n| `CHROME_CDP_PORT`       | `9222`          | Port to hook into browser session via Chrome DevTools Protocol.                                   |\n| `KEEP_CHROME_OPEN`      | `false`         | Keep debug browser window open after completing the test case push.                               |\n| `ADO_CREATION_DELAY_MS` | `300`           | Delay in milliseconds between consecutive work item creation requests (prevents ADO rate issues). |\n\n### LLM Providers Configuration\n\n| Provider             | Variable           | Default Model              | Key / Endpoint Setup                                              |\n| :------------------- | :----------------- | :------------------------- | :---------------------------------------------------------------- |\n| **Ollama (Local)**   | `OLLAMA_URL`       | `llama3.2`                 | Defaults: URL `http://localhost:11434`, model `llama3.2`          |\n| **OpenAI**           | `OPENAI_API_KEY`   | `gpt-4o`                   | Get key at [platform.openai.com](https://platform.openai.com)     |\n| **Anthropic Claude** | `CLAUDE_API_KEY`   | `claude-sonnet-4-20250514` | Get key at [console.anthropic.com](https://console.anthropic.com) |\n| **Google Gemini**    | `GOOGLE_API_KEY`   | `gemini-2.0-flash`         | Get key at [aistudio.google.com](https://aistudio.google.com)     |\n| **OpenCode**         | `OPENCODE_API_KEY` | `deepseek-v4-flash-free`   | API endpoint `https://api.opencode.ai/v1` via `OPENCODE_URL`      |\n\n---\n\n## API Reference\n\nTestForge can be operated programmatically. All API endpoints support the optional `Authorization` header if `AUTH_TOKEN` is configured.\n\n### General \u0026 Setup Routes\n\n- **`GET /health`** / **`GET /ready`**: System health and writable logs status checks.\n- **`GET /api/config`**: Fetches environment config statuses, supported LLM providers, and settings schema.\n- **`GET /api/agents`**: Returns a list of active agent prompt personas.\n\n### Pipeline Execution\n\n- **`POST /api/agents/run`**: Begins the orchestrated agent pipeline using SSE.\n  - **Payload**:\n    ```json\n    {\n      \"input\": \"User requirements text here\",\n      \"provider\": \"ollama\",\n      \"providerConfig\": { \"model\": \"llama3.2\" },\n      \"mode\": \"regular\"\n    }\n    ```\n- **`GET /api/agents/run/:runId/status`**: Get status/summary details of pipeline output by `runId`.\n- **`GET /api/agents/run/:runId/export`**: Export structured run logs as plain text.\n\n### Azure DevOps Actions\n\n- **`POST /api/ado/launch-chrome`**: Launch Chrome in debug mode (port 9222) preloaded to Dev DevOps portal URL.\n- **`POST /api/ado/fetch-work-item`**: Connects via CDP to fetch raw text data from an active work item.\n  - **Payload**: `{ \"org\": \"myOrg\", \"project\": \"myProject\", \"id\": \"12345\" }`\n- **`POST /api/ado/run`**: Publishes test cases using browser CDP session credentials. (SSE stream)\n- **`POST /api/ado/run-pat`**: Publishes test cases headlessly using configured server-side `ADO_PAT`. (SSE stream)\n\n### Document Processing\n\n- **`POST /api/parse/document`**: Multi-part upload endpoint to parse text from `.pdf`, `.docx`, `.md`, or `.txt` files.\n\n---\n\n## Development \u0026 Testing\n\n### Project Structure\n\n```\ntestforge/\n├── server.js                 # Express server — registers routes \u0026 server setup\n├── config.js                 # Central config manager (env reading + validation defaults)\n├── validators.js             # Zod validation schemas for all incoming API routes\n├── types.js                  # Shared type definitions for structures\n├── logger.js                 # Pino logger configuration\n├── llm-caller.js             # General abstraction for invoking LLM APIs with retries\n├── llm-providers.js          # Specific LLM providers adapters \u0026 configuration schemas\n├── agent-pipeline.js         # Sequential multi-agent pipeline orchestration logic\n├── agents/                   # Directory containing Markdown prompts defining agent personas\n│   ├── 00-classifier.md\n│   ├── 01-requirements-analyst.md\n│   ├── 02-ui-designer.md\n│   ├── 02-api-designer.md\n│   ├── 02-mock-designer.md\n│   ├── 03-test-case-writer.md\n│   └── 03-test-case-writer-bdd.md\n├── client/                   # React frontend application (Vite-powered)\n│   ├── src/\n│   │   ├── App.jsx           # Entry dashboard page\n│   │   ├── components/       # UI tabs, input sources, log, and settings components\n│   │   ├── hooks/            # usePipeline hooks for SSE event tracking\n│   │   ├── styles/           # Tokens, layout, and component stylesheets\n│   │   └── utils/            # Shared formatting helpers\n│   ├── public/               # Static assets \u0026 public stylesheets\n│   └── package.json\n├── package.json              # Main workspace scripts and package metadata\n└── .env.example\n```\n\n### Running Tests\n\nUnit and integration tests are configured for both backend components (using Jest) and frontend client hooks (using Vitest).\n\n#### 1. Backend / Server Tests\n\nExecuted using Jest. Includes coverage for endpoints, config parsing, and pipeline orchestration.\n\n```bash\n# Run server test suite\nnpm test\n\n# Run tests in watch mode\nnpm run test:watch\n```\n\n#### 2. Frontend / Client Tests\n\nExecuted using Vitest. Verifies rendering hooks and state selectors.\n\n```bash\n# From client folder\ncd client\nnpm test\n\n# Run client tests in watch mode\nnpm run test:watch\n```\n\n### Linting \u0026 Code Formatting\n\nWe use ESLint and Prettier to enforce styling guidelines. Run these checks locally before pushing code:\n\n```bash\n# Check code style constraints\nnpm run lint\n\n# Automatically resolve minor styling/formatting problems\nnpm run lint:fix\nnpm run format\n```\n\n---\n\n## Contributing\n\nWe welcome contributions to TestForge! Before submitting a pull request:\n\n1. Ensure all formatting and lint constraints pass cleanly (`npm run format:check` \u0026 `npm run lint`).\n2. Make sure both backend and frontend unit test suites pass completely (`npm test` and `cd client \u0026\u0026 npm test`).\n3. Follow the branch naming convention: `feat/amazing-feature` or `fix/critical-bug`.\n\n---\n\n## License\n\nMIT © 2026 Sritaj Kumar Patel\n\n---\n\n\u003cdiv align=\"center\"\u003e\n  \u003cp\u003eBuilt by \u003ca href=\"https://github.com/sritajkumarpatel\"\u003eSritaj Patel\u003c/a\u003e\u003c/p\u003e\n  \u003cp\u003eIf TestForge saves you time, \u003cstrong\u003estar the repo ⭐\u003c/strong\u003e — it helps others find it too.\u003c/p\u003e\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsritajkumarpatel%2Ftestforge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsritajkumarpatel%2Ftestforge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsritajkumarpatel%2Ftestforge/lists"}