{"id":50782707,"url":"https://github.com/da0101/test-inspector","last_synced_at":"2026-06-12T05:01:25.110Z","repository":{"id":358355805,"uuid":"1240300918","full_name":"da0101/test-inspector","owner":"da0101","description":"Catch theater, weak, and missing unit tests in VS Code. Local-first detective with line-anchored evidence and an optional anti-hallucination AI second opinion (OpenAI / Claude / Gemini).","archived":false,"fork":false,"pushed_at":"2026-05-16T23:47:44.000Z","size":374,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-17T01:41:58.329Z","etag":null,"topics":["anthropic","code-quality","developer-tools","flutter","gemini","llm","local-first","openai","react","test-quality","testing","typescript","vscode","vscode-extension"],"latest_commit_sha":null,"homepage":"https://github.com/da0101/test-inspector#readme","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/da0101.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":"ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-05-16T01:31:03.000Z","updated_at":"2026-05-16T23:53:16.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/da0101/test-inspector","commit_stats":null,"previous_names":["da0101/test-inspector"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/da0101/test-inspector","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/da0101%2Ftest-inspector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/da0101%2Ftest-inspector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/da0101%2Ftest-inspector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/da0101%2Ftest-inspector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/da0101","download_url":"https://codeload.github.com/da0101/test-inspector/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/da0101%2Ftest-inspector/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34229624,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-12T02:00:06.859Z","response_time":109,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anthropic","code-quality","developer-tools","flutter","gemini","llm","local-first","openai","react","test-quality","testing","typescript","vscode","vscode-extension"],"created_at":"2026-06-12T05:01:24.196Z","updated_at":"2026-06-12T05:01:25.073Z","avatar_url":"https://github.com/da0101.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Test Inspector\n\n\u003e A local-first VS Code extension that names which of your unit tests are **theater**, **weak**, or **missing** — and explains why, in plain English, with line-anchored evidence.\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](./LICENSE)\n[![VS Code ^1.90](https://img.shields.io/badge/VS%20Code-%5E1.90.0-blue.svg)](https://code.visualstudio.com/)\n[![Tests](https://img.shields.io/badge/tests-68%2F68-brightgreen.svg)](./test/unit)\n[![Version](https://img.shields.io/badge/version-1.0.0-blue.svg)](./package.json)\n\n---\n\n## Why this exists\n\nA growing share of new unit tests in modern codebases are written by LLM\nassistants (Copilot, Cursor, Claude) or by developers still building test\ntaste. The failure modes are predictable: tests that pass without proving\nanything — `expect(true).toBe(true)`, mock-only assertions, snapshot-only\nfiles, \"render didn't throw\" widget tests, tests that mock the very unit\nunder test.\n\n**Existing test extensions answer \"did the tests pass?\". Test Inspector\nanswers \"do the tests prove anything?\".** When 100% green can hide 60%\nuseless tests, this extension surfaces the bad ones — by name, with\nevidence, and (optionally) with a verified second opinion from an AI\nreviewer that cannot hallucinate beyond the file content.\n\n---\n\n## Features\n\n- 🔴 **Theater / 🟡 Weak / ⚪ Missing / 🟢 Strong** verdicts on every test and source file\n- **Line-anchored evidence** — every signal points at a specific line, copy-pasteable to the editor\n- **Multi-project monorepos** — Flutter app + Firebase Functions + React frontend, all scanned together with per-project tabs\n- **Reactive KPI tiles** — click `Theater` to filter, switch project tab and the counts recompute for that scope\n- **Optional AI reviewer** — OpenAI / Anthropic Claude / Google Gemini. Strict anti-hallucination contract: every claim the LLM makes must cite a real line and a verbatim excerpt; fabricated anchors are dropped before display\n- **Inline AI Reviewer sidebar** — manage provider, model, and API key without ever leaving the sidebar\n- **Mark Reviewed** — false-positive cards hide with content-hash invalidation (they come back if the file changes)\n- **Per-card Rescan** + **Markdown export** for PR descriptions\n- **Local-first** — every core feature works without any API key. The AI is purely additive.\n- **Never mutates source** — Test Inspector reads, scores, reports. You make the changes.\n\n### Frameworks supported\n\n| Framework | Detection | Coverage parsing | Quality smells |\n|---|---|---|---|\n| **React** (Jest / Vitest) | ✅ | LCOV, Istanbul JSON (partial) | ✅ trivial-assertion, mocks-unit-under-test, mock-only, vague-title, snapshot-only, render-only, orphan-test |\n| **Flutter / Dart** | ✅ | LCOV | ✅ trivial-Dart-assertion, render-only-widget-test, Mocktail mock-only, vague-title |\n| **Firebase Functions** | ✅ | LCOV (Istanbul) | Same as React |\n| **Django** | ✅ | coverage.py JSON / XML | ✅ skipped-test (more coming) |\n| **FastAPI** | ✅ | coverage.py JSON / XML | ✅ skipped-test (more coming) |\n| **Vue.js** | 🟡 Detected via Node | — | Inherits React rules; native adapter on the roadmap |\n\n---\n\n## Install\n\n### From source (current)\n\n```bash\ngit clone https://github.com/da0101/test-inspector.git\ncd test-inspector\n./scripts/install.sh\n```\n\nThe script does, in order:\n\n1. Cleans `out/` (`rm -rf out`) — avoids stale build artifacts\n2. `npm install` (if `node_modules/` is missing)\n3. `npm run compile` (TypeScript → `out/`)\n4. `npm test` (runs the 68-test unit suite)\n5. `npx @vscode/vsce package` → produces `test-inspector-\u003cversion\u003e.vsix`\n6. `code --install-extension \u003cvsix\u003e --force` → installs into your local VS Code\n\nIf the `code` CLI is not on your PATH, the script prints fallback\ninstructions: open VS Code → Command Palette → `Extensions: Install from\nVSIX…` → pick the `.vsix` file the script produced.\n\n### Requirements\n\n- Node.js 18+ (20 recommended)\n- VS Code 1.90+\n- macOS, Linux, or Windows\n- `git` (for changed-file scoring)\n- An API key from OpenAI, Anthropic, or Google AI Studio — **only if** you want the AI reviewer layer (everything else works without)\n\n### From VS Code Marketplace\n\n🚧 Not yet published. Tracking issue: TODO. Until then, install from source.\n\n---\n\n## Quick start\n\n1. **Install** (see above).\n2. **Open** any project that contains tests — single-repo or monorepo.\n3. **Click the 🧪 Test Inspector icon** in the Activity Bar (left side).\n4. The sidebar auto-scans within ~1 second and shows verdicts grouped by project + verdict.\n5. **Click \"Open Case File\"** to see narrative explanations per case.\n6. (Optional) **Configure an AI reviewer** in the sidebar's \"AI Reviewer\" section, then click \"Ask AI reviewer\" on any card.\n\n---\n\n## Configuring the AI reviewer (optional)\n\nThe deterministic detective layer works without any AI. The AI reviewer is a\nsecond opinion that verifies — or challenges — the deterministic verdict.\n\n### Get an API key\n\n| Provider | Where | Free tier |\n|---|---|---|\n| **Google Gemini** ⭐ recommended for testing | https://aistudio.google.com/apikey | ✅ Generous free tier — `gemini-2.5-flash` up to ~1,500 requests/day |\n| **OpenAI** | https://platform.openai.com/api-keys | ❌ Pay-per-use, but `gpt-4o-mini` is ~$0.001 per card |\n| **Anthropic Claude** | https://console.anthropic.com/settings/keys | ❌ Pay-per-use, `claude-3-5-haiku-latest` ~$0.005 per card |\n\n### Add the key\n\n1. Open the Test Inspector sidebar (Activity Bar icon)\n2. In the **AI Reviewer** section:\n   - Pick your **Provider** from the dropdown\n   - Pick a **Model** (defaults are sensible per provider)\n   - Paste your API key into the **API key** field\n   - Click **Save** — the extension tests the key automatically\n3. You'll see ✅ Ready if the key works, or ✗ with the error if it doesn't.\n\nAPI keys are stored in VS Code's encrypted SecretStorage. They are **never**:\n- written to `settings.json`\n- logged to the OutputChannel\n- sent to the webview after storing\n- included in Markdown exports\n\n### Use it\n\nOn any case in the Case File panel, click **Ask AI reviewer**. The extension:\n\n1. Sends the case file (verdict + signals) + the target file content + up to 3 related files to the configured provider\n2. Asks for a strict JSON response with line citations\n3. **Verifies every cited line:excerpt against the actual file** before showing it\n4. Shows the verified explanation, anchors, and suggested fix inline in the card\n5. Reports how many anchors were dropped because they couldn't be verified (the trust signal)\n\n### Anti-hallucination contract\n\nThe LLM is constrained at five layers:\n\n1. **Strict JSON output** — non-JSON responses are rejected with a fallback regex-extract of the prose explanation only\n2. **Line-anchored claims** — every claim must cite `{lineNumber, excerpt, issue}`\n3. **Verification** — the excerpt must appear at the cited line in the actual file content, or the anchor is silently dropped\n4. **Verdict locked** — the LLM can AGREE or CHALLENGE the deterministic verdict but cannot replace it; the deterministic case file remains authoritative\n5. **User-explicit per-card** — never auto-runs; you click \"Ask AI reviewer\" on the card you care about\n\n---\n\n## Commands\n\nOpen via `Cmd+Shift+P` / `Ctrl+Shift+P`.\n\n| Command | What it does |\n|---|---|\n| `Test Inspector: Open Case File` | Opens the Case File panel as an editor tab |\n| `Test Inspector: Refresh` | Re-scans the workspace (also auto-runs on first sidebar open) |\n| `Test Inspector: Configure LLM (optional reviewer)` | Legacy QuickPick wizard — the sidebar form is preferred |\n| `Test Inspector: Run Tests in Current File` | Runs the test command for the active file (Phase C wiring) |\n| `Test Inspector: Export Case File` | Saves the current Case File as a Markdown report |\n\n---\n\n## Settings reference\n\n| Key | Default | Purpose |\n|---|---|---|\n| `testInspector.slowTestThresholdMs` | `1000` | Flag tests slower than this in the future runner integration |\n| `testInspector.maxWorkspaceFiles` | `8000` | Cap on files scanned per workspace folder (prevents runaway scans on huge monorepos) |\n| `testInspector.llm.provider` | `none` | One of: `none`, `openai`, `claude`, `gemini` |\n| `testInspector.llm.model` | `\"\"` | Per-provider model. Empty falls back to the provider's default |\n| `testInspector.llm.openaiBaseUrl` | `https://api.openai.com/v1` | Override for Azure OpenAI, OpenRouter, etc. |\n| `testInspector.llm.claudeBaseUrl` | (default) | Override Anthropic base URL |\n| `testInspector.llm.geminiBaseUrl` | (default) | Override Gemini base URL |\n\nPer-provider API keys live in VS Code SecretStorage under the keys\n`testInspector.llm.\u003cprovider\u003e.apiKey`. They are not exposed in `settings.json`.\n\n---\n\n## Update\n\nThe install script always builds from the current source — so to update,\njust `git pull` and re-run it:\n\n```bash\ncd test-inspector\ngit pull origin main\n./scripts/install.sh\n```\n\nThen in your VS Code window: `Cmd+Shift+P → Developer: Reload Window`\n(or quit + reopen the window) so VS Code picks up the new bytecode.\n\nYou can verify the new version is loaded by looking at the OUTPUT panel\n(`Cmd+Shift+P → View: Toggle Output → \"Test Inspector\"` filter). The first\nline on activation is `[activate] Test Inspector \u003cversion\u003e activated`.\n\n---\n\n## Uninstall\n\n```bash\ncode --uninstall-extension local.test-inspector\n```\n\nThen optionally remove the locally-tracked review state from any project\nyou scanned:\n\n```bash\n# In each workspace where you scanned:\nrm -rf .test-inspector\n```\n\n(Add `.test-inspector/` to your project's `.gitignore` if you scanned a\nproject regularly — the folder holds `reviewed.json` with the content\nhashes of cards you marked as reviewed.)\n\n---\n\n## How it works (30 seconds)\n\n```\n┌────────────────────────────────────────────────────────────────────────┐\n│                         VS Code Extension Host                         │\n│                                                                        │\n│  Activity Bar icon                                                     │\n│         │                                                              │\n│         ▼                                                              │\n│  ┌──────────────────────┐    ┌───────────────────────────┐             │\n│  │  Cases sidebar tree  │    │  AI Reviewer sidebar      │             │\n│  │  (per-project,       │    │  (provider / model / key) │             │\n│  │   per-verdict)       │    └───────────────────────────┘             │\n│  └─────────┬────────────┘                                              │\n│            │                                                           │\n│            ▼                                                           │\n│  ┌────────────────────────────────────────────────────────────────┐    │\n│  │  Case File webview                                             │    │\n│  │  ┌──────────────────────────────────────────────────────┐      │    │\n│  │  │ KPI tiles (Theater · Weak · Missing · Strong)        │      │    │\n│  │  │ Project tabs · Filter pills                          │      │    │\n│  │  └──────────────────────────────────────────────────────┘      │    │\n│  │  ┌──────────────────────────────────────────────────────┐      │    │\n│  │  │ 🔴 THEATER  flutter  test/auth_state_test.dart       │      │    │\n│  │  │ auth_state_test.dart — 2 trivial-assertion signals   │      │    │\n│  │  │ Theater test: the assertions are tautological…       │      │    │\n│  │  │ [Open file] [Copy] [Ask AI reviewer] [Evidence] …    │      │    │\n│  │  └──────────────────────────────────────────────────────┘      │    │\n│  └────────────────────────────────────────────────────────────────┘    │\n└────────────────────────────────────────────────────────────────────────┘\n       │                              │\n       ▼                              ▼\n   Local filesystem +       Optional OpenAI-compatible /\n   `git` CLI                Anthropic / Gemini HTTPS endpoint\n                            (API key in SecretStorage)\n```\n\nThe deterministic detective layer:\n\n1. **Adapter detection** — each framework's adapter finds projects in the workspace\n2. **Static discovery** — regex / AST parsing finds test files and test cases (no test execution)\n3. **Quality heuristics** — pattern detectors per language for trivial assertions, mock-only tests, render-only widgets, vague titles, etc.\n4. **Source-risk scoring** — criticality keywords + coverage + related-test count → which source files matter most\n5. **Synthesis** — combine signals into a per-file verdict (`THEATER` ≥ 60 weight / `WEAK` 1–59 / `STRONG` 0) plus a one-paragraph narrative explanation\n\nThe optional AI reviewer:\n\n6. **Grounded prompt** — sends the deterministic verdict + signals + the actual file content (line-numbered) to the configured provider\n7. **Structured JSON response** — strict schema with line-anchored citations\n8. **Verification** — every cited line:excerpt is checked against the file content; fabricated ones are dropped\n9. **Display** — verified explanation + suggested fix, with the \"X dropped\" trust signal visible\n\n---\n\n## Contributing\n\nWe follow Git Flow — `main` is sacred, all feature work branches from\n`develop`. See [CONTRIBUTING.md](./CONTRIBUTING.md) for the full workflow\n(branch model, semver rules, PR template, code conventions, bug reporting).\n\nCode owners are defined in [.github/CODEOWNERS](./.github/CODEOWNERS).\n\n### Development quick start\n\n```bash\ngit clone https://github.com/da0101/test-inspector.git\ncd test-inspector\nnpm install\nnpm test                   # 68 unit tests, all stack-aware\n```\n\nThen in VS Code:\n\n- Open the repo\n- Press `F5` → launches an Extension Development Host with the extension loaded\n- Open any project in the EDH window and click the Test Inspector icon\n\n### Project layout\n\n```\nsrc/\n├── extension.ts            extension entry, command registration, refresh pipeline\n├── adapters/               per-framework adapters (react, flutter, python, firebase)\n├── services/\n│   ├── caseFile.ts         verdict synthesis (THEATER/WEAK/MISSING/STRONG)\n│   ├── quality.ts          test-quality heuristics\n│   ├── sourceRisk.ts       criticality scoring\n│   ├── reviewed.ts         content-hash store for Mark Reviewed\n│   ├── exportMarkdown.ts   Case File Markdown export\n│   ├── heuristics/         LLM-pattern detectors (mocks-unit, mock-only, vague-title)\n│   └── llm/                provider abstraction + anti-hallucination prompt\n├── views/\n│   ├── caseFile/           webview (template split: constants, icons, render, script, style)\n│   ├── reviewer/           AI Reviewer sidebar webview\n│   └── casesView.ts        sidebar tree\ntest/\n├── fixtures/\u003cstack\u003e/       sample workspaces per stack\n└── unit/                   `node --test` specs (68 total)\n```\n\n---\n\n## Roadmap\n\nSee [ROADMAP.md](./ROADMAP.md) for the phased backlog. Highlights of what's\nstill planned:\n\n- Vue.js first-class adapter\n- AST-based heuristics (currently regex)\n- Istanbul JSON, Vitest, Flutter URI normalization for coverage\n- Persistence + scan-over-time trend\n- esbuild bundler for smaller VSIX and faster cold start\n- Public Marketplace listing\n- LLM \"Reviewer notes\" co-worker panel (cross-card pattern detection)\n\n---\n\n## License\n\n[MIT](./LICENSE) © 2026 Danil Ulmashev\n\nTest Inspector is open source. Contributions welcome — please read\n[CONTRIBUTING.md](./CONTRIBUTING.md) first.\n\n---\n\n## Acknowledgements\n\n- Built around the product thesis that **tests that exist are not the same as tests that prove behavior** — particularly true now that LLM-generated tests are a growing share of every test suite\n- Anti-hallucination design inspired by retrieval-augmented patterns: ground the model in the actual artifact, verify every claim, never let the model speak unsupported\n\n---\n\n## Reporting issues\n\nOpen an issue at https://github.com/da0101/test-inspector/issues with:\n\n- Test Inspector version (visible in `[activate]` line of the OUTPUT panel)\n- VS Code version + OS\n- A short reproduction (which framework, what verdict was wrong, what you expected)\n- The relevant snippet from the `Test Inspector` OUTPUT channel\n- If LLM-related: which provider + model, and the dropped-anchor count\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fda0101%2Ftest-inspector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fda0101%2Ftest-inspector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fda0101%2Ftest-inspector/lists"}