{"id":42016487,"url":"https://github.com/voicetestdev/voicetest","last_synced_at":"2026-04-19T21:12:25.731Z","repository":{"id":333788774,"uuid":"1134393472","full_name":"voicetestdev/voicetest","owner":"voicetestdev","description":"Test harness for voice agents. Import from Retell, VAPI, Bland, LiveKit. Run autonomous simulations. Evaluate with LLM judges.","archived":false,"fork":false,"pushed_at":"2026-02-25T01:19:13.000Z","size":18711,"stargazers_count":8,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-25T07:21:04.274Z","etag":null,"topics":["bland","dspy","livekit","llm","retell","testing","vapi","voice-ai"],"latest_commit_sha":null,"homepage":"https://voicetest.dev","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/voicetestdev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-01-14T16:53:52.000Z","updated_at":"2026-02-25T01:14:25.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/voicetestdev/voicetest","commit_stats":null,"previous_names":["voicetestdev/voicetest"],"tags_count":25,"template":false,"template_full_name":null,"purl":"pkg:github/voicetestdev/voicetest","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voicetestdev%2Fvoicetest","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voicetestdev%2Fvoicetest/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voicetestdev%2Fvoicetest/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voicetestdev%2Fvoicetest/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/voicetestdev","download_url":"https://codeload.github.com/voicetestdev/voicetest/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voicetestdev%2Fvoicetest/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30054049,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-03T17:46:22.538Z","status":"ssl_error","status_checked_at":"2026-03-03T17:46:22.036Z","response_time":61,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bland","dspy","livekit","llm","retell","testing","vapi","voice-ai"],"created_at":"2026-01-26T03:15:20.098Z","updated_at":"2026-04-19T21:12:25.724Z","avatar_url":"https://github.com/voicetestdev.png","language":"Python","funding_links":[],"categories":["Developer Communities \u0026 Resources | 开发者社区与资源","Testing and Evaluation","Software"],"sub_categories":["Platforms \u0026 Tools | 开发平台与工具","AI \u0026 LLM Testing"],"readme":"[![PyPI](https://img.shields.io/pypi/v/voicetest)](https://pypi.org/project/voicetest/) [![Release](https://img.shields.io/github/v/release/voicetestdev/voicetest)](https://github.com/voicetestdev/voicetest/releases) [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) [![Tests](https://github.com/voicetestdev/voicetest/actions/workflows/test.yml/badge.svg)](https://github.com/voicetestdev/voicetest/actions/workflows/test.yml)\n\n\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"assets/logo-dark.svg\"\u003e\n  \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"assets/logo-light.svg\"\u003e\n  \u003cimg alt=\"voicetest\" src=\"assets/logo-light.svg\" width=\"300\"\u003e\n\u003c/picture\u003e\n\nOpen-source test harness for voice agent workflows.\n\n- **Simulate conversations** — LLM-powered users talk to your agent, LLM judges score the results\n- **Test any platform** — Retell, VAPI, LiveKit, Bland, Telnyx, or custom agents\n- **Convert between formats** — Import from one platform, export to another via a unified graph IR\n- **Diagnose failures** — Auto-fix broken prompts with an LLM-powered repair loop\n- **Run anywhere** — CLI, Web UI, REST API, CI/CD\n\n## Installation\n\n```bash\nuv tool install voicetest\n```\n\nOr add to a project (`uv run voicetest` to run):\n\n```bash\nuv add voicetest\n```\n\nOr with pip:\n\n```bash\npip install voicetest\n```\n\n## Quick start\n\nTry voicetest with a sample healthcare receptionist agent and 8 test cases:\n\n```bash\n# Set up an API key (free, no credit card at https://console.groq.com)\nexport GROQ_API_KEY=gsk_...\n\n# Load demo and start web UI\nvoicetest demo --serve\n```\n\n\u003e **Tip:** If you have [Claude Code](https://claude.ai/claude-code) installed, skip API key setup and use `claudecode/sonnet` as your model. See [Claude Code Passthrough](https://voicetest.dev/docs/configuration/#claude-code-passthrough).\n\n![Web UI Demo (light)](docs/demos/web-demo-light.gif)\n\n## Web UI\n\n```bash\nvoicetest serve\n```\n\nAgent import, graph visualization, test execution with real-time streaming transcripts, run history, diagnosis, and more at http://localhost:8000.\n\n## Platform support\n\nImport from any supported format, convert through the unified AgentGraph, and export to any other:\n\n```\nRetell CF ─────┐                  ┌───▶ Retell LLM\n               │                  │\nRetell LLM ────┼                  ├───▶ Retell CF\n               │                  │\nVAPI ──────────┼                  ├───▶ VAPI\n               │                  │\nBland ─────────┼───▶ AgentGraph ──┼───▶ Bland\n               │                  │\nTelnyx ────────┤                  ├───▶ Telnyx\n               │                  │\nLiveKit ───────┤                  ├───▶ LiveKit\n               │                  │\nXLSForm ───────┤                  ├───▶ Mermaid\n               │                  │\nCustom ────────┘                  └───▶ Voicetest JSON\n```\n\n| Platform | Import | Push | Sync | API Key Env Var                          |\n| -------- | ------ | ---- | ---- | ---------------------------------------- |\n| Retell   | ✓      | ✓    | ✓    | `RETELL_API_KEY`                         |\n| VAPI     | ✓      | ✓    | ✓    | `VAPI_API_KEY`                           |\n| Bland    | ✓      | ✓    |      | `BLAND_API_KEY`                          |\n| Telnyx   | ✓      | ✓    | ✓    | `TELNYX_API_KEY`                         |\n| LiveKit  | ✓      | ✓    | ✓    | `LIVEKIT_API_KEY` + `LIVEKIT_API_SECRET` |\n\n## CI/CD\n\nRun voice agent tests in GitHub Actions to catch regressions before production:\n\n```yaml\nname: Voice Agent Tests\non:\n  push:\n    paths: [\"agents/**\"]\n\njobs:\n  test:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v4\n      - uses: astral-sh/setup-uv@v5\n      - run: uv tool install voicetest\n      - run: voicetest run --agent agents/receptionist.json --tests agents/tests.json --all\n        env:\n          GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}\n```\n\n## Configuration\n\nSettings are stored in `.voicetest/settings.toml`:\n\n```toml\n[models]\nagent = \"groq/llama-3.1-8b-instant\"\nsimulator = \"groq/llama-3.1-8b-instant\"\njudge = \"groq/llama-3.1-8b-instant\"\n\n[run]\nmax_turns = 20\naudio_eval = false\nstreaming = false\n```\n\nAny [LiteLLM-compatible model](https://docs.litellm.ai/docs/providers) works — OpenAI, Anthropic, Google, Ollama, and more. See the [full configuration reference](https://voicetest.dev/docs/configuration/).\n\n## Documentation\n\nFull documentation is at [voicetest.dev/docs](https://voicetest.dev/docs/).\n\n| Topic                                                          | Description                                                  |\n| -------------------------------------------------------------- | ------------------------------------------------------------ |\n| [Getting Started](https://voicetest.dev/docs/getting-started/) | Install, demo, first test walkthrough                        |\n| [Core Concepts](https://voicetest.dev/docs/concepts/)          | Agent graphs, node types, test cases                         |\n| [CLI Reference](https://voicetest.dev/docs/cli/)               | All commands and options                                     |\n| [Features](https://voicetest.dev/docs/features/)               | Format conversion, diagnosis, audio eval, snippets, and more |\n| [Configuration](https://voicetest.dev/docs/configuration/)     | Models, settings, Claude Code, platform credentials          |\n| [Architecture](https://voicetest.dev/docs/architecture/)       | Engine internals, DI, storage                                |\n| [Development](https://voicetest.dev/docs/development/)         | Contributing, Docker setup, code quality                     |\n\n## Contact\n\nQuestions, feedback, or partnerships: [hello@voicetest.dev](mailto:hello@voicetest.dev)\n\n## License\n\nApache 2.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvoicetestdev%2Fvoicetest","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvoicetestdev%2Fvoicetest","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvoicetestdev%2Fvoicetest/lists"}