{"id":37085294,"url":"https://github.com/voiceeval/voiceeval","last_synced_at":"2026-01-14T10:29:08.103Z","repository":{"id":325917052,"uuid":"1103324974","full_name":"voiceeval/voiceeval","owner":"voiceeval","description":"VoiceEval: World-Class Open-source Voice AI Testing Framework","archived":false,"fork":false,"pushed_at":"2025-11-24T18:40:34.000Z","size":11,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-11-27T10:11:49.389Z","etag":null,"topics":["open-source","pytest","python","testing","testing-framework","voice-agents","voice-ai","voice-testing"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/voiceeval.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-24T18:10:29.000Z","updated_at":"2025-11-24T19:16:41.000Z","dependencies_parsed_at":"2025-11-27T11:11:35.039Z","dependency_job_id":null,"html_url":"https://github.com/voiceeval/voiceeval","commit_stats":null,"previous_names":["voiceeval/voiceeval"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/voiceeval/voiceeval","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voiceeval%2Fvoiceeval","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voiceeval%2Fvoiceeval/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voiceeval%2Fvoiceeval/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voiceeval%2Fvoiceeval/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/voiceeval","download_url":"https://codeload.github.com/voiceeval/voiceeval/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voiceeval%2Fvoiceeval/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28417279,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T10:25:19.714Z","status":"ssl_error","status_checked_at":"2026-01-14T10:22:49.371Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["open-source","pytest","python","testing","testing-framework","voice-agents","voice-ai","voice-testing"],"created_at":"2026-01-14T10:29:07.422Z","updated_at":"2026-01-14T10:29:08.098Z","avatar_url":"https://github.com/voiceeval.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# VoiceEval\n\n\u003cdiv align=\"center\"\u003e\n\n**The Open-Source Voice AI Testing Framework**\n\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)[![PyPI version](https://badge.fury.io/py/voiceeval.svg)](https://pypi.org/project/voiceeval/)\n\n\n\u003c/div\u003e\n\n\n\n---\n\n## 🚧 Under Active Development\n\nVoiceEval is currently in **early development**. We're building the world's most comprehensive voice AI testing framework—**making voice agents reliable, testable, and production-ready**.\n\n**⭐ Star this repo** to follow our progress and get notified when we launch!\n\n---\n\n## What is VoiceEval?\n\nVoiceEval is an open-source testing framework built specifically for voice AI agents. As voice AI systems become critical business infrastructure, testing them manually doesn't scale—edge cases slip through, latency issues compound across the ASR-LLM-TTS pipeline, and there's no standardized way to measure quality.\n\nVoiceEval solves this by providing developers with comprehensive automated testing tools: measure transcription accuracy (WER), validate conversation flow, test task completion, monitor latency, and evaluate response quality using LLM-as-judge validation. All with familiar pytest-style workflows and seamless CI/CD integration.\n\n\n### Key Features (Coming Soon)\n\n- ✅ **Comprehensive Metrics**: WER, latency, task completion, conversation flow\n- ✅ **LLM-as-Judge**: Intelligent evaluation using GPT-4, Claude, local LLMs\n- ✅ **Pytest Integration**: Familiar testing workflow developers already know\n- ✅ **CI/CD Native**: GitHub Actions, GitLab CI, Jenkins ready\n- ✅ **Enterprise Ready**: SOC2 compliant, self-hosted options\n\n---\n## Roadmap\nComing Soon...\n\n## Contributing\n\nWe welcome contributions! More details coming soon.\n\nFor now:\n- 🐛 [Report bugs](https://github.com/voiceeval/voiceeval/issues)\n- 💡 [Request features](https://github.com/voiceeval/voiceeval/issues)\n- ⭐ Star the repo to show support!\n\n---\n\n## License\n\nApache 2.0 - See [LICENSE](LICENSE) for details.\n\n---\n\n\u003cdiv align=\"center\"\u003e\n\n**Built with ❤️ by developers who believe voice AI should just work.**\n\n\u003c/div\u003e\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvoiceeval%2Fvoiceeval","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvoiceeval%2Fvoiceeval","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvoiceeval%2Fvoiceeval/lists"}