Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/voxos-ai/voxprobe
A python package for automated testing and evaluation for voice ai agents
https://github.com/voxos-ai/voxprobe
Last synced: 10 days ago
JSON representation
A python package for automated testing and evaluation for voice ai agents
- Host: GitHub
- URL: https://github.com/voxos-ai/voxprobe
- Owner: voxos-ai
- Created: 2024-08-28T06:10:17.000Z (4 months ago)
- Default Branch: develop
- Last Pushed: 2024-09-02T14:14:27.000Z (4 months ago)
- Last Synced: 2024-11-07T19:15:45.565Z (2 months ago)
- Language: Python
- Homepage:
- Size: 144 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome_ai_agents - Voxprobe - A python package for automated testing and evaluation for voice ai agents (Building / Testing)
- awesome_ai_agents - Voxprobe - A python package for automated testing and evaluation for voice ai agents (Building / Testing)
README
# VoxProbe
VoxProbe is a tool for automated testing and evaluation of Voice AI agents.
## Installation
```
pip install -e .
```
## Usage
As a package:
```python
from voxprobe import VoxProbe
from voxprobe.datasets import Dataset
from voxprobe.agents import BolnaAgent
from voxprobe.testing.telephony_tester.twilio_tester import TwilioTester
bolna_agent = BolnaAgent(api_key = "")
bolna_agent.pull()
dataset = Dataset(agent_prompt)
twilio_tester = TwilioTester(bolna_agent, dataset,
twilio_account_sid="",
twilio_auth_token="",
twilio_incoming_number="")
# Run the tester
twilio_tester.run()
```
As a CLI tool:
```
voxprobe [command] [arguments]
```
Available commands:
- import: Import an agent
- generate: Generate seed data
- test: Run automated tests
- ingest: Ingest recordings
- evaluate: Evaluate conversations
## Features
- Import agents from various platforms (Vocode, Retell, Vapi, Bolna)
- Generate seed data for testing
- Automate testing using generated datasets
- Ingest call recordings and create golden datasets
- Create and use evaluation datasets
- Evaluate actual conversations
## Development
To start developing, navigate to the respective directories and implement the TODO items in each file.