https://github.com/alexandresajus/codedex-voice
https://github.com/alexandresajus/codedex-voice
Last synced: 11 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/alexandresajus/codedex-voice
- Owner: AlexandreSajus
- Created: 2025-03-12T13:58:25.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-03-13T22:14:40.000Z (over 1 year ago)
- Last Synced: 2025-03-13T23:22:59.646Z (over 1 year ago)
- Language: MDX
- Size: 4.74 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
---
title: Create a Voice Virtual Assistant
author: Alexandre Sajus
uid: TODO
datePublished: TODO
description: Learn how to build a Voice Virtual Assistant using ElevenLabs API and Python for seamless AI-powered interactions.
published: TODO
header: TODO
bannerImage: TODO
tags:
- intermediate
- python
- AI
---
# Create a Voice Virtual Assistant
**Prerequisites:** Python fundamentals, API usage
**Versions:** Python 3.11, python-dotenv 1.0.1, elevenlabs 1.54.0, elevenlabs[pyaudio]
**Read Time:** 60 minutes
## Introduction
Voice assistants like Siri, Google Assistant, and Alexa have revolutionized the way we interact with technology. In this tutorial, we’ll learn how to create a Voice Virtual Assistant using [ElevenLabs](https://elevenlabs.io/) API and Python. This assistant will be able to engage in natural conversations and provide helpful responses in real time.
The final assistant will:
- Process user voice and text input
- Use ElevenLabs API for voice synthesis
- Provide a seamless conversational experience
Here is a sneak peek of the final assistant in action:
Let's dive in!
## Setting Up the Environment
### 1. Install Required Packages
Before we start, make sure you have Python installed. Then, install the required dependencies:
```sh
pip install elevenlabs elevenlabs[pyaudio] python-dotenv
```
Processing audio requires additional dependencies on Linux and MacOS:
- For Linux, you need to install `portaudio19`:
```sh
sudo apt install portaudio19
```
- For MacOS, you need to install `portaudio`:
```sh
brew install portaudio
```
### 2. Setting up ElevenLabs
ElevenLabs provides a Conversational AI API that we will use to create our Voice Assistant. - The API records the user's voice through the microphone
- 🎤 It processes it to know when the user has finished speaking or is interrupting the assistant
- 🤖 It calls an LLM model to generate a response
- 📈 It synthesizes the response into speech
- 🔊 It plays the synthesized speech through the speakers
1. Sign up at [ElevenLabs](https://elevenlabs.io/app/sign-up) and follow the instructions to create an account.
2. Once signed in, go to "Conversational AI"
3. Go to "Agents"
4. Click on "Start from blank"
5. Create a ".env" file at the root of your project folder. We will use this file to store our API credentials securely. This way they won't be hardcoded in the script. In this ".env" file, add your Agent ID:
```bash
AGENT_ID=your_agent_id
```
6. Go to the "Security" tab, enable the "First message" and "System prompt" overrides, and save. This will allow us to customize the assistant's first message and system prompt using Python code.
7. Click on your profile and go to "API keys". Create a new API key and copy it to your ".env" file:
```bash
API_KEY="sk_XXX...XXX"
```
**Please make sure to save your ".env" file after adding the credentials.**
ElevenLabs is now set up and ready to be used in our Python script!
**Note:** ElevenLabs works with a credit system. When you sign up, you get 10,000 free credits which amount to 15 minutes of conversation. You can buy more credits if needed.
## Building the Voice Assistant
The full code for this application is available [here](TODO).
### 1. Load Environment Variables
Create a Python file (e.g., `voice_assistant.py`) and load your API credentials:
```python
import os
from dotenv import load_dotenv
load_dotenv()
AGENT_ID = os.getenv("AGENT_ID")
API_KEY = os.getenv("API_KEY")
```
### 2. Configure ElevenLabs Conversation API
We will set up the ElevenLabs client and configure a conversation instance.
We'll start by importing the necessary modules:
```python
from elevenlabs.client import ElevenLabs
from elevenlabs.conversational_ai.conversation import Conversation
from elevenlabs.conversational_ai.default_audio_interface import DefaultAudioInterface
from elevenlabs.types import ConversationConfig
```
We will then configure the conversation with the agent's first message and system prompt. We are going to inform the assistant that the user has a schedule and prompt it to help the user. In this part you can customize:
- The **user's name**: what the assistant will call the user
- The **schedule**: the user's schedule that the assistant will use to provide help
- The **prompt**: the message that the assistant will receive when the conversation starts to understand the context of the conversation
- The **first message**: the first message the assistant will say to the user
**Prompts** are used to provide context to the assistant and help it understand the user's needs. They can be used to provide information about the user's environment, preferences, or any other information that can help the assistant provide better responses.
```python
user_name = "Alex"
schedule = "Sales Meeting with Taipy at 10:00; Gym with Sophie at 17:00"
prompt = f"You are a helpful assistant. Your interlocutor has the following schedule: {schedule}."
first_message = f"Hello {user_name}, how can I help you today?"
```
We are then going to set this configuration to our ElevenLabs agent:
```python
conversation_override = {
"agent": {
"prompt": {
"prompt": prompt,
},
"first_message": first_message,
},
}
config = ConversationConfig(
conversation_config_override=conversation_override,
extra_body={},
dynamic_variables={},
)
client = ElevenLabs(api_key=API_KEY)
conversation = Conversation(
client,
AGENT_ID,
config=config,
requires_auth=True,
audio_interface=DefaultAudioInterface(),
)
```
### 3. Implement Callbacks for Responses
To improve user experience, define callback functions to handle assistant responses. These functions will print the assistant's responses and user transcripts. We also define a function to handle the situation where the user interrupts the assistant:
```python
def print_agent_response(response):
print(f"Agent: {response}")
def print_interrupted_response(original, corrected):
print(f"Agent interrupted, truncated response: {corrected}")
def print_user_transcript(transcript):
print(f"User: {transcript}")
```
### 4. Start the Voice Assistant Session
Finally, initiate the conversation session:
```python
conversation = Conversation(
client,
AGENT_ID,
config=config,
requires_auth=True,
audio_interface=DefaultAudioInterface(),
callback_agent_response=print_agent_response,
callback_agent_response_correction=print_interrupted_response,
callback_user_transcript=print_user_transcript,
)
conversation.start_session()
```
## Running the Assistant
**Please make sure your audio devices are correctly set up in your system settings before running the code.**
Execute the script:
```bash
python voice_assistant.py
```
The assistant will start listening for input and responding in real time!
You can stop the assistant at any time by closing the terminal.
## Conclusion
Congratulations! 🎉 You've successfully built a Voice Virtual Assistant using ElevenLabs API. You can extend its capabilities by integrating it with home automation, calendars, or other APIs to make it even more useful.
Stay creative and keep experimenting with AI-powered assistants!
## More Resources
- ElevenLabs Conversational AI [Overview](https://elevenlabs.io/docs/conversational-ai/overview)
- ElevenLabs Python SDK [Documentation](https://elevenlabs.io/docs/conversational-ai/libraries/python)
- Enable your assistant to execute Python functions with [Client Tools](https://elevenlabs.io/docs/conversational-ai/customization/tools-events/client-tools)
- Provide documents as context to your assistant with [RAG](https://elevenlabs.io/docs/conversational-ai/customization/knowledge-base/rag)