{"id":26702327,"url":"https://github.com/alexandresajus/codedex-voice","last_synced_at":"2025-08-01T04:14:45.765Z","repository":{"id":282314149,"uuid":"947319574","full_name":"AlexandreSajus/codedex-voice","owner":"AlexandreSajus","description":null,"archived":false,"fork":false,"pushed_at":"2025-03-13T22:14:40.000Z","size":4972,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-13T23:22:59.646Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"MDX","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AlexandreSajus.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-12T13:58:25.000Z","updated_at":"2025-03-13T22:14:43.000Z","dependencies_parsed_at":"2025-03-13T23:33:05.526Z","dependency_job_id":null,"html_url":"https://github.com/AlexandreSajus/codedex-voice","commit_stats":null,"previous_names":["alexandresajus/codedex-voice"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlexandreSajus%2Fcodedex-voice","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlexandreSajus%2Fcodedex-voice/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlexandreSajus%2Fcodedex-voice/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlexandreSajus%2Fcodedex-voice/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AlexandreSajus","download_url":"https://codeload.github.com/AlexandreSajus/codedex-voice/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245770531,"owners_count":20669391,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-27T02:37:21.559Z","updated_at":"2025-03-27T02:37:22.074Z","avatar_url":"https://github.com/AlexandreSajus.png","language":"MDX","funding_links":[],"categories":[],"sub_categories":[],"readme":"---\ntitle: Create a Voice Virtual Assistant\nauthor: Alexandre Sajus\nuid: TODO\ndatePublished: TODO\ndescription: Learn how to build a Voice Virtual Assistant using ElevenLabs API and Python for seamless AI-powered interactions.\npublished: TODO\nheader: TODO\nbannerImage: TODO\ntags:\n  - intermediate\n  - python\n  - AI\n---\n\n\u003cBannerImage link=\"TODO\" description=\"Title Image\" uid={true} cl=\"for-sidebar\"/\u003e\n\n# Create a Voice Virtual Assistant\n\n\u003cAuthorAvatar\nauthor_name=\"Alexandre Sajus\"\nauthor_avatar=\"/media/Alexandre.jpg\"\nusername=\"TODO\"\nuid={true}\n/\u003e\n\n\u003cBannerImage link=\"TODO\" description=\"Title Image\" uid={true}/\u003e\n\n**Prerequisites:** Python fundamentals, API usage\n**Versions:** Python 3.11, python-dotenv 1.0.1, elevenlabs 1.54.0, elevenlabs[pyaudio]\n**Read Time:** 60 minutes  \n\n## Introduction\n\nVoice assistants like Siri, Google Assistant, and Alexa have revolutionized the way we interact with technology. In this tutorial, we’ll learn how to create a Voice Virtual Assistant using [ElevenLabs](https://elevenlabs.io/) API and Python. This assistant will be able to engage in natural conversations and provide helpful responses in real time.\n\nThe final assistant will:\n\n- Process user voice and text input\n- Use ElevenLabs API for voice synthesis\n- Provide a seamless conversational experience\n\nHere is a sneak peek of the final assistant in action:\n\n\u003cp align=\"center\"\u003e\n    \u003cvideo width=\"1280\" height=\"480\" controls\u003e\n        \u003csource src=\"media/demo.mp4\" type=\"video/mp4\"\u003e\n    \u003c/video\u003e\n\u003c/p\u003e\n\nLet's dive in!\n\n## Setting Up the Environment\n\n### 1. Install Required Packages\n\nBefore we start, make sure you have Python installed. Then, install the required dependencies:\n\n```sh\npip install elevenlabs elevenlabs[pyaudio] python-dotenv\n```\n\nProcessing audio requires additional dependencies on Linux and MacOS:\n- For Linux, you need to install `portaudio19`:\n```sh\nsudo apt install portaudio19\n```\n- For MacOS, you need to install `portaudio`:\n```sh\nbrew install portaudio\n```\n\n### 2. Setting up ElevenLabs\n\nElevenLabs provides a Conversational AI API that we will use to create our Voice Assistant. - The API records the user's voice through the microphone\n- 🎤 It processes it to know when the user has finished speaking or is interrupting the assistant\n- 🤖 It calls an LLM model to generate a response\n- 📈 It synthesizes the response into speech\n- 🔊 It plays the synthesized speech through the speakers\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"media/elevenlabs_diagram.png\" alt=\"ElevenLabs Function Diagram\" width=\"80%\"/\u003e\n\u003c/p\u003e\n\n1. Sign up at [ElevenLabs](https://elevenlabs.io/app/sign-up) and follow the instructions to create an account.\n\n2. Once signed in, go to \"Conversational AI\"\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"media/conversationalAI.png\" alt=\"Conversational AI button on dashboard\" width=\"80%\"/\u003e\n\u003c/p\u003e\n\n3. Go to \"Agents\"\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"media/agents.png\" alt=\"Agents button on dashboard\" width=\"80%\"/\u003e\n\u003c/p\u003e\n\n4. Click on \"Start from blank\"\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"media/startFromBlank.png\" alt=\"Start from blank button\" width=\"80%\"/\u003e\n\u003c/p\u003e\n\n5. Create a \".env\" file at the root of your project folder. We will use this file to store our API credentials securely. This way they won't be hardcoded in the script. In this \".env\" file, add your Agent ID:\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"media/agentID.png\" alt=\"Agent ID\" width=\"80%\"/\u003e\n\u003c/p\u003e\n\n```bash\nAGENT_ID=your_agent_id\n```\n\n6. Go to the \"Security\" tab, enable the \"First message\" and \"System prompt\" overrides, and save. This will allow us to customize the assistant's first message and system prompt using Python code.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"media/security.png\" alt=\"Security tab\" width=\"80%\"/\u003e\n\u003c/p\u003e\n\n7. Click on your profile and go to \"API keys\". Create a new API key and copy it to your \".env\" file:\n\n```bash\nAPI_KEY=\"sk_XXX...XXX\"\n```\n\n**Please make sure to save your \".env\" file after adding the credentials.**\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"media/apiKeys.png\" alt=\"API keys\" width=\"80%\"/\u003e\n\u003c/p\u003e\n\nElevenLabs is now set up and ready to be used in our Python script!\n\n**Note:** ElevenLabs works with a credit system. When you sign up, you get 10,000 free credits which amount to 15 minutes of conversation. You can buy more credits if needed.\n\n\n## Building the Voice Assistant\n\nThe full code for this application is available [here](TODO).\n\n### 1. Load Environment Variables\n\nCreate a Python file (e.g., `voice_assistant.py`) and load your API credentials:\n\n```python\nimport os\nfrom dotenv import load_dotenv\n\nload_dotenv()\n\nAGENT_ID = os.getenv(\"AGENT_ID\")\nAPI_KEY = os.getenv(\"API_KEY\")\n```\n\n### 2. Configure ElevenLabs Conversation API\n\nWe will set up the ElevenLabs client and configure a conversation instance.\n\nWe'll start by importing the necessary modules:\n\n```python\nfrom elevenlabs.client import ElevenLabs\nfrom elevenlabs.conversational_ai.conversation import Conversation\nfrom elevenlabs.conversational_ai.default_audio_interface import DefaultAudioInterface\nfrom elevenlabs.types import ConversationConfig\n```\n\nWe will then configure the conversation with the agent's first message and system prompt. We are going to inform the assistant that the user has a schedule and prompt it to help the user. In this part you can customize:\n- The **user's name**: what the assistant will call the user\n- The **schedule**: the user's schedule that the assistant will use to provide help\n- The **prompt**: the message that the assistant will receive when the conversation starts to understand the context of the conversation\n- The **first message**: the first message the assistant will say to the user\n\n**Prompts** are used to provide context to the assistant and help it understand the user's needs. They can be used to provide information about the user's environment, preferences, or any other information that can help the assistant provide better responses.\n\n```python\nuser_name = \"Alex\"\nschedule = \"Sales Meeting with Taipy at 10:00; Gym with Sophie at 17:00\"\nprompt = f\"You are a helpful assistant. Your interlocutor has the following schedule: {schedule}.\"\nfirst_message = f\"Hello {user_name}, how can I help you today?\"\n```\n\nWe are then going to set this configuration to our ElevenLabs agent:\n\n```python\nconversation_override = {\n    \"agent\": {\n        \"prompt\": {\n            \"prompt\": prompt,\n        },\n        \"first_message\": first_message,\n    },\n}\n\nconfig = ConversationConfig(\n    conversation_config_override=conversation_override,\n    extra_body={},\n    dynamic_variables={},\n)\n\nclient = ElevenLabs(api_key=API_KEY)\nconversation = Conversation(\n    client,\n    AGENT_ID,\n    config=config,\n    requires_auth=True,\n    audio_interface=DefaultAudioInterface(),\n)\n```\n\n### 3. Implement Callbacks for Responses\n\nTo improve user experience, define callback functions to handle assistant responses. These functions will print the assistant's responses and user transcripts. We also define a function to handle the situation where the user interrupts the assistant:\n\n```python\ndef print_agent_response(response):\n    print(f\"Agent: {response}\")\n\ndef print_interrupted_response(original, corrected):\n    print(f\"Agent interrupted, truncated response: {corrected}\")\n\ndef print_user_transcript(transcript):\n    print(f\"User: {transcript}\")\n```\n\n### 4. Start the Voice Assistant Session\n\nFinally, initiate the conversation session:\n\n```python\nconversation = Conversation(\n    client,\n    AGENT_ID,\n    config=config,\n    requires_auth=True,\n    audio_interface=DefaultAudioInterface(),\n    callback_agent_response=print_agent_response,\n    callback_agent_response_correction=print_interrupted_response,\n    callback_user_transcript=print_user_transcript,\n)\n\nconversation.start_session()\n```\n\n## Running the Assistant\n\n**Please make sure your audio devices are correctly set up in your system settings before running the code.**\n\nExecute the script:\n\n```bash\npython voice_assistant.py\n```\n\nThe assistant will start listening for input and responding in real time!\n\nYou can stop the assistant at any time by closing the terminal.\n\n## Conclusion\n\nCongratulations! 🎉 You've successfully built a Voice Virtual Assistant using ElevenLabs API. You can extend its capabilities by integrating it with home automation, calendars, or other APIs to make it even more useful.\n\nStay creative and keep experimenting with AI-powered assistants!\n\n## More Resources\n\n- ElevenLabs Conversational AI [Overview](https://elevenlabs.io/docs/conversational-ai/overview)\n- ElevenLabs Python SDK [Documentation](https://elevenlabs.io/docs/conversational-ai/libraries/python)\n- Enable your assistant to execute Python functions with [Client Tools](https://elevenlabs.io/docs/conversational-ai/customization/tools-events/client-tools)\n- Provide documents as context to your assistant with [RAG](https://elevenlabs.io/docs/conversational-ai/customization/knowledge-base/rag)\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falexandresajus%2Fcodedex-voice","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falexandresajus%2Fcodedex-voice","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falexandresajus%2Fcodedex-voice/lists"}