{"id":27344470,"url":"https://github.com/PromtEngineer/Verbi","last_synced_at":"2025-04-12T17:06:29.618Z","repository":{"id":240636604,"uuid":"803104512","full_name":"PromtEngineer/Verbi","owner":"PromtEngineer","description":"A modular voice assistant application for experimenting with state-of-the-art transcription, response generation, and text-to-speech models. Supports OpenAI, Groq, Elevanlabs, CartesiaAI, and Deepgram APIs, plus local models via Ollama. Ideal for research and development in voice technology.","archived":false,"fork":false,"pushed_at":"2024-11-05T13:58:29.000Z","size":311,"stargazers_count":954,"open_issues_count":19,"forks_count":178,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-04-06T20:02:06.894Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PromtEngineer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-20T04:36:58.000Z","updated_at":"2025-04-06T19:03:27.000Z","dependencies_parsed_at":"2024-05-20T09:27:49.843Z","dependency_job_id":"27075d69-c675-4946-8417-0c1f05fdf93b","html_url":"https://github.com/PromtEngineer/Verbi","commit_stats":null,"previous_names":["promtengineer/verbi"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PromtEngineer%2FVerbi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PromtEngineer%2FVerbi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PromtEngineer%2FVerbi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PromtEngineer%2FVerbi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PromtEngineer","download_url":"https://codeload.github.com/PromtEngineer/Verbi/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248602311,"owners_count":21131615,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-12T17:02:18.504Z","updated_at":"2025-04-12T17:06:29.609Z","avatar_url":"https://github.com/PromtEngineer.png","language":"Python","funding_links":[],"categories":["语音识别与合成_其他"],"sub_categories":["资源传输下载"],"readme":"# VERBI - Voice Assistant 🎙️\n\u003cp align=\"center\"\u003e\n\u003ca href=\"https://trendshift.io/repositories/11584\" target=\"_blank\"\u003e\u003cimg src=\"https://trendshift.io/api/badge/repositories/11584\" alt=\"PromtEngineer%2FVerbi | Trendshift\" style=\"width: 250px; height: 55px;\" width=\"250\" height=\"55\"/\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n[![GitHub Stars](https://img.shields.io/github/stars/PromtEngineer/Verbi?style=social)](https://github.com/PromtEngineer/Verbi/stargazers)\n[![GitHub Forks](https://img.shields.io/github/forks/PromtEngineer/Verbi?style=social)](https://github.com/PromtEngineer/Verbi/network/members)\n[![GitHub Issues](https://img.shields.io/github/issues/PromtEngineer/Verbi)](https://github.com/PromtEngineer/Verbi/issues)\n[![GitHub Pull Requests](https://img.shields.io/github/issues-pr/PromtEngineer/Verbi)](https://github.com/PromtEngineer/Verbi/pulls)\n[![License](https://img.shields.io/github/license/PromtEngineer/Verbi)](https://github.com/PromtEngineer/Verbi/blob/main/LICENSE)\n\n## Motivation ✨✨✨\n\nWelcome to the Voice Assistant project! 🎙️ Our goal is to create a modular voice assistant application that allows you to experiment with state-of-the-art (SOTA) models for various components. The modular structure provides flexibility, enabling you to pick and choose between different SOTA models for transcription, response generation, and text-to-speech (TTS). This approach facilitates easy testing and comparison of different models, making it an ideal platform for research and development in voice assistant technologies. Whether you're a developer, researcher, or enthusiast, this project is for you!\n\n## Features 🧰\n\n- **Modular Design**: Easily switch between different models for transcription, response generation, and TTS.\n- **Support for Multiple APIs**: Integrates with OpenAI, Groq, and Deepgram APIs, along with placeholders for local models.\n- **Audio Recording and Playback**: Record audio from the microphone and play generated speech.\n- **Configuration Management**: Centralized configuration in `config.py` for easy setup and management.\n\n## Project Structure 📂\n\n```plaintext\nvoice_assistant/\n├── voice_assistant/\n│   ├── __init__.py\n│   ├── audio.py\n│   ├── api_key_manager.py\n│   ├── config.py\n│   ├── transcription.py\n│   ├── response_generation.py\n│   ├── text_to_speech.py\n│   ├── utils.py\n│   ├── local_tts_api.py\n│   ├── local_tts_generation.py\n├── .env\n├── run_voice_assistant.py\n├── setup.py\n├── requirements.txt\n└── README.md\n```\n\n## Setup Instructions  📋\n\n#### Prerequisites ✅\n\n- Python 3.10 or higher\n- Virtual environment (recommended)\n\n#### Step-by-Step Instructions 🔢\n\n1. 📥 **Clone the repository**\n\n```shell\n   git clone https://github.com/PromtEngineer/Verbi.git\n   cd Verbi\n```\n2. 🐍 **Set up a virtual environment**\n\n  Using `venv`:\n\n```shell\n    python -m venv venv\n    source venv/bin/activate  # On Windows use `venv\\Scripts\\activate`\n```\n  Using `conda`:\n\n```shell\n    conda create --name verbi python=3.10\n    conda activate verbi\n```\n3.  📦 **Install the required packages**\n\n```shell\n   pip install -r requirements.txt\n```\n4. 🛠️ **Set up the environment variables**\n\nCreate a  `.env` file in the root directory and add your API keys:\n```shell\n    OPENAI_API_KEY=your_openai_api_key\n    GROQ_API_KEY=your_groq_api_key\n    DEEPGRAM_API_KEY=your_deepgram_api_key\n    LOCAL_MODEL_PATH=path/to/local/model\n```\n5. 🧩 **Configure the models**\n\nEdit config.py to select the models you want to use:\n\n```shell\n    class Config:\n        # Model selection\n        TRANSCRIPTION_MODEL = 'groq'  # Options: 'openai', 'groq', 'deepgram', 'fastwhisperapi' 'local'\n        RESPONSE_MODEL = 'groq'       # Options: 'openai', 'groq', 'ollama', 'local'\n        TTS_MODEL = 'deepgram'        # Options: 'openai', 'deepgram', 'elevenlabs', 'local', 'melotts'\n\n        # API keys and paths\n        OPENAI_API_KEY = os.getenv(\"OPENAI_API_KEY\")\n        GROQ_API_KEY = os.getenv(\"GROQ_API_KEY\")\n        DEEPGRAM_API_KEY = os.getenv(\"DEEPGRAM_API_KEY\")\n        LOCAL_MODEL_PATH = os.getenv(\"LOCAL_MODEL_PATH\")\n```\n\nIf you are running LLM locally via [Ollama](https://ollama.com/), make sure the Ollama server is runnig before starting verbi. \n\n6. 🔊 **Configure ElevenLabs Jarvis' Voice**\n- Voice samples [here](https://github.com/PromtEngineer/Verbi/tree/main/voice_samples).\n- Follow this [link](https://elevenlabs.io/app/voice-lab/share/de3746fa51a09e771604d74b5d1ff6797b6b96a5958f9de95cef544dde31dad9/WArWzu0z4mbSyy5BfRKM) to add the Jarvis voice to your ElevenLabs account.\n- Name the voice 'Paul J.' or, if you prefer a different name, ensure it matches the ELEVENLABS_VOICE_ID variable in the text_to_speech.py file.\n\n7. 🏃 **Run the voice assistant**\n\n```shell\n   python run_voice_assistant.py\n```\n8. 🎤 **Install FastWhisperAPI**\n\n   _Optional step if you need a local transcription model_\n\n   ***Clone the repository***\n   ```shell\n      cd..\n      git clone https://github.com/3choff/FastWhisperAPI.git\n      cd FastWhisperAPI\n   ```\n   ***Install the required packages:***\n   ```shell\n      pip install -r requirements.txt\n   ```\n   ***Run the API***\n   ```shell\n      fastapi run main.py\n   ```\n   ***Alternative Setup and Run Methods***\n\n   The API can also run directly on a Docker container or in Google Colab.\n\n   ***Docker:***\n\n   ***Build a Docker container:***\n   ```shell\n      docker build -t fastwhisperapi .\n   ```\n   ***Run the container***\n   ```shell\n      docker run -p 8000:8000 fastwhisperapi\n   ```\n   Refer to the repository documentation for the Google Colab method: https://github.com/3choff/FastWhisperAPI/blob/main/README.md\n\n8. 🎤 **Install Local TTS - MeloTTS**\n\n   _Optional step if you need a local Text to Speech model_\n\n   ***Install MeloTTS from Github***\n\n   Use the following [link](https://github.com/myshell-ai/MeloTTS/blob/main/docs/install.md#linux-and-macos-install) to install MeloTTS for your operating system. \n\n   Once the package is installed on your local virtual environment, you can start the api server using the following command. \n   ```shell\n      python voice_assistant/local_tts_api.py\n   ```\n   The `local_tts_api.py` file implements as fastapi server that will listen to incoming text and will generate audio using MeloTTS model. \n   In order to use the local TTS model, you will need to update the `config.py` file by setting: \n\n   ```shell\n      TTS_MODEL = 'melotts'        # Options: 'openai', 'deepgram', 'elevenlabs', 'local', 'melotts'\n   ```\n   You can run the main file to start using verbi with local models. \n\n## Model Options ⚙️\n\n#### Transcription Models  🎤\n\n- **OpenAI**: Uses OpenAI's Whisper model.\n- **Groq**: Uses Groq's Whisper-large-v3 model.\n- **Deepgram**: Uses Deepgram's transcription model.\n- **FastWhisperAPI**: Uses FastWhisperAPI, a local transcription API powered by Faster Whisper.\n- **Local**: Placeholder for a local speech-to-text (STT) model.\n\n#### Response Generation Models  💬\n\n- **OpenAI**: Uses OpenAI's GPT-4 model.\n- **Groq**: Uses Groq's LLaMA model.\n- **Ollama**: Uses any model served via Ollama.\n- **Local**: Placeholder for a local language model.\n\n#### Text-to-Speech (TTS) Models  🔊\n\n- **OpenAI**: Uses OpenAI's TTS model with the 'fable' voice.\n- **Deepgram**: Uses Deepgram's TTS model with the 'aura-angus-en' voice.\n- **ElevenLabs**: Uses ElevenLabs' TTS model with the 'Paul J.' voice.\n- **Local**: Placeholder for a local TTS model.\n\n## Detailed Module Descriptions  📘\n\n- **`run_verbi.py`**: Main script to run the voice assistant.\n- **`voice_assistant/config.py`**: Manages configuration settings and API keys.\n- **`voice_assistant/api_key_manager.py`**: Handles retrieval of API keys based on configured models.\n- **`voice_assistant/audio.py`**: Functions for recording and playing audio.\n- **`voice_assistant/transcription.py`**: Manages audio transcription using various APIs.\n- **`voice_assistant/response_generation.py`**: Handles generating responses using various language models.\n- **`voice_assistant/text_to_speech.py`**: Manages converting text responses into speech.\n- **`voice_assistant/utils.py`**: Contains utility functions like deleting files.\n- **`voice_assistant/local_tts_api.py`**: Contains the api implementation to run the MeloTTS model.\n- **`voice_assistant/local_tts_generation.py`**: Contains the code to use the MeloTTS api to generated audio.\n- **`voice_assistant/__init__.py`**: Initializes the `voice_assistant` package.\n\n## Roadmap 🛤️🛤️🛤️\n\nHere's what's next for the Voice Assistant project:\n\n1. **Add Support for Streaming**: Enable real-time streaming of audio input and output.\n2. **Add Support for ElevenLabs and Enhanced Deepgram for TTS**: Integrate additional TTS options for higher quality and variety.\n3. **Add Filler Audios**: Include background or filler audios while waiting for model responses to enhance user experience.\n4. **Add Support for Local Models Across the Board**: Expand support for local models in transcription, response generation, and TTS.\n\n## Contributing 🤝\n\nWe welcome contributions from the community! If you'd like to help improve this project, please follow these steps:\n\n1. Fork the repository.\n2. Create a new branch (`git checkout -b feature-branch`).\n3. Make your changes and commit them (`git commit -m 'Add new feature'`).\n4. Push to the branch (`git push origin feature-branch`).\n5. Open a pull request detailing your changes.\n\n## Star History ✨✨✨\n\n[![Star History Chart](https://api.star-history.com/svg?repos=PromtEngineer/Verbi\u0026type=Date)](https://star-history.com/#PromtEngineer/Verbi\u0026Date)\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPromtEngineer%2FVerbi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FPromtEngineer%2FVerbi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPromtEngineer%2FVerbi/lists"}