{"id":28411458,"url":"https://github.com/lpalbou/voicellm","last_synced_at":"2025-07-25T05:37:20.105Z","repository":{"id":290030730,"uuid":"973161248","full_name":"lpalbou/VoiceLLM","owner":"lpalbou","description":"A modular Python library for voice interactions with AI systems, featuring high-quality TTS, STT with Whisper, and memory persistence.","archived":false,"fork":false,"pushed_at":"2025-04-27T01:08:41.000Z","size":51,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-05T23:06:05.549Z","etag":null,"topics":["ai","llm","python","speech-to-text","stt","text-to-speech","tts","voice","voice-assistant","voice-interface","voice-recognition","whisper"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/voicellm/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lpalbou.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-26T11:51:19.000Z","updated_at":"2025-05-15T22:44:10.000Z","dependencies_parsed_at":"2025-04-26T12:38:46.489Z","dependency_job_id":"1850426a-999c-4607-b964-763cd83b5b56","html_url":"https://github.com/lpalbou/VoiceLLM","commit_stats":null,"previous_names":["lpalbou/voicellm"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/lpalbou/VoiceLLM","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lpalbou%2FVoiceLLM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lpalbou%2FVoiceLLM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lpalbou%2FVoiceLLM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lpalbou%2FVoiceLLM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lpalbou","download_url":"https://codeload.github.com/lpalbou/VoiceLLM/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lpalbou%2FVoiceLLM/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264648521,"owners_count":23643669,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","llm","python","speech-to-text","stt","text-to-speech","tts","voice","voice-assistant","voice-interface","voice-recognition","whisper"],"created_at":"2025-06-02T16:00:49.520Z","updated_at":"2025-07-10T20:12:44.183Z","avatar_url":"https://github.com/lpalbou.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# VoiceLLM\n\n[![PyPI version](https://img.shields.io/pypi/v/voicellm.svg)](https://pypi.org/project/voicellm/)\n[![Python Version](https://img.shields.io/pypi/pyversions/voicellm)](https://pypi.org/project/voicellm/)\n[![License](https://img.shields.io/pypi/l/voicellm)](https://github.com/lpalbou/voicellm/blob/main/LICENSE)\n[![GitHub stars](https://img.shields.io/github/stars/lpalbou/voicellm?style=social)](https://github.com/lpalbou/voicellm/stargazers)\n\nA modular Python library for voice interactions with AI systems, providing text-to-speech (TTS) and speech-to-text (STT) capabilities with interrupt handling.\n\nWhile we provide CLI and WEB examples, VoiceLLM is designed to be integrated in other projects.\n\n## Features\n\n- **Text-to-Speech**: High-quality speech synthesis with adjustable speed\n- **Speech-to-Text**: Accurate voice recognition using OpenAI's Whisper\n- **Voice Activity Detection**: Efficient speech detection using WebRTC VAD\n- **Interrupt Handling**: Stop TTS by speaking or using stop commands\n- **Modular Design**: Easily integrate with any text generation system\n\n## Installation\n\n### Prerequisites\n\n- Python 3.8+ (3.11 recommended)\n- PortAudio for audio input/output\n\n### Basic Installation\n\n```bash\n# Install from PyPI\npip install voicellm\n\n# Or clone the repository\ngit clone https://github.com/lpalbou/voicellm.git\ncd voicellm\npip install -e .\n```\n\n### Development Installation\n\n```bash\n# Install with development dependencies\npip install \"voicellm[dev]\"\n```\n\n### From Requirements File\n\n```bash\n# Install all dependencies including the package\npip install -r requirements.txt\n```\n\n## Quick Start\n\n```python\nfrom voicellm import VoiceManager\nimport time\n\n# Initialize voice manager\nvoice_manager = VoiceManager(debug_mode=False)\n\n# Text to speech\nvoice_manager.speak(\"Hello, I am an AI assistant. How can I help you today?\")\n\n# Wait for speech to complete\nwhile voice_manager.is_speaking():\n    time.sleep(0.1)\n\n# Speech to text with callback\ndef on_transcription(text):\n    print(f\"User said: {text}\")\n    if text.lower() != \"stop\":\n        # Process with your text generation system\n        response = f\"You said: {text}\"\n        voice_manager.speak(response)\n\n# Start voice recognition\nvoice_manager.listen(on_transcription)\n\n# Wait for user to say \"stop\" or press Ctrl+C\ntry:\n    while voice_manager.is_listening():\n        time.sleep(0.1)\nexcept KeyboardInterrupt:\n    pass\n\n# Clean up\nvoice_manager.cleanup()\n```\n\n## Running Examples\n\nThe package includes several examples that demonstrate different ways to use VoiceLLM.\n\n### Voice Mode (Default)\n\nIf installed globally, you can launch VoiceLLM directly in voice mode:\n\n```bash\n# Start VoiceLLM in voice mode (TTS ON, STT ON)\nvoicellm\n\n# With options\nvoicellm --debug --whisper base --model gemma3:latest --api http://localhost:11434/api/chat\n```\n\nCommand line options:\n- `--debug`: Enable debug mode with detailed logging\n- `--api`: URL of the Ollama API (default: http://localhost:11434/api/chat)\n- `--model`: Ollama model to use (default: granite3.3:2b)\n  - Other examples : cogito:3b, phi4-mini:latest, qwen2.5:latest, cogito:latest, gemma3:latest, etc.\n- `--whisper`: Whisper model to use (tiny, base, small, medium, large)\n- `--no-voice`: Start in text mode instead of voice mode\n- `--system`: Custom system prompt\n\n### Command-Line REPL\n\n```bash\n# Run the CLI example (TTS ON, STT OFF)\nvoicellm-cli cli\n\n# With debug mode\nvoicellm-cli cli --debug\n```\n\n### Web API\n\n```bash\n# Run the web API example\nvoicellm-cli web\n\n# With different host and port\nvoicellm-cli web --host 0.0.0.0 --port 8000\n```\n\nYou can also run a simplified version that doesn't load the full models:\n\n```bash\n# Run the web API with simulation mode\nvoicellm-cli web --simulate\n```\n\n#### Troubleshooting Web API\n\nIf you encounter issues with the web API:\n\n1. **404 Not Found**: Make sure you're accessing the correct endpoints (e.g., `/api/test`, `/api/tts`)\n2. **Connection Issues**: Ensure no other service is using the port\n3. **Model Loading Errors**: Try running with `--simulate` flag to test without loading models\n4. **Dependencies**: Ensure all required packages are installed:\n   ```bash\n   pip install flask soundfile numpy requests\n   ```\n5. **Test with a simple Flask script**:\n   ```python\n   from flask import Flask\n   app = Flask(__name__)\n   @app.route('/')\n   def home():\n       return \"Flask works!\"\n   app.run(host='127.0.0.1', port=5000)\n   ```\n\n### Simple Demo\n\n```bash\n# Run the simple example\nvoicellm-cli simple\n```\n\n## Component Overview\n\n### VoiceManager\n\nThe main class that coordinates TTS and STT functionality:\n\n```python\n# Initialize\nmanager = VoiceManager(tts_model=\"tts_models/en/ljspeech/tacotron2-DDC\", \n                      whisper_model=\"tiny\", debug_mode=False)\n\n# TTS\nmanager.speak(text, speed=1.0, callback=None)\nmanager.stop_speaking()\nmanager.is_speaking()\n\n# STT\nmanager.listen(on_transcription, on_stop=None)\nmanager.stop_listening()\nmanager.is_listening()\n\n# Configuration\nmanager.change_whisper_model(model_name)\nmanager.change_vad_aggressiveness(aggressiveness)\n\n# Cleanup\nmanager.cleanup()\n```\n\n### TTSEngine\n\nHandles text-to-speech synthesis:\n\n```python\nfrom voicellm.tts import TTSEngine\n\ntts = TTSEngine(model_name=\"tts_models/en/ljspeech/tacotron2-DDC\", debug_mode=False)\ntts.speak(text, speed=1.0, callback=None)\ntts.stop()\ntts.is_active()\n```\n\n### VoiceRecognizer\n\nManages speech recognition with VAD:\n\n```python\nfrom voicellm.recognition import VoiceRecognizer\n\ndef on_transcription(text):\n    print(f\"Transcribed: {text}\")\n\ndef on_stop():\n    print(\"Stop command detected\")\n\nrecognizer = VoiceRecognizer(transcription_callback=on_transcription,\n                           stop_callback=on_stop, \n                           whisper_model=\"tiny\",\n                           debug_mode=False)\nrecognizer.start(tts_interrupt_callback=None)\nrecognizer.stop()\nrecognizer.change_whisper_model(\"base\")\nrecognizer.change_vad_aggressiveness(2)\n```\n\n## Integration with Text Generation Systems\n\nVoiceLLM is designed to be used with any text generation system:\n\n```python\nfrom voicellm import VoiceManager\nimport requests\n\n# Initialize voice manager\nvoice_manager = VoiceManager()\n\n# Function to call text generation API\ndef generate_text(prompt):\n    response = requests.post(\"http://localhost:11434/api/chat\", json={\n        \"model\": \"granite3.3:2b\",\n        \"messages\": [{\"role\": \"user\", \"content\": prompt}],\n        \"stream\": False\n    })\n    return response.json()[\"message\"][\"content\"]\n\n# Callback for speech recognition\ndef on_transcription(text):\n    if text.lower() == \"stop\":\n        return\n        \n    print(f\"User: {text}\")\n    \n    # Generate response\n    response = generate_text(text)\n    print(f\"AI: {response}\")\n    \n    # Speak response\n    voice_manager.speak(response)\n\n# Start listening\nvoice_manager.listen(on_transcription)\n```\n\n## Perspectives\n\nThis is a test project that I designed with examples to work with Ollama, but I will adapt the examples and voicellm to work with any LLM provider (anthropic, openai, etc).\n\n## License and Acknowledgments\n\nVoiceLLM is licensed under the [MIT License](LICENSE).\n\nThis project depends on several open-source libraries and models, each with their own licenses. Please see [ACKNOWLEDGMENTS.md](ACKNOWLEDGMENTS.md) for a detailed list of dependencies and their respective licenses.\n\nSome dependencies, particularly certain TTS models, may have non-commercial use restrictions. If you plan to use VoiceLLM in a commercial application, please ensure you are using models that permit commercial use or obtain appropriate licenses. ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flpalbou%2Fvoicellm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flpalbou%2Fvoicellm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flpalbou%2Fvoicellm/lists"}