{"id":23536190,"url":"https://github.com/fatma-moanes/voice-assistant","last_synced_at":"2026-02-15T09:04:40.097Z","repository":{"id":269629598,"uuid":"908038611","full_name":"Fatma-Moanes/voice-assistant","owner":"Fatma-Moanes","description":"Voice Assistant for FM-Clinic: A multilingual AI-powered voice assistant for booking doctor appointments, leveraging advanced speech-to-text, text-to-speech, and large language models for seamless, natural user interactions.","archived":false,"fork":false,"pushed_at":"2025-01-21T11:57:18.000Z","size":215,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-17T06:41:51.222Z","etag":null,"topics":["ai-assistant","arabic","arabic-nlp","aws-polly","chatbot","gpt","groq","langchain","langsmith","llm","mongodb","multilingual","openai","speech-recognition","speech-to-text","streamlit","text-to-speech","transcription","voice-assistant","whisper"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Fatma-Moanes.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-25T00:21:24.000Z","updated_at":"2025-01-29T00:12:44.000Z","dependencies_parsed_at":null,"dependency_job_id":"923ff10b-dd15-4980-9cce-9ba304260f81","html_url":"https://github.com/Fatma-Moanes/voice-assistant","commit_stats":null,"previous_names":["fatma-moanes/voice-assistant"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Fatma-Moanes%2Fvoice-assistant","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Fatma-Moanes%2Fvoice-assistant/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Fatma-Moanes%2Fvoice-assistant/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Fatma-Moanes%2Fvoice-assistant/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Fatma-Moanes","download_url":"https://codeload.github.com/Fatma-Moanes/voice-assistant/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239248397,"owners_count":19607012,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-assistant","arabic","arabic-nlp","aws-polly","chatbot","gpt","groq","langchain","langsmith","llm","mongodb","multilingual","openai","speech-recognition","speech-to-text","streamlit","text-to-speech","transcription","voice-assistant","whisper"],"created_at":"2024-12-26T02:17:26.303Z","updated_at":"2025-11-01T02:30:25.082Z","avatar_url":"https://github.com/Fatma-Moanes.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Voice Assistant for a Clinic\n\n🎙️ A *bilingual, AI-powered voice assistant** designed to simplify doctor appointment bookings at a Clinic. This solution combines **speech-to-text (STT)**, **text-to-speech (TTS)**, and **large language models (LLMs)** to deliver intuitive and efficient interactions.\n\n![Screenshot from 2025-01-07 00-25-22](https://github.com/user-attachments/assets/e8f27261-bf9e-4fe0-8354-6e1a7f6b3799)\n\n---\n\n## 🌟 Key Features\n\n### 🗣️ Speech-to-Text (STT)\n- **Real-Time Transcription**:\n  - Powered by **Whisper Streaming** and **AWS Transcribe**.\n  - Supports **Deepgram** for prerecorded audio files.\n  - Multilingual capability: **English** and **Arabic**.\n  - Real-time **partial transcription** for better interactivity.\n  \n### 🔊 Text-to-Speech (TTS)\n- Natural voice responses using **AWS Polly**.\n- Supports high-quality voices for both **English** and **Arabic**.\n\n### 🤖 AI-Powered Conversations\n- **Context-Aware Dialogues**:\n    A ReAct LLM agent that can:\n  - Collect patient details like name, age, and insurance status.\n  - Suggest available clinic locations and doctor specialties.\n  - Schedule and book appointments with MongoDB integration.\n- Powered by **GROQ** or **OpenAI GPT**, offering flexible LLM backends.\n\n### 🎨 Intuitive User Interface\n- Built with **Streamlit** for a clean and responsive design.\n- Sidebar for language selection and session management.\n- Real-time chat display with user-friendly message bubbles.\n- **Debug Mode**:\n  - View intermediate tool outputs and chat history.\n  - Separate debug panel for tracing.\n\n### 📊 Data Integration\n- **MongoDB** as the database for managing:\n  - Patient records.\n  - Doctor information (schedules, specialties, locations).\n  - Appointment bookings.\n- Ready-to-use database seeding script with dummy data for testing.\n\n---\n\n## 🛠️ Tech Stack\n\n### Core Components\n- **Frontend**: [Streamlit](https://streamlit.io/)\n- **Backend**: Python\n- **Database**: MongoDB\n- **LLM**: [GROQ](https://www.groq.com/) or [OpenAI GPT](https://openai.com/)\n- **Speech Processing**:\n  - **Whisper** [WhisperStreaming](https://github.com/ufal/whisper_streaming)\n  - AWS Transcribe\n  - Deepgram (prerecorded audio)\n- **Text-to-Speech**: AWS Polly\n\n### Libraries \u0026 Tools\n- **LangChain**: Manages LLM conversation workflows and tools.\n- **LangSmith**: For tracing and debugging LLMs.\n- **Boto3**: AWS SDK for Polly and Transcribe integrations.\n- **Librosa** \u0026 **SoundDevice**: Audio handling and preprocessing.\n- **Pymongo**: MongoDB integration.\n\n---\n\n## 🚀 Getting Started\n\n### Prerequisites\n1. **Python 3.10+**\n2. **MongoDB** (Ensure it’s running locally or on the cloud).\n3. **API Keys**:\n   - **AWS Polly \u0026 Transcribe** (or use Whisper Streaming).\n   - **GROQ** or **OpenAI**\n   - **Deepgram** (optional for prerecorded audio).\n\n---\n\n### Installation\n\n1. **Clone the Repository**:\n   ```bash\n   git clone https://github.com/Fatma-Moanes/voice-assistant.git\n   cd voice-assistant\n   ```\n\n2. **Install Dependencies**:\n   ```bash\n   poetry install\n   ```\n\n3. **Set Up Environment Variables**:\n   - Copy `.env.example` to `.env`:\n     ```bash\n     cp .env.example .env\n     ```\n   - Fill in your API keys and database credentials in the `.env` file.\n\n4. **Seed the Database**:\n   Populate the database with dummy data:\n   ```bash\n   python utils/create_db.py\n   ```\n\n5. **Run the Application**:\n   Start the Streamlit app for live audio:\n   ```bash\n   poetry run streamlit run app/streamlit_app_streaming.py\n   ```\n\n---\n\n## 🎨 UI Overview\n\n### Chat Interface\n- **Dynamic Conversation**: User and assistant messages are displayed in visually distinct bubbles.\n- **Audio Input**:\n  - Live microphone recording.\n- **Error Handling**:\n  - Clear notifications for failed transcription or processing.\n\n### Sidebar Settings\n- **Language Selection**:\n  - Choose between **English**, **Arabic**, or **Auto Detect**.\n- **Session Controls**:\n  - Clear chat history.\n  - Toggle Debug Mode.\n\n### Debug Panel\n- Real-time insight into:\n  - Chat history used by the LLM.\n  - Intermediate tool calls and responses.\n\n---\n\n## 🧰 Configuration\n\nAll application settings are managed through:\n1. **`config.yml`**:\n   - **Speech-to-Text** model selection (`WhisperStreaming`, `AWSStreaming`, or `Deepgram`).\n   - Language preferences and model-specific configurations.\n   - **Text-to-Speech** (AWS Polly voices and region).\n   - **LLM** provider (GROQ or OpenAI).\n\n2. **`.env`**:\n   - Store sensitive credentials such as API keys and database connection strings.  Use `.env.example` as a template.\n\n---\n\n## 🌐 Environment Variables\n\nBelow is a summary of required `.env` variables:\n\n```dotenv\n# MongoDB\nMONGODB_CONNECTION_STRING=\"your_connection_string\"\nDB_NAME=\"DoctorAppointmentDB\"\n\n# GROQ API\nGROQ_API_KEY=\"your_groq_api_key\"\n\n# OpenAI API\nOPENAI_API_KEY=\"your_openai_api_key\"\n\n# AWS Polly \u0026 Transcribe\nAWS_ACCESS_KEY_ID=\"your_aws_access_key\"\nAWS_SECRET_ACCESS_KEY=\"your_aws_secret_key\"\n\n# Deepgram API (optional)\nDEEPGRAM_API_KEY=\"your_deepgram_api_key\"\n```\n\n---\n\n## ✅ Completed Milestones\n\n- **Speech-to-Text Enhancements**:\n  - Real-time and partial transcription.\n  - Improved error handling for silent inputs.\n- **Streamlined Conversations**:\n  - Context-aware doctor booking logic.\n  - Integration with MongoDB for data persistence.\n- **Debug Mode**:\n  - Displays:\n    - Intermediate steps (tool calls).\n    - Processed chat history used by the AI agent.\n- **Extensible Configuration**:\n  - Support for multiple STT and LLM models.\n\n\n---\n\n## ✨ Acknowledgments\n\n- **LangChain** for managing LLM integrations.\n- **Amazon Polly** for high-quality voice synthesis.\n- **Whisper Streaming** and **Deepgram** for advanced transcription.\n- **Streamlit** for the responsive and interactive UI.\n- **GROQ** and **OpenAI GPT** for powering the conversational AI.\n\n---\n\n## 💬 Support\n\nIf you have any questions or encounter issues, feel free to open an [issue](https://github.com/Fatma-Moanes/voice-assistant/issues) or reach out via email at fmoanesnoureldin@gmail.com\n\n---\nEnjoy using the **Voice Assistant for FM-Clinic**! 🚀\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffatma-moanes%2Fvoice-assistant","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffatma-moanes%2Fvoice-assistant","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffatma-moanes%2Fvoice-assistant/lists"}