Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/kristofferv98/semanthavoiceassistant

A comprehensive AI companion leveraging advanced semantic analysis, sentiment detection, and voice processing to provide personalized and context-aware interactions using Autogen, semantic-router, and VoiceProcessingToolkit.
https://github.com/kristofferv98/semanthavoiceassistant

ai-companion autogen elevenlabs intent-detection local-llm natural-language-processing openai personalized-interactions picovoice python rag semantic-routing sentiment-analysis text-to-speech voice-activity-detection voice-assistant voice-processing voice-recognition websearch whisper

Last synced: 4 days ago
JSON representation

A comprehensive AI companion leveraging advanced semantic analysis, sentiment detection, and voice processing to provide personalized and context-aware interactions using Autogen, semantic-router, and VoiceProcessingToolkit.

Awesome Lists containing this project

README 

        
# Semantha Voice Assistant

Welcome to the Semantha Voice Assistant, a AI companion designed to provide personalized and context-aware 

interactions through the integration of autogen, semantic routing and, VoiceProcessingToolkit. This system is adept 

at understanding user intent and sentiment. Responses are tailored with semantic routing based RAG, websearch 

capabilities using autogen websurfer and perplexity, and changing functionability during usage. It also has support 

for Local LLm usage. It works seamlessly to create effortless communication with the assistant.



## Setup and Installation

Before installing the requirements, navigate to the SemanthaVoiceAssistant project directory:



Then, get started with Semantha Voice Assistant by following these steps:

1. Clone the project: `git clone https://github.com/kristofferv98/SemanthaVoiceAssistant.git`

2. Navigate to the project dir `cd SemanthaVoiceAssistant`

3. Install dependencies: `pip install -r requirements.txt`.

4. Set enviorment variables in `.env`

5. Start the assistant: `python main.py`.

6. Type for example "Explain the Samantha program to me. and tell me how to customize it for a demo and instruction of usage"

7. Type "change input", "toggle voice feedback" or similar to enable or disable voice feedback. 

8. Say "hey Computer" to trigger the voice based interaction when input is set to recording. (Visit https://github.com/kristofferv98/VoiceProcessingToolkit for details)



Recommended:

PICOVOICE_APIKEY is for voice interperation is available free at picovoices website

ELEVENLABS_API_KEY for natural language responses



(optional):

BING_API_KEY for webscraping using autogen websurfer (visit autogens notebook on github for further information)

PERPLEXITY_API_KEY for more advanced webscraping.



## Core Functionality

The Semantha Voice Assistant features a semantic routing core that accurately interprets the essence of user 

inquiries. This core functionality allows for efficient management of a wide range of requests, including toggling 

between input modes and feedback types, and switching between assistant profiles. An intelligent layer precedes the 

evaluation by large language models (LLMs), ensuring precise interpretation and routing of queries for optimal user 

interaction.



## Behind the Scenes

Semantha's initial processing layer is the backbone of the system, designed for seamless operation. It is here that 

user intent is interpreted and queries are routed. This layer adapts to various interaction modes and user 

preferences, allowing for adjustments through natural language commands and facilitating profile customization. 

Users can switch between different LLM endpoints, enhancing the quality of interactions.



## Customization Options

The system offers extensive customization options through editable configurations in the `Config/OAI_CONFIG_LIST.json` 

file. Users can specify their preferred GPT models and set up custom local LLM endpoints, tailoring the assistant 

to their specific needs and preferences.



## Advanced Features

For users seeking deeper insights, Semantha includes advanced features such as detailed automated agentic 

webscraping and the ability to read PDFs from web links. The `Router_logic/Research_router.py` module adds a layer 

of sophistication, enhancing the system's capability to provide nuanced replies and understand complex queries. 

The copilot feature, combined with perplexity analysis, enriches web searches, offering comprehensive responses.



## Environment Setup

To utilize voice responses or the advanced research copilot function, configure the necessary API keys in the `.env` 

file at the project's root. The following keys are mandatory:

- `OPENAI_API_KEY`: For embeddings and assistant functionalities.



## Conclusion

Semantha Voice Assistant exemplifies the seamless integration of semantic embeddings and voice processing 

technologies. It is designed to meet a wide range of user needs with precision, offering empathetic engagement, 

humor, or expert advice as needed. This system ensures a rich and personalized user experience through meaningful 

and context-aware responses.



## ACKNOWLEDGEMENTS:

Special thanks to James Briggs and his team for developing the semantic-routing library and showcasing its use case on the YouTube channel https://www.youtube.com/@jamesbriggs. 

Their contributions have been invaluable to the development of this project. The link to the library can be found here https://github.com/aurelio-labs/semantic-router.



For more information on voice processing and additional functionalities, visit my library at https://github.com/kristofferv98/VoiceProcessingToolkit.