https://github.com/natserract/talk2-me
Talk2me Voice Assistant
https://github.com/natserract/talk2-me
Last synced: 7 days ago
JSON representation
Talk2me Voice Assistant
- Host: GitHub
- URL: https://github.com/natserract/talk2-me
- Owner: natserract
- Created: 2024-06-25T02:06:50.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-10-14T06:34:07.000Z (about 1 year ago)
- Last Synced: 2025-04-04T07:34:15.923Z (7 months ago)
- Language: Python
- Size: 70.3 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Talk2-Me
Talk to me?
## **Mechanism Overview:**
**Pre-requisites:**
- Use a webcam to capture video stream.
- Provide WebRTC (e.g., Livekit) to stream the video in real-time.
**Input Processing:**
- Capture the image from the video stream.
- Use speech-to-text technology (Deepgram) to convert the user's voice input into text.
- Pass the captured image and text input as context to the LLM (OpenAI GPT-4).
**Response Generation:**
- LLM (OpenAI GPT-4) processes the input context and generates a text response.
- Convert the text response back to speech using a text-to-speech
## Running
```sh
rye pin 3.11.9
# Install packages
rye sync
# Download livekit plugins
rye run talk2-me download-files
# Running
rye run talk2-me start
```
## Playground
Go to https://agents-playground.livekit.io/
## WIP
- [x] Voice talks
- [ ] Video track & capturing
**Related Issue:**
```sh
LOG: [W NNPACK.cpp:64] Could not initialize NNPACK! Reason: Unsupported hardware.
```
> Still looking how to solve this issue. Currently only able to voice talks.