https://github.com/second-state/echokit_server
Open Source Voice Agent Platform
https://github.com/second-state/echokit_server
agentic-ai ai asr esp32 llm mcp mcp-client tts voice voice-assistant
Last synced: 4 months ago
JSON representation
Open Source Voice Agent Platform
- Host: GitHub
- URL: https://github.com/second-state/echokit_server
- Owner: second-state
- License: gpl-3.0
- Created: 2025-02-26T12:40:07.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-09-21T17:40:44.000Z (8 months ago)
- Last Synced: 2025-10-05T15:07:24.115Z (8 months ago)
- Topics: agentic-ai, ai, asr, esp32, llm, mcp, mcp-client, tts, voice, voice-assistant
- Language: Rust
- Homepage: https://echokit.dev/
- Size: 354 KB
- Stars: 357
- Watchers: 3
- Forks: 64
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# EchoKit Server
EchoKit Server is the central component that manages communication between the [EchoKit device](https://echokit.dev/) and AI services. It can be deployed locally or connected to preset servers, allowing developers to customize LLM endpoints, plan the LLM prompt, configure speech models, and integrate additional AI features like MCP servers.
You will need an [EchoKit device](https://echokit.dev/), or create your own ESP32 device with the [EchoKit firmware](https://github.com/second-state/echokit_box).
## Features
EchoKit Server powers the full voice–AI interaction loop, making it easy for developers to run end-to-end speech pipelines with flexible model choices and custom integrations.
### ASR → LLM → TTS Pipeline
Seamlessly connect **ASR → LLM → TTS** for real-time, natural conversations.
Each stage can be configured independently with your preferred models or APIs.
#### Model Compatibility
* **ASR (Speech Recognition):** Works with any API that’s *OpenAI-compatible*.
* **LLM (Language Model):** Connect to any *OpenAI-spec* endpoint — local or cloud.
* **TTS (Text-to-Speech):** Use any *OpenAI-spec* voice model for flexible deployment.
* ElevenLabs (Streaming Mode)
### End-to-End Model Pipelines
Out-of-the-box support for:
* **Gemini** — Google’s multimodal model
* **Qwen Real-Time** — Alibaba’s powerful open LLM
### Developer Customization
* Deploy **locally** or connect to **remote inference servers**
* Define your own **LLM prompts** and **response workflows**
* Configure **speech and voice models** for different personas or use cases
* Integrate **MCP servers** for extended functionality
## Set up the EchoKit server
### Build
```
git clone https://github.com/second-state/echokit_server
```
Edit `config.toml` to customize the VAD, ASR, LLM, TTS services, as well as prompts and MCP servers. You can [see many examples](examples/).
```
cargo build --release
```
### Configure AI services
The `config.toml` can use any combination of open-source or proprietary AI services, as long as they offer OpenAI-compatible API endpoints. Here are instructions to start open source AI servers for the EchoKit server.
* ASR: https://llamaedge.com/docs/ai-models/speech-to-text/quick-start-whisper
* LLM: https://llamaedge.com/docs/ai-models/llm/quick-start-llm
* Streaming TTS: https://github.com/second-state/gsv_tts
Alternatively, you could use Google Gemini Live services for VAD + ASR + LLM, and even optionally, TTS. See [config.toml examples](examples/gemini).
You can also [configure MCP servers](examples/mcp/config.toml) to give the EchoKit server tool use capabilities.
### Configure the voice prompt
The `hello.wav` file on the server is sent to the EchoKit device when it connects. It is the voice prompt the device will say to tell the user that it is ready.
### Run the EchoKit server
```
export RUST_LOG=debug
nohup target/release/echokit_server &
```
### Test on a web page
Go here: https://echokit.dev/chat/
Click on the link to save the `index.html` file to your local hard disk.
Double click the local `index.html` file and open it in your browser.
In the web page, set the URL to your own EchoKit server address, and start chatting!
### Configure a new device
Go to web page: https://echokit.dev/setup/ and use Bluetooth to connect to the `GAIA ESP332` device.
Configure WiFi and server
* WiFi SSID (e.g., `MyHome`)
* WiFi password (e.g., `MyPassword`)
* Web Socket server URL for `echokit_server`
* US: `ws://indie.echokit.dev/ws/`
* Taiwan: `ws://tw.echokit.dev/ws/`
* Rest of the world: `ws://edge.echokit.dev/ws/`
### Use the device
**Chat:** press the `K0` button once or multiple times until the status bar shows "Ready". You can now speak and it will show "Listening ...". The device answers after it decides that you have done speaking.
**Config:** press `RST`. While it is restarting, press and hold `K0` to enter the configuration mode. Then [open the configuration UI](https://echokit.dev/setup/) to connect to the device via BT.