https://github.com/ten-framework/ten-agent
TEN Agent is a conversational voice AI agent powered by TEN, integrating Deepseek, Gemini, OpenAI, RTC, and hardware like ESP32. It enables realtime AI capabilities like seeing, hearing, and speaking, and is fully compatible with platforms like Dify and Coze.
https://github.com/ten-framework/ten-agent
agent ai asr cpp gemini golang gpt-4 gpt-4o llm low-latency multimodal nextjs14 openai python rag real-time realtime tts vision voice-assistant
Last synced: 16 days ago
JSON representation
TEN Agent is a conversational voice AI agent powered by TEN, integrating Deepseek, Gemini, OpenAI, RTC, and hardware like ESP32. It enables realtime AI capabilities like seeing, hearing, and speaking, and is fully compatible with platforms like Dify and Coze.
- Host: GitHub
- URL: https://github.com/ten-framework/ten-agent
- Owner: TEN-framework
- License: apache-2.0
- Created: 2024-06-19T14:26:15.000Z (10 months ago)
- Default Branch: main
- Last Pushed: 2025-03-30T02:10:55.000Z (17 days ago)
- Last Synced: 2025-03-30T03:19:25.871Z (17 days ago)
- Topics: agent, ai, asr, cpp, gemini, golang, gpt-4, gpt-4o, llm, low-latency, multimodal, nextjs14, openai, python, rag, real-time, realtime, tts, vision, voice-assistant
- Language: Python
- Homepage: https://agent.theten.ai/
- Size: 30.8 MB
- Stars: 5,427
- Watchers: 55
- Forks: 612
- Open Issues: 75
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Code of conduct: docs/code-of-conduct/code-of-conduct.md
Awesome Lists containing this project
- awesome-ChatGPT-repositories - TEN-Agent - TEN Agent is the world’s first real-time multimodal agent integrated with the OpenAI Realtime API, RTC, and features weather checks, web search, vision, and RAG capabilities. (NLP)
README
[](https://twitter.com/intent/follow?screen_name=TenFramework)
[](https://github.com/TEN-framework/ten-agent/discussions/)
[](https://github.com/TEN-framework/ten-agent/graphs/commit-activity)
[](https://github.com/TEN-framework/ten-agent/issues)
[](https://github.com/TEN-framework/ten-agent/pulls)
[](https://github.com/TEN-framework/ten-agent/blob/main/LICENSE)
[](https://discord.gg/VnPftUzAMJ)
[](https://GitHub.com/TEN-framework/ten-agent/watchers/?WT.mc_id=academic-105485-koreyst)
[](https://GitHub.com/TEN-framework/ten-agent/network/?WT.mc_id=academic-105485-koreyst)
[](https://GitHub.com/TEN-framework/ten-agent/stargazers/?WT.mc_id=academic-105485-koreyst)
[Documentation](https://doc.theten.ai/ten-agent/overview)
•
[Getting Started](https://doc.theten.ai/ten-agent/getting_started)
•
[TEN Framework Repository](https://github.com/TEN-framework/ten_framework)
✨ TEN Agent + Trulience
TEN Agent + Trulience
Build engaging AI avatars with TEN Agent using [Trulience](https://trulience.com)'s diverse collection of free avatar options. To get it up and running, you only need 2 steps:
1. Follow the README to finish setting up and running the Playground in localhost:3000
2. Enter the avatar ID and [token](https://trulience.com/docs#/authentication/jwt-tokens/jwt-tokens?id=use-your-custom-userid) you get from [Trulience](https://trulience.com)
TEN Agent + Deepseek
[TEN Agent + Deepseek](https://ten-framework.medium.com/deepgram-deepseek-fish-audio-build-your-own-voice-assistant-with-ten-agent-d3ee65faabe8)
TEN is a very versatile framework. That said, TEN Agent is compatible with DeepSeek R1, try experiencing realtime conversations with DeepSeek R1!
TEN Agent + ESP32
[TEN Agent ESP32 Client](https://github.com/TEN-framework/TEN-Agent/tree/main/esp32-client)
TEN Agent is now running on the Espressif ESP32-S3 Korvo V3 development board, an excellent way to integrate realtime communication with LLM on hardware.
TEN Agent + Dify with RAG + Coze
TEN Agent + Dify Agent with RAG
[TEN Agent + Dify](https://doc.theten.ai/ten-agent/quickstart-1/use-cases/run_va/run_dify)
[TEN Agent + Coze](https://doc.theten.ai/ten-agent/quickstart-1/use-cases/run_va/run_coze)
TEN offers a great support to make the realtime interactive experience even better on other LLM platform as well, check out docs for more.
TEN Agent + Gemini Multimodal Live API
Gemini 2.0 Multimodal Live API
Try **Google Gemini Multimodal Live API** with **realtime vision** and **realtime screenshare detection** capabilities, it is a ready-to-use extension, along with powerful tools like **Weather Check** and **Web Search** integrated perfectly into TEN Agent.
TEN Agent + Storyteller + Image Generator
Storyteller + Image Generator
Describe a topic and ask TEN Agent to tell you a story while also generating images of the story to provide a more immersive experience for kids.
TEN Agent Usecases
Ready-to-use Extensions
TEN Agent Playground in Local Environment
### Prerequisites
| Category | Requirements |
|----------|-------------|
| **Keys** | • Agora [App ID](https://docs.agora.io/en/video-calling/get-started/manage-agora-account?platform=web#create-an-agora-project) and [App Certificate](https://docs.agora.io/en/video-calling/get-started/manage-agora-account?platform=web#create-an-agora-project) (free minutes every month)
• [OpenAI](https://openai.com/index/openai-api/) API key (any LLM that is compatible with OpenAI)
• [Deepgram](https://deepgram.com/) ASR (free credits available with signup)
• [Elevenlabs](https://elevenlabs.io/) TTS (free credits available with signup)|
| **Installation** | • [Docker](https://www.docker.com/) / [Docker Compose](https://docs.docker.com/compose/)
• [Node.js(LTS) v18](https://nodejs.org/en) |
| **Minimum System Requirements** | • CPU >= 2 Core
• RAM >= 4 GB |
### macOS: Docker setting on Apple Silicon
For Apple Silicon Macs, uncheck "Use Rosetta for x86/amd64 emulation" in Docker settings. Note: This may result in slower build times on ARM, but performance will be normal when deployed to x64 servers.
### Next step
#### 1. Create `.env` file
```bash
cp ./.env.example ./.env
```
#### 2. Setup Agora App ID and App Certificate in `.env`
```bash
AGORA_APP_ID=
AGORA_APP_CERTIFICATE=
```
#### 3. Start agent development containers
```bash
docker compose up -d
```
#### 4. Enter container
```bash
docker exec -it ten_agent_dev bash
```
#### 5. Build agent
```bash
task use
```
#### 6. Start the web server
```bash
task run
```
#### 7. Edit playground settings
Open the playground at [localhost:3000](http://localhost:3000) to configure your agent.
1. Select a graph type (e.g. Voice Agent, Realtime Agent)
2. Choose a corresponding module
3. Select an extension and configure its API key settings
Now, we have successfully set up the playground. This is just the beginning of TEN Agent. There are many different ways to explore and utilize TEN Agent. To learn more, please refer to the [documentation](https://doc.theten.ai/ten-agent/overview).
Deployment
Once you have customized your agent (either by using the playground or editing `property.json` directly), you can deploy it by creating a release Docker image for your service.
Read the [Deployment Guide](https://doc.theten.ai/ten-agent/deployment/deploy_agent_service) for detailed information about deployment.
TEN Agent Architecture
Stay Tuned
Before we get started, be sure to star our repository and get instant notifications for all new releases!
Join Community
- [Discord](https://discord.gg/VnPftUzAMJ): Ideal for sharing your applications and engaging with the community.
- [GitHub Discussion](https://github.com/TEN-framework/ten-agent/discussions): Perfect for providing feedback and asking questions.
- [GitHub Issues](https://github.com/TEN-framework/ten-agent/issues): Best for reporting bugs and proposing new features. Refer to our [contribution guidelines](./docs/code-of-conduct/contributing.md) for more details.
- [X](https://img.shields.io/twitter/follow/TenFramework?logo=X&color=%20%23f5f5f5): Great for sharing your agents and interacting with the community.
Star History
[](https://star-history.com/#ten-framework/ten-agent&Date)
Code Contributors
[](https://github.com/TEN-framework/ten-agent/graphs/contributors)
Contribution Guidelines
Contributions are welcome! Please read the [contribution guidelines](./docs/code-of-conduct/contributing.md) first.
License
This project is licensed under the Apache 2.0 License - see the [LICENSE](LICENSE) file for details.