https://github.com/livekit/client-sdk-esp32
LiveKit Client SDK for ESP32 series chips. Easily enable real-time audio, video, and data for embedded projects.
https://github.com/livekit/client-sdk-esp32
agents c embedded esp32 real-time webrtc
Last synced: 4 months ago
JSON representation
LiveKit Client SDK for ESP32 series chips. Easily enable real-time audio, video, and data for embedded projects.
- Host: GitHub
- URL: https://github.com/livekit/client-sdk-esp32
- Owner: livekit
- License: apache-2.0
- Created: 2025-05-14T02:50:28.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2026-02-10T22:14:23.000Z (4 months ago)
- Last Synced: 2026-02-11T01:33:14.339Z (4 months ago)
- Topics: agents, c, embedded, esp32, real-time, webrtc
- Language: C
- Homepage: https://docs.livekit.io/
- Size: 5.19 MB
- Stars: 94
- Watchers: 12
- Forks: 24
- Open Issues: 13
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
- Agents: AGENTS.md
Awesome Lists containing this project
- awesome-embedded-software - LiveKit - ESP32 SDK for LiveKit: real-time audio, video, and data streaming. (Protocols / Network protocols)
README
# ESP32 SDK for LiveKit
[](https://livekit.github.io/client-sdk-esp32/)
[](https://components.espressif.com/components/livekit/livekit)
[](https://github.com/livekit/client-sdk-esp32/actions/workflows/ci.yml)
Use this SDK to add realtime video, audio and data features to your ESP32 projects. By connecting to [LiveKit](https://livekit.io/) Cloud or a self-hosted server, you can quickly build applications such as multi-modal AI, live streaming, or video calls with minimal setup.
> [!IMPORTANT]
> This SDK is currently in Developer Preview mode and not ready for production use.
> There may be bugs and APIs are subject to change during this period.
## Features
- **Supported chips**: ESP32-S3 and ESP32-P4
- **Bidirectional audio**: Opus encoding, acoustic echo cancellation (AEC)
- **Video publishing**: H.264 encoding, subscribing coming soon
- **AI Agents**: interact with agents in the cloud built with [LiveKit Agents](https://docs.livekit.io/agents/)
- **Real-time data**: data packets, remote method calls (RPC)
## Getting Started
Before you begin, please ensure you have ESP IDF v5.4 or higher installed on your system (see installation guide for [macOS/Linux](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/get-started/linux-macos-setup.html) or [Windows](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/get-started/windows-setup.html)).
### Examples
One of the best ways to get started with LiveKit is by reviewing the [examples](./components/livekit/examples/) and choosing one as a starting point for your project:
- [**Voice AI Agent**](./components/livekit/examples/voice_agent/README.md): Conversational AI voice agent that interacts with hardware based on user requests.
- [**Minimal**](./components/livekit/examples/minimal/README.md): Basic example of connecting to a LiveKit room with bidirectional audio.
- [**Minimal Video**](./components/livekit/examples/minimal_video/README.md): Equivalent to the minimal example with video publishing.
Once you have chosen an example to be your starting point, create a fresh project from it locally using the following command:
```sh
idf.py create-project-from-example "livekit/livekit=0.3.5:"
```
Substitute *\* for the example's directory name.
### Installation
If you would like to add LiveKit to your existing application, add it as a dependency using IDF:
```sh
idf.py add-dependency "livekit/livekit=0.3.5"
```
> [!IMPORTANT]
> Be sure to pin to a specific version as shown in the command above, as subsequent v0.x.x releases may have breaking changes.
## API Overview
With LiveKit added as a dependency to your application, include the LiveKit header and invoke
`livekit_system_init` early in your application's main function:
```c
#include "livekit.h"
void app_main(void)
{
livekit_system_init();
// Your application code...
}
```
### Configure media pipeline
LiveKit for ESP32 puts your application in control of the media pipeline; your application configures a capturer and/or renderer and provides their handles when creating a room.
#### Capturer: input from camera/microphone
- Required for rooms which will publish media tracks
- Created using the Espressif [*esp_capture*](https://components.espressif.com/components/espressif/esp_capture/) component
- Capture audio capture over I2S, video from MIPI CSI or DVI cameras
- After configuration, you will provide the `esp_capture_handle_t` when creating a room
#### Renderer: output to display/speaker
- Required for rooms which will subscribe to media tracks
- Created using the Espressif [*av_render*](https://components.espressif.com/components/tempotian/av_render/) component
- Playback audio over I2S, video on LCD displays supported by *esp_lcd*
- After configuration, you will provide the `av_render_handle_t` when creating a room
### Create room
Create a room object, specifying your capturer, renderer, and handlers for room events:
```c
static livekit_room_handle_t room_handle = NULL;
livekit_room_options_t room_options = {
.publish = {
.kind = LIVEKIT_MEDIA_TYPE_AUDIO,
.audio_encode = {
.codec = LIVEKIT_AUDIO_CODEC_OPUS,
.sample_rate = 16000,
.channel_count = 1
},
.capturer = my_capturer
},
.subscribe = {
.kind = LIVEKIT_MEDIA_TYPE_AUDIO,
.renderer = my_renderer
},
.on_state_changed = on_state_changed,
.on_participant_info = on_participant_info
};
if (livekit_room_create(&room_handle, &room_options) != LIVEKIT_ERR_NONE) {
ESP_LOGE(TAG, "Failed to create room object");
}
```
This example does not show all available fields in room options—please refer to the [API reference](https://livekit.github.io/client-sdk-esp32/group__Lifecycle.html#structlivekit__room__options__t)
for an extensive list.
Typically, you will want to create the room object early in your application's lifecycle, and connect/disconnect as necessary based on user interaction.
### Connect room
With a room room handle, connect by providing a server URL and token:
```c
livekit_room_connect(room_handle, "", "");
```
The connect method is asynchronous; use your `on_state_changed` handler provided in room options
to get notified when the connection is established or fails (e.g. due to an expired token, etc.).
Once connected, media exchange will begin:
1. If a capturer was provided, video and/or audio tracks will be published.
2. If a renderer was provided, the first video and/or audio tracks in the room will be subscribed to.
### Real-time data
In addition to real-time audio and video, LiveKit offers several methods for exchange real-time data between participants in a room.
#### Remote method call (RPC)
Define an RPC handler:
```c
static void get_cpu_temp(const livekit_rpc_invocation_t* invocation, void* ctx)
{
float temp = board_get_temp();
char temp_string[16];
snprintf(temp_string, sizeof(temp_string), "%.2f", temp);
livekit_rpc_return_ok(temp_string);
}
```
Register the handler on the room to allow it to be invoked by remote participants:
```c
livekit_room_rpc_register(room_handle, "get_cpu_temp", get_cpu_temp);
```
> [!TIP]
> In the [*voice_agent*](./components/livekit/examples/voice_agent/) example, RPC is used to allow an AI agent to interact
> with hardware by defining a series of methods for the agent to invoke.
#### User packets
Publish a user packet containing a raw data payload under a specific topic:
```c
const char* command = "G5 I0 J3 P0 Q-3 X2 Y3";
livekit_payload_t payload = {
.bytes = (uint8_t*)command,
.size = strlen(command)
};
livekit_data_publish_options_t options = {
.payload = &payload,
.topic = "gcode",
.lossy = false,
.destination_identities = (char*[]){ "printer-1" },
.destination_identities_count = 1
};
livekit_room_publish_data(room_handle, &options);
```
## Documentation
Please refer to the [LiveKit Docs](https://docs.livekit.io/home/) for an introduction to the platform and its features. For more detail about the APIs available in this SDK, please refer to the [API Reference](https://livekit.github.io/client-sdk-esp32/).
## Known Issues
- In some cases, a remote participant leaving the room can lead to a disconnect.
## Getting Help & Contributing
We invite you to join our [developer community](https://community.livekit.io/c/robotics) to ask questions, suggest improvements, or discuss how you can best contribute to this SDK.
LiveKit Ecosystem
LiveKit SDKsBrowser · iOS/macOS/visionOS · Android · Flutter · React Native · Rust · Node.js · Python · Unity · Unity (WebGL) · ESP32
Server APIsNode.js · Golang · Ruby · Java/Kotlin · Python · Rust · PHP (community) · .NET (community)
UI ComponentsReact · Android Compose · SwiftUI · Flutter
Agents FrameworksPython · Node.js · Playground
ServicesLiveKit server · Egress · Ingress · SIP
ResourcesDocs · Example apps · Cloud · Self-hosting · CLI