Use this SDK to add realtime video, audio and data features to your ESP32 projects. By connecting to LiveKit Cloud or a self-hosted server, you can quickly build applications such as multi-modal AI, live streaming, or video calls with minimal setup.

• Supported chips: ESP32-S3 and ESP32-P4
• Bidirectional audio: Opus encoding, acoustic echo cancellation (AEC)
• Video publishing: H.264 encoding, subscribing coming soon
• AI Agents: interact with agents in the cloud built with LiveKit Agents
• Real-time data: data streams, data packets, remote method calls (RPC)

Before you begin, please ensure you have ESP IDF v5.4 or higher installed on your system (see installation guide for macOS/Linux or Windows).

One of the best ways to get started with LiveKit is by reviewing the examples and choosing one as a starting point for your project:

• Voice AI Agent: Conversational AI voice agent that interacts with hardware based on user requests.
• Minimal: Basic example of connecting to a LiveKit room with bidirectional audio.
• Minimal Video: Equivalent to the minimal example with video publishing.
• Custom Hardware: Bidirectional audio on custom ESP32-S3 hardware with manual codec initialization (ES8311 + ES7210).

Once you have chosen an example to be your starting point, create a fresh project from it locally using the following command:

idf.py create-project-from-example "livekit/livekit=0.3.10:<example>"

Substitute <example> for the example's directory name.

If you would like to add LiveKit to your existing application, add it as a dependency using IDF:

idf.py add-dependency "livekit/livekit=0.3.10"

With LiveKit added as a dependency to your application, include the LiveKit header and invoke livekit_system_init early in your application's main function:

#include "livekit.h"

void app_main(void)
{
    livekit_system_init();
    // Your application code...
}

LiveKit for ESP32 puts your application in control of the media pipeline; your application configures a capturer and/or renderer and provides their handles when creating a room.

• Required for rooms which will publish media tracks
• Created using the Espressif esp_capture component
• Capture audio capture over I2S, video from MIPI CSI or DVI cameras
• After configuration, you will provide the esp_capture_handle_t when creating a room

• Required for rooms which will subscribe to media tracks
• Created using the Espressif av_render component
• Playback audio over I2S, video on LCD displays supported by esp_lcd
• After configuration, you will provide the av_render_handle_t when creating a room

Create a room object, specifying your capturer, renderer, and handlers for room events:

static livekit_room_handle_t room_handle = NULL;

livekit_room_options_t room_options = {
    .publish = {
        .kind = LIVEKIT_MEDIA_TYPE_AUDIO,
        .audio_encode = {
            .codec = LIVEKIT_AUDIO_CODEC_OPUS,
            .sample_rate = 16000,
            .channel_count = 1
        },
        .capturer = my_capturer
    },
    .subscribe = {
        .kind = LIVEKIT_MEDIA_TYPE_AUDIO,
        .renderer = my_renderer
    },
    .on_state_changed = on_state_changed,
    .on_participant_info = on_participant_info
};
if (livekit_room_create(&room_handle, &room_options) != LIVEKIT_ERR_NONE) {
    ESP_LOGE(TAG, "Failed to create room object");
}

This example does not show all available fields in room options—please refer to the API reference for an extensive list.

Typically, you will want to create the room object early in your application's lifecycle, and connect/disconnect as necessary based on user interaction.

With a room room handle, connect by providing a server URL and token:

livekit_room_connect(room_handle, "<your server URL>", "<token>");

The connect method is asynchronous; use your on_state_changed handler provided in room options to get notified when the connection is established or fails (e.g. due to an expired token, etc.).

Once connected, media exchange will begin:

1. If a capturer was provided, video and/or audio tracks will be published.
2. If a renderer was provided, the first video and/or audio tracks in the room will be subscribed to.

In addition to real-time audio and video, LiveKit offers several methods for exchange real-time data between participants in a room.

Define an RPC handler:

static void get_cpu_temp(const livekit_rpc_invocation_t* invocation, void* ctx)
{
    float temp = board_get_temp();
    char temp_string[16];
    snprintf(temp_string, sizeof(temp_string), "%.2f", temp);
    livekit_rpc_return_ok(temp_string);
}

Register the handler on the room to allow it to be invoked by remote participants:

livekit_room_rpc_register(room_handle, "get_cpu_temp", get_cpu_temp);

Data streams deliver ordered sequences of chunks (Header → Chunks → Trailer) over the reliable data channel. There are two kinds:

• Text streams: Each chunk is independently valid UTF-8 and can be used directly as a string. When sending, data is automatically chunked at UTF-8 character boundaries so multi-byte characters are never split. Used for chat messages, transcriptions, etc.
• Byte streams: Chunks are raw binary fragments of a larger payload, split without delimiters. Chunks must be aggregated by the application to reconstruct the original data. Used for file transfers, binary payloads, etc.

Receiving — register a handler for a topic:

static void on_text_chunk(const livekit_data_stream_chunk_t* chunk, void* ctx)
{
    ESP_LOGI(TAG, "%.*s", (int)chunk->content_size, (const char*)chunk->content);
}

livekit_data_stream_handler_t handler = {
    .on_recv = on_text_chunk,
};
livekit_room_data_stream_topic_register(room_handle, "lk.chat", &handler);

The optional on_open and on_close callbacks in the handler can be set for additional behavior such as logging stream metadata or detecting abnormal closure.

Sending a text stream:

livekit_data_stream_options_t opts = { .topic = "lk.chat", .is_text = true };
livekit_data_stream_handle_t stream;
livekit_room_data_stream_open(room_handle, &opts, &stream);
livekit_room_data_stream_write(room_handle, stream, (const uint8_t*)"hello world", 11);
livekit_room_data_stream_close(room_handle, stream);

Sending a byte stream:

uint8_t image_data[4096];
size_t image_size = read_image(image_data, sizeof(image_data));

livekit_data_stream_options_t opts = {
    .topic = "image",
    .is_text = false,
    .total_length = image_size,
    .has_total_length = true,
};
livekit_data_stream_handle_t stream;
livekit_room_data_stream_open(room_handle, &opts, &stream);
livekit_room_data_stream_write(room_handle, stream, image_data, image_size);
livekit_room_data_stream_close(room_handle, stream);

Publish a user packet containing a raw data payload under a specific topic:

const char* command = "G5 I0 J3 P0 Q-3 X2 Y3";

livekit_payload_t payload = {
     .bytes = (uint8_t*)command,
     .size = strlen(command)
};
livekit_data_publish_options_t options = {
    .payload = &payload,
    .topic = "gcode",
    .lossy = false,
    .destination_identities = (char*[]){ "printer-1" },
    .destination_identities_count = 1
};
livekit_room_publish_data(room_handle, &options);

Please refer to the LiveKit Docs for an introduction to the platform and its features. For more detail about the APIs available in this SDK, please refer to the API Reference.

• In some cases, a remote participant leaving the room can lead to a disconnect.

We invite you to join our developer community to ask questions, suggest improvements, or discuss how you can best contribute to this SDK.