completely open source including embeddings

The rust sdk works very well, i did a trim down version of the ai agent with rust, it turned out very nicely well done the livekit team

quick question, how do you capture raw wav audio ( no wav headers ) to a livekit room. ( bytes::Bytes / Vecu8 to Veci16 ). testing out deepgram's aura model ( text to speech ) https://developers.deepgram.com/docs/text-to-speech ( figured it out ). I also tried sending audio to deepgram's tts platform without resampling and it works great

hey is there a way to consistently get the

RoomEvent::TrackSubscribed

event after connecting to a room. When I run my livekit server locally I get the event like 6/10 times. feels random at the moment. Can I make a request to livekit to get the room event if i don't receive it after a while?

only 3 events I've been getting about 50% of the time -

hello ! do you guys have a timeline for bringing the rust sdk at feature parity with other native sdks ? (svc codecs, adaptive streaming, dynacast) even a rough idea 🙂

Hey everyone! Elliot here- mucking about with getting webRTC running on visionOS. Looks like its pretty close to the same config as iOS. Anyone else looking into this? Looks like there's been a couple commits to the libwebrtc repo to that effect.

Anyone ever encountered this issue on

cloud-sdk-unity

for iOS build?

Has anyone ever encountered error like this when trying to connect to a room in Unity iOS?

UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)

[2024-08-01T03:04:45Z ERROR livekit_ffi::server::room] error while connecting to a room: engine: signal failure: ws failure: IO error: invalid peer certificate: UnknownIssuer
Failed to connect to the room !
<Start>d__11:MoveNext()
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)

Hey all, I did a fresh clone off https://github.com/livekit/rust-sdks/blob/main/examples/save_to_disk/src/main.rs. I'm experiencing an extensive error about webrtc-sys (I will send a text snippet in this thread). All I did was run

cargo run

My system is Ubuntu 22.04 (PopOS) I'm wondering if anyone have any ideas how to resolve this?

I'm seeing a lot of warnings coming from

webrtc-sys

. Is it supposed like this? or are there any local setup I'm missing? This is coming from running

cargo run

on

rust-sdks/examples

We're seeing some linker conflicts from the liveKit ios library ffi and another library we use which also has some audio-related webrtc symbols. would it be possible to wrap all the webrtc symbols with a namespace to avoid conflicts?

Hi, This is my first day of checking Livekit and am kind of in a hurry. I'm using the Unity example project of LiveKit. I want to publish my audio but don't want to hear it back. How can I turn it off? Appreciate your help

Hi. I am facing some issues while developing a command-line voice-chat app using livekit rust sdk. 1. I am using cpal package to capture and play audio. While playing audio I am using livekit's AudioResampler to resample the received frames like this

// Resample the audio frame
                    let data_i16 = resampler.remix_and_resample(
                        &frame.data,
                        frame.samples_per_channel,
                        frame.num_channels,
                        frame.sample_rate,
                        num_channels,
                        sample_rate,
                    );

                    // Convert i16 samples to f32 before sending to cpal as cpal works with float samples
                    let data_f32: Vec<f32> = data_i16
                        .iter()
                        .map(|&sample| sample as f32 / i16::MAX as f32) // Normalize i16 to f32
                        .collect();

                    let _ = sender.send(data_f32).await; // Send the f32 samples to the cpal callback

The main issue is that there is quite a bit of buzz in the sound. Any idea how to fix this? I have a sample app in Flutter, where this low level audio handling is not required and it seems to have perfect audio. 2. The

ActiveSpeakersChanged

event seems to be flaky. It immediately goes from a list of one member (when I start speaking) to empty list in an instant.

from the unity sdk on ios: [2024-11-05T134419Z ERROR livekit_ffi::cabi] failed to handle request FfiRequest { message: Some(CaptureAudioFrame(CaptureAudioFrameRequest { source_handle: 85, buffer: Some(AudioFrameBufferInfo { data_ptr: 5878539280, num_channels: 2, sample_rate: 24000, samples_per_channel: 1024 }) })) }: InvalidRequest("handle not found")

if I delay starting the rtcAudioSource then I can get around that error on publish but I get it at room disconnect too: [2024-11-06T085901Z ERROR livekit_ffi::cabi] failed to handle request FfiRequest { message: Some(CaptureAudioFrame(CaptureAudioFrameRequest { source_handle: 195, buffer: Some(AudioFrameBufferInfo { data_ptr: 5926510608, num_channels: 2, sample_rate: 24000, samples_per_channel: 1024 }) })) }: InvalidRequest("handle not found") I also see the same thing I was seeing before where the audio track isn't showing up on my server (though the unity client says audio track published

Hi. We are using Livekit with C#, we have a rust library built to connect the two, and for some reason on my local machine it has stopped successfully creating a livekit room. It was working previously and we have not changed the code for this, it works fine when pushed up to our sandbox environment, other people are not experiencing this same issue when checking out my local branch. This is the error I am getting: {"failed creating room: Twirp(Request(reqwest::Error { kind: Decode, source: Error(\"expected value\", line: 1, column: 1) }))"} Our use case seems pretty uncommon as we have not found anything online from someone having a similar experience, can anyone assist?

Hello everyone, I'm trying to run the

save_to_disk

example on Linux arm64 (Raspbian to be precise). It compiles fine, but the app crashes with the following:

free(): invalid pointer
[1]    20607 abort      RUST_LOG=debug cargo run --bin livekit-test

By commenting bits out, I narrowed the issue down to the following line:

let (room, mut rx) = Room::connect(&url, &token, RoomOptions::default())
        .await
        .unwrap();

The

LIVEKIT_URL

and

LIVEKIT_TOKEN

env vars are set. Am I missing something basic here? Any way I can debug this further? The exact same code runs fine on a Mac.

Hello everyone, I am developing a Tauri-based WebRTC application in Rust targeting macOS and Windows platforms. For this, I’ve integrated the LiveKit Rust SDK and am using PeerConnection and related classes directly, as I am not utilizing the LiveKit cloud or OSS. However, I couldn’t find APIs in the SDK to retrieve the microphone stream, which is required to pass to the add_track function of the peer connection. Similarly, I haven’t located any APIs to play the audio stream received in the on_track() callback through the speakers. I couldn’t find relevant examples that demonstrate this functionality. Could someone confirm whether such APIs for managing media devices are available in the LiveKit Rust SDK? Any guidance or pointers would be greatly appreciated. Thank you!

has renamed the channel from "rust-developer-preview" to "sdk-rust"

Hi team, I have a question that I hope can be answered. LiveKit uses Go as the server-side development language, and the core of the client SDK is built using

rust-sdks

. Regarding the

server-sdk-go

, this library is primarily for server-side interactions. Since WebRTC is P2P-based, I'm wondering if I can use the

server-sdk-go

library as the client SDK in the core layer? I don't fully understand the differences between the

rust-sdks

and the

server-sdk-go

. What I know is that media stream capture and platform-related features are likely implemented in Android or iOS layers, but I'm concerned about the differences between the

rust-sdks

used in the core layer and the

server-sdk-go

. As far as I know, both libraries may only expose the core WebRTC interfaces, but I wonder if there are significant differences. Regarding using Go and Rust for the core layer, I already understand that Rust performs better, has smaller package sizes, and provides better client-side support. If we ignore these factors, can I directly use the

server-sdk-go

to build a core layer like the

rust-sdks

? Apologies for the detailed question—this is based on the needs of our project. Our core layer is implemented in Go, but due to Go's runtime limitations, we cannot include two Go libraries simultaneously. So we are considering merging them into one SDK. However, we're unsure if the core capabilities of

server-sdk-go

(which is designed for the server side) can be used similarly to the Rust

rust-sdks

. We can add platform-specific features (e.g., Android and iOS), but we need to confirm whether this is feasible. Thank you very much!

I think if LiveKit and Dioxus had a rusty baby, it would be beautiful.

With Dioxus you could have one codebase in rust for core logic and UI on both mobile and desktop plus embedded systems with the upcoming release meant to take on Slint and EGUI.

The nice thing about Dioxus the you don't need advanced skills just CSS or Tailwind to build your GUI binary for everything at once . . Android Play Store, Apple Store, Windows Store, raspberry Pi etc.

Hey guys, I'm facing a weird problem with rust sdk, when I publish 3 tracks with different codecs all three tracks will publish with the codec of first published track. I mean consider I want to publish 3 tracks like these : Track 1 with AV1 Track 2 with VP9 Track 3 with VP8 all 3 tracks will be published with AV1 codec and if I change the order of publish like this : Track 1 with VP8 Track 2 with VP9 Track 3 with AV1 then all 3 tracks will be published with VP8 codec. I tested these scenario with python sdk and there was no problem with that and it happens only with rust sdk. @dry-elephant-14928 @boundless-energy-78552 Sorry for tagging you, do you have any comment on this? Thanks

hi, team . I use java call rust , jvm crash . @plain-wall-1315 @dry-elephant-14928 Here is dump info . I need the reason , and method to fix crash .

#  SIGSEGV (0xb) at pc=0x00007a7be9bfc1f1, pid=36709, tid=0x00007a7bd8c00640
#
# JRE version: Java(TM) SE Runtime Environment (8.0_333-b02) (build 1.8.0_333-b02)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.333-b02 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [liblivekit_rust_sdk.so+0x1fc1f1]  tokio::sync::mpsc::list::Tx$LT$T$GT$::push::h14314639d7a74154+0x11
#

Stack: [0x00007a7bd8a01000,0x00007a7bd8c01000],  sp=0x00007a7bd8bfe6d0,  free space=2037k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [liblivekit_rust_sdk.so+0x1fc1f1]  tokio::sync::mpsc::list::Tx$LT$T$GT$::push::h14314639d7a74154+0x11

[error occurred during error reporting (printing native stack), id 0xb]

Hi Livekit team and community. Has anyone ever needed to retrieve video track in Python with a fixed maximum resolution? I see in the JS SDK that you can set the desired resolution with

setVideoQuality

, but I can't find an equivalent on the Python side.

Hi team When I try to unpublish tracks multiple times using rust sdk there will be some memory leaks. I used heaptrack in order to track memory leak location and it seems that it's from VP8 - VP9 and AV1 encoders Do you have any idea? also I created an issue for that on github : https://github.com/livekit/rust-sdks/issues/612 here is the heaptrack screenshot :

🚀 Rust + LiveKit on macOS – Need Your Expertise! 🚀 Hey everyone! I’m building a Rust client app using the LiveKit SDK on macOS, but as soon as I call

connect()

, it fails immediately and I get disconnected. I’ve already: • Tried all the official Rust SDK examples • Cross-checked community snippets • Double-checked my Access Token and URL …and I’m still stuck. 🤔 Has anyone run into this issue before or spotted something I might be overlooking? Any tips, pointers, or dark-horse workarounds are more than welcome! 🙏 Thanks a ton in advance! 💪

*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[RTCVideoCodecInfo nativeSdpVideoFormat]: unrecognized selector sent to instance 0x157044e00'
*** First throw call stack:
(
        0   CoreFoundation                      0x000000018add2ca0 __exceptionPreprocess + 176
        1   libobjc.A.dylib                     0x000000018a896b90 objc_exception_throw + 88
        2   CoreFoundation                      0x000000018ae8fb90 -[NSObject(NSObject) __retain_OA] + 0
        3   CoreFoundation                      0x000000018ad40ddc ___forwarding___ + 1500
        4   CoreFoundation                      0x000000018ad40740 _CF_forwarding_prep_0 + 96
        5   interscribe-desktop-app             0x0000000104263454 _ZNK6webrtc23ObjCVideoEncoderFactory19GetSupportedFormatsEv + 256
        6   interscribe-desktop-app             0x00000001044c023c _ZNK7livekit19VideoEncoderFactory15InternalFactory19GetSupportedFormatsEv + 184
        7   interscribe-desktop-app             0x00000001044c1404 _ZNK7livekit19VideoEncoderFactory19GetSupportedFormatsEv + 52
        8   interscribe-desktop-app             0x0000000104448c8c _ZNK7cricket17WebRtcVideoEngine11send_codecsEb + 100
        9   interscribe-desktop-app             0x0000000103c0fb80 _ZN7cricket30MediaSessionDescriptionFactoryC2EPNS_20MediaEngineInterfaceEbPN3rtc23UniqueRandomIdGeneratorEPKNS_27TransportDescriptionFactoryE + 324
        10  interscribe-desktop-app             0x0000000103c4d320 _ZN6webrtc31WebRtcSessionDescriptionFactoryC2EPNS_17ConnectionContextEPKNS_16SdpStateProviderERKNSt3__112basic_stringIcNS6_11char_traitsIcEENS6_9allocatorIcEEEEbNS6_10unique_ptrIN3rtc32RTCCertifi        11  interscribe-desktop-app             0x0000000103bb5b44 _ZN6webrtc21SdpOfferAnswerHandler10InitializeERKNS_23PeerConnectionInterface16RTCConfigurationERNS_26PeerConnectionDependenciesEPNS_17ConnectionContextE + 312
        12  interscribe-desktop-app             0x0000000103c6f584 _ZN6webrtc14PeerConnection10InitializeERKNS_23PeerConnectionInterface16RTCConfigurationENS_26PeerConnectionDependenciesE + 500
        13  interscribe-desktop-app             0x0000000103c6f18c _ZN6webrtc14PeerConnection6CreateERKNS_11EnvironmentENS_13scoped_refptrINS_17ConnectionContextEEERKNS_30PeerConnectionFactoryInterface7OptionsENSt3__110unique_ptrINS_4CallENSB_14default_deleteISD        14  interscribe-desktop-app             0x0000000103c1f200 _ZN6webrtc21PeerConnectionFactory27CreatePeerConnectionOrErrorERKNS_23PeerConnectionInterface16RTCConfigurationENS_26PeerConnectionDependenciesE + 500
        15  interscribe-desktop-app             0x0000000103c20934 _ZN6webrtc10ReturnTypeINS_10RTCErrorOrINS_13scoped_refptrINS_23PeerConnectionInterfaceEEEEEE6InvokeINS_30PeerConnectionFactoryInterfaceEMS8_FS5_RKNS3_16RTCConfigurationENS_26PeerConnectionDepende        16  interscribe-desktop-app             0x0000000103c20a48 _ZN4absl22internal_any_invocable12LocalInvokerILb0EvOZN6webrtc10MethodCallINS2_30PeerConnectionFactoryInterfaceENS2_10RTCErrorOrINS2_13scoped_refptrINS2_23PeerConnectionInterfaceEEEEEJRKNS7_16RTC   17  interscribe-desktop-app             0x0000000103d2bdf0 _ZN3rtc6Thread8DispatchEN4absl12AnyInvocableIFvvOEEE + 160
        18  interscribe-desktop-app             0x0000000103d2af24 _ZN3rtc6Thread15ProcessMessagesEi + 220
        19  interscribe-desktop-app             0x0000000103d2c2d0 _ZN3rtc6Thread6PreRunEPv + 96
        20  libsystem_pthread.dylib             0x000000018ac76c0c _pthread_start + 136
        21  libsystem_pthread.dylib             0x000000018ac71b80 thread_start + 8
)
libc++abi: terminating due to uncaught exception of type NSException