LiveKit Community

just released! helm releases are manual (for now) and I frequently forget :sweat_smile:

are you able to reproduce this with either the <https://example.livekit.io|react example> or js example app?

if so, please share the steps and we'll take a look.

I've narrowed it down to a specific line of code that causes the problem. When a track is unsubscribed, we are calling track.mediaStreamTrack.stop(). This seems to put LiveKit in a bad state. If this line is removed, the track publications don't have an issue going back to the other side. Sample app that illustrates this issue

<https://gist.github.com/jezell/208e9ec3d042ad9a44347fb95f892696>

<https://gist.github.com/jezell/e6fc211ce990ca229e6b233af5ec2e91>

Maybe there is some kind of bug introduced when tracks are manually stopped during a TrackUnsubscribed event?

This is likely the cause of a similar issue I had reported earlier (<https://livekit-users.slack.com/archives/C01KZ61HZRR/p1663740429177429>), since that code path would have also been activating this code which cleans up tracks when they are removed.

thank you for the detailed report. That is indeed the issue. LiveKit re-uses the transceivers in the peerconnection to avoid bloating the clients.

because of that, when you `stop` a MediaStreamTrack directly, it would fail the next time the transceiver is reused. We'll add a note about this in our documentation

Maybe it could also provide some kind of error in the console when that happens or it could detect that it needs to not re-use the transceiver if the track has been stopped? We probably don't need to stop the mediaStreamTrack, but would have been a lot easier to track down the issue if at something had been written out to the console.

that's a great idea.. we should log a warning here.

<https://github.com/livekit/client-sdk-js/pull/466>

I think failing the subscription is the prudent thing to do here. We cannot control if the server reuses the transceiver or not. (the spec allows for it, and client stopping it locally doesn't give server any indication of it)

I believe you set the codec on the config.yaml at launch.

I also believe(i could be totally wrong) that H264 is only on 1:1. when you add another participany i believe it defaults to VP8

<https://github.com/livekit/livekit/blob/master/config-sample.yaml#L116>

You can change the global default for a room or explicitly configure at room creation.

What Mark suggested should work. But it'll cause everyone in the room to only be able to publish VP8

Swift SDK always prefers to publish H.264 due to it having hardware encoding. This is true even if multiple participants join the room

Thanks a lot to all ! I make change on config.yaml on vp8 - and it is realy work! thanks a lot

Yep this is for mobile platforms IOS&amp;Android,
We are having this issue on Livekit react native example app.
```<https://github.com/livekit/client-sdk-react-native/tree/main/example>```

Hi <@U01NZKTNE1H>, I can see that you are using react-native-webrtc that supports transceiver APIs, but it doesn’t support peerconnection removeTrack method. I am getting the above error from this point. Are there any solution for this?

This is one of a related issue: <https://github.com/livekit/client-sdk-react-native/issues/14>

ah, looks like it's not in our fork. react-native-webrtc officially added transceiver support just recently, which has peerconnection removeTrack. we're moving to the official version soon which should fix this issue

I'm only a novice at livekit, but I experienced this when initially deploying. How are you supplying your config file? and have you specified a `keys` entry or key-file?
i.e.
```keys:
  my_key: key_value```
in `livekit-config.yml`

I supply this config as an environment variable `LIVEKIT_CONFIG`

I had not supplied the config as an environment variable. Thanks

Hello, i started livekit server on machine has 32gb   DDR3, Xeon E5-2690 3.50 GHz.

checkout: <https://docs.livekit.io/deploy/benchmark/>

You can beat that machine up and find out :slightly_smiling_face:

have you configured your AudioSession with `AudioSession.configureAudio`? by default, the preferred output is still speaker, so if you want the default to output to the earpiece, you'll need to set that. `force_speaker` is only for overriding all audio routes (regardless of whether headphones/bluetooth is connected). You may also want to check out `AudioSession.showAudioRoutePicker`

i used AudioSession.configureAudio by default
Iphone 6 version 12.5.7

expect behavior is
• I can switch between speaker and earpiece 

again, you'll either need to set your default output to earpiece with `configureAudio` or use `showAudioRoutePicker`. iOS is very particular about its audio controls, and we can't directly force output to the earpiece.

Thanks for all the kind words, David. You really made my morning. :pray::skin-tone-4:

Which checkbox don’t we cover yet? Curious if we can consider how we might be able to! :blush: 

I'm working with David on the LiveKit evaluation at Unblu. One checkbox you did not immediately tick is the Kubernetes deployment design. I would question the use of node ports in your Kubernetes deployment which is, IMHO, not a recommended practice in Kubernetes. But I think I know why you did it that way and I also don't have a better proposal right now. (We've designed our own COTURN cluster on GCP and struggled quite a bit...)

But let me double down on the positive feedback: I enjoyed reading your docs a lot!

<@U01KZ0SCDKL> another point was a missing Java / JVM based SDK. But you guys outdid yourselves there! <@U02EU2ZPTQU> mentioned you just released the Kotlin SDK a couple of days ago and <@U01NZKTNE1H> implemented the missing AccessToken feature in like 2h.
I'm speechless :open_mouth:

Thanks <@U045C8LMMCN> for the feedback! cc <@U01KZ61GV5Z> <@U02BBTRGF98> <@U02Q4J8KS85> 

hi <@U046B0LNLD7>
this was just released last week, but there’s a kotlin server sdk now, too! <https://github.com/livekit/server-sdk-kotlin>

Hey <@U02EU2ZPTQU>, thanks this looks very interesting! I'll have a look into it to see how well it would work with Java.

<@U02EU2ZPTQU> So I've checked the lib out and it seems to work fine with Java which is great.
The *Access Token* functionality doesn't seem to be part of it yet, though.
What are you plans here?
I didn't find a GitHub Issue for this so it's hard to tell how far away we are from an implementation. We definitely need the feature to be able to issue the tokens from our server and send them to the client.
*Do you want me to do a PR for it or is it already in the pipeline?*

great to hear that it’s suitable in general!
cc <@U01NZKTNE1H> regarding the access token functionality

should be pretty quick to whip up access token generation

<@U0414095DAB> access tokens are up now: <https://github.com/livekit/server-sdk-kotlin#creating-access-tokens>

<@U01NZKTNE1H> dude you're absolutely incredible!! I'll check it out right now

I just finished integrating it and it works like a charm :+1:

<@U046B0LNLD7> btw, was wondering if you had any feedback on the server api methods? the raw calls are kind of clunky at the moment, but I'm unsure of what's normal for async code on java servers

<@U01NZKTNE1H> Yes, it's not super intuitive but I got around it.
In java there's no real standard for async code. There are several approaches like RxJava (which is what we use), Project Reactor or classic Java features like Future.
Generally I'd say the majority of Java / Java Libraries are still synchronous. - As we are currently only using a small subset of your API (room management &amp; Token API) it's not a big deal. Once we add more maybe I have more feedback

That’s cool, so where a GM or players could alter their voice? :crossed_swords: 

We don’t have any official support (yet) for custom plugins/processors (either on the client or server), but it’s something we’ve been thinking about for a while. I think the first thing we’ll probably offer is a client-side framework for doing things like this — on the sender and/or receiver sides. Server-side processing has use cases, too (we do active speaker detection this way), but there are latency implications with decoding, blocking on processing and re-encoding a stream. Audio is much more doable than video.

I'm thinking the DM could use a pitch transposer for deep-voiced demons, high pitch for halflings or fairies.  Players too.  But maybe distortion for gravely voices, flanger for speaking through a dimensional portal, etc.  But also rooms where you could add a touch of reverb for big empty rooms, so players can hear their own voices, and of course massive reverb and echo for caverns.  I suppose effects could be added on the client side between mic input and sending it to the server.

this is possible on the client side w/ <https://github.com/w3c/webrtc-encoded-transform/blob/main/explainer.md|insertable streams> but at the moment only supported by chromium

at the moment we're using an approach heavily borrowed from jitsi to implement end2end encryption with this strategy: <https://github.com/Invisv-Privacy/client-sdk-js/blob/de7fe20af2beea25d2eab1dbfff48d0d29f3c4ee/src/room/track/LocalTrack.ts#L63-L71>

<@U042ZDYEB37> Perfect!  Thanks for this.  Looks a bit cutting edge, but by the time I get around to needing it, it will likely mature a bit.  So, this is the right system to jump on board with.  I can just steal the plugins from LADSPA and with a little duct-tape and bailing wire ... :rolling_on_the_floor_laughing:. Thank you!

yeah it does seem like this is (eventually) the future: <https://github.com/mozilla/standards-positions/issues/330> <https://bugzilla.mozilla.org/show_bug.cgi?id=1631263>

Image from iOS.gif

hi team, I am new to livekit. It is very helpful for our project. I would like to know, if any APIs or feature in livekit supports to get/set RTP header Extenstions, and how to get RTP header Extensions like `rtphdrextclientaudiolevel,rtphdrextcolorspace,rtphdrexttwcc`

Hope I am not being annoying about reposting this here. Trying to get a rtsp to vp8 with audio working using gstreamer. I have a rtsp to h264 however that doesn’t work on the iOS simulator which is annoying. Also my sound transformation while allowing the microphone to connect the audio does not come in clearly so I’m guessing something is wrong their as well. <https://livekit-users.slack.com/archives/C01KVTJH6BX/p1665338037721469|https://livekit-users.slack.com/archives/C01KVTJH6BX/p1665338037721469>

I would love to know the roadmap on e2ee. It's a top priority for me right now as I would prefer to use livekit over mediasoup.

<@U046CJ4SRGU> we’re aiming to support e2ee by Jan 2023!

<@U03Q1K8NXE3> we’re prepping for a launch which I’ll post about next week, but soon after I want to put up a public roadmap for folks.

<@U01KZ0SCDKL> Thank you so much for the response! Looking forward to it. The developer experience with Livekit is good enough to wait.

<@U01KZ0SCDKL>
Is the E2EE available for mobile devices too? (react native)

Thank you.

ref:
<https://jitsi.org/blog/a-stepping-stone-towards-end-to-end-encryption-on-mobile/>

<@U02SHDC9J7N>
&gt; Is the E2EE available for mobile devices too? (react native)
Our e2ee solution should be able to cover mainstream platforms, currently supports client-sdk-js, client-sdk-flutter, and we have made a FrameCryptor (aes-gcm) patch for libwebrtc, supporting Android, iOS, macOS, Windows,
So the answer is Yes, support mobile
but react-native-webrtc is maintained by another open source community, we may contribute a patch to their community, or make a patch for  react-native-webrtc  fork.
once we finish e2ee testing, it should be possible to make a FrameCryptor patch for RN,

<@U02SHDC9J7N> <@U02GAC58A79> React Native is what I was hoping to be using as well :confused:

If I recall correctly, I think currently its: VP8, H.264 &amp; AV1 but I'm sure a team member can confirm this.

I think some folks have done integrations, but this is something we’re hoping to add in 2023.

if there's a livekit dev/whoever's working on this, we implemented a pretty POC/demo-worthy version of e2ee on our fork: <https://github.com/invisv-privacy/client-sdk-js/>

it borrows pretty heavily from jitsi's implementation and is definitely not something that I feel is ready/worthy/etc of a PR but in case there's anything useful for y'all thought it would be worth sharing

Thanks Max! I've added link to your repo to the issue.

hello every one， how to fix this question？

when/how does this happen? Do you have repro steps?

Hi <@U046ASB08LU>, we try to keep this space focused on engineering or product-related discussions. Please shoot an email to <mailto:hello@livekit.io|hello@livekit.io> if you’d like to discuss a partnership. Thanks very much! :pray::skin-tone-4: 

~you can probe by sending larger padding packets. we do it here.~

We have seen it grow a little faster than that. Going from mid layer which is around 600 kbps to high layer at 1.7 Mbps happens in about 15 - 18 seconds the last I measured. I was attributing the slowness to the REMB algorithm too and the delay based BWE that is used with TWCC could be faster. That is a speculation on my part and unconfirmed.

With temporal layer filtering with VP8 (H.264 does not have temporal layers), we can actually move to high spatial layer sooner at lower fps and then catch up with further probing.

David, padding packets will be limited to 255 bytes because of RTP limitation, but we send a bunch at a time.

sorry I misread the question. thanks Raja for the clarification!

&gt; Going from mid layer which is around 600 kbps to high layer at 1.7 Mbps happens in about 15 - 18 seconds the last I measured. 
&gt; 
And you achieve this probing with padding packets only? 

In our case it's impossible to achieve such results. We spend at least 2 minutes to get into 1,5Mb/s. We must be doing something wrong

Yeah that is using padding packets only. I am sure you have done this, but just wanted to check if you ensure sending padding only at frame boundaries. Also this measurement is using Chrome. I don’t remember the details for Safari/Firefox. But they are in the same range.

Yeah, we are probing at frame boundaries. Anyway, thank you very much! That's really helpful :slightly_smiling_face:

Just wanted to say that we finally debugged the problem and it turned out that we had a bug in our GCC implementation.

We were calculating exp moving avg over all r_hats while we should use only r_hats from the decrease state.
Because of that we were starting in multiplicative mode but pretty fastly going into additive mode and the whole estimation was slowing down.

Thanks a lot for the help!

Great to hear that you found the issue Michal!!!

image.png

Looks awesome!

By "broadcasting code sample", are you looking for a full blown sample app? or are you looking for examples of how to do specific things?

I'm not a coder, but I'm trying to learn.
I understamd that LiveKit can do broadcasting, where 1 person share their live feed (from their PC I assume), while the rest of us see the feed via WebGL (cinema app like shown above).

I'm hoping someone can lead me to a right direction with a code sample of broadcasting (instead of multi user conference).

IMO it should be good on the stability side as it's a simple wrapper to make HTTP requests to the server.

Though as a 0.x release, it's possible for the APIs to change.

Give it a try and please let us know if you have feedback!

I don't think there any persistent logs being kept currently.

You can calculate the call duration using Webhooks, the rest of data I think is not possible at the moment

hey Guillermo! Give <https://docs.livekit.io> and if you have any questions or need clarifications, there’s plenty of helpful folks in here. :slightly_smiling_face: