LiveKit Community

also wondering if there is a way donate to devs ? does it have opencollective, etc?

for TURN/TLS, you'd place a network load balancer in front of it. We just updated the docs on the various ports and how to handle them here: <https://docs.livekit.io/deploy/ports-firewall#ports>

<@U01KZ61GV5Z> I already have LB in front of LiveKit, I can attach turn also to it, but that would mean that bandwidth limitation for turn would be limited to LoadBalancer throughput?

it shouldn't be.. TURN/TLS is used by a fairly small % of all of your users. Also, cloud load balancers can handle quite a bit of load. (it scales to multiple physical machines behind the scenes)

<@U01KZ61GV5Z> I understand, I using single machine to make load balancing of my websocket servers and used it for livekit websocket also.

Horewer it has only 100mbit/s bandwitch (as websockets not require more).

Now I understand that can be a bottleneck with LiveKit turn. Need to upgrade that machine to have more bandwidth at least :eyes:

Connection tester is very helpfull! Thanks alot for that tool.

for some reason reconnection after disconnect not works for me in connection tester, but I cant find anything wrong from the server side configurations.

<@U02GTF5KZ6H> reconnect currently only works with redis-based setup. it'll require a bit of work to make it work for single node installs. See <https://github.com/livekit/livekit-server/issues/83>

<@U01KZ61GV5Z> I have single instance, but it attached to redis.

image.png

ok, I just tried again and it seems to be working well, no idea what was wrong first time

BTW, the new deployment guides are great.   For Digital Ocean it’s quite nice to combine the cloud-init given with automated certbot certificates: <https://www.digitalocean.com/community/questions/using-cloud-init-to-automate-the-let-s-encrypt-process-on-new-ubuntu-nginx-droplets>
I set this up last week and it’s :butter:

<https://github.com/livekit/livekit-server/releases/tag/v0.13.3>

yes, eventually. Currently it supports web, ios, and android

thanks David! is that planned in the near future? or is it something the dev team is not interested in ?

definitely interested :slightly_smiling_face: just a lot of things waiting to be built!

that's great!! I'll just use web for windows until we get there :wink:

sounds like a plan! we are here to help if you run into any issues

great, I've been using ion and ion-sfu until I was refered to livekit from the ion slack :slightly_smiling_face:

so far, I love the features from livekit, it's awesome

<@U02HN8GPUBY> glad to hear things are working well for you! We're also happy to consider a PR if you want to contribute Windows support for Flutter. :blush: 

I'm new to LiveKit, but that doesn't seem to be possible. (from the docs: "When a new room is created, the node that received this request is able to choose an available node from the cluster to host the room.")

not currently. It is a good idea though, we've only recently added region-awareness to the routing layer, but have not hooked it up to room creation

<https://docs.livekit.io/guides/server-api#createroom>

in some cases it is useful to have only the streams in original format

hey <@U02BBKNMF1C>, you can do this today with our <https://github.com/livekit/server-sdk-go|Go SDK>. You could connect to the room (and be invisible as a bot). It'll have access to all of the tracks in the room, just as another client would.

Thanks <@U01KZ61GV5Z>, would be wonderful that livekit-recorder could do it automatically and upload it to the s3 bucket

I will try to build something using Go-SDK

right, I think that'll be a frequent use case too. it'd be great if you can contribute! I think feeding the stream via gstreamer to save as mp4 would be great.

it's still there, tho that's not the right link to it

<https://github.com/livekit/protocol/tree/main/auth|https://github.com/livekit/protocol/tree/main/auth>

Hi Alex, looks like you using the Flutter SDK. Did you try the example app included with the SDK?

<@U029G6PA4D7> I just tried it, and it had the same problem

Sounds like the PeerConnections are failing to establish connections. Is your server set-up locally ?

no, I deployed it on a virtual machine behind NAT with SSL certificates (accessing it through wss://) in the flutter sdk example app, I get this line when it disconnects:

`I/flutter ( 4253): FINE: [EngineEvent] RTCEngine#952893545 EnginePublisherIceStateUpdatedEvent`

On browsers I know how to debug this, using the about:webrtc or <chrome://webrtc-internals>  to see the ice connections, I'll see how to get ICE related info to be printed out using livekit

any ideas how to get the SignalEvent's on the client side ?

Can you try the connection tester to check server is set-up correctly? <https://livekit.io/connection-test>

seems to work, I have everything except TURN, which shouldn't be needed for this

I think I've found it, the port_range declared in config.yaml were not correctly mapped in my docker-compose

oops, no, it's still happening, it works fine with chrome, but it fails on android.

I'm pinpointing the issue: it's the port 7881/tcp that makes the connection drop after 60 seconds, if I block that port so it's not trickled, everything works fine.. I understand that is a fallback when UDP is not reachable, but UDP is reachable, so I'm missing something in the big picture

interesting.. that's ICE/TCP, which isn't normally used. what does your docker compose look like? Is it using host networking?

<https://docs.livekit.io/guides/deploy/recorder/>
you’ll want to either deploy enough replicas to cover your rooms, or use autoscaling based on CPU usage

In the future we plan on adding custom autoscaling (e.g. always have at least 3 recorders available), but we don’t have that functionality yet

Ok, I see, it reminds me jitsi and its jibri extension. is it possible to record in custom backend using server api and what drawbacks you can see in this approach?

Yeah, the drawback would be that you need to decode and combine the individual video tracks into a visual layout (and also combine the audio tracks). Our recorder lets chrome handle all the webrtc and rendering, and then only needs to record a single audio and video source

How Recording request is handled when all the instances are busy ? Is it queued somehow or just rejected/ignored ?

yup, Published means the client has received the track, but not yet subscribed to it. TrackSubscribed would give you non-null track

Hi <@U01KZ61GV5Z> The TrackSubscribedEvent is not fired for tracks I manually subscribe to (by setting the subscribed=true attribute). Is that normal?

edit: I just tested some more, it is fired when running flutter on chrome/web-server, but not on flutter android.

That sounds like a bug. The behavior should be consistent across platforms. CC: <@U029G6PA4D7> <@U01NZKTNE1H>

<@U01KZ61GV5Z> <@U029G6PA4D7> <@U01NZKTNE1H> same problem on iOS, android and macos, it seems the subscribed=true is not working (it keeps false after setting it, thus no firing the event)

btw: on macos I had to change the dependencies to flutter_webrtc: 0.7.0 to compile

hey Alex, we found the issue and will be fixing it shortly. tl;dr: selective subscription does not work on Flutter.

Thanks Alex for trying out Flutter SDK! It is a known problem and already fixed. Fix will be in the next update. Yes we just rolled out flutter_webrtc 0.7.0 a few hours ago with macOS support and Android Simulcast :slightly_smiling_face: The next SDK version will use 0.7.0.

that's great news! I was checking out the code, and saw it was missing the sendUpdateSubscription, I was trying to figure it out to fix it, then I read this comment :smile:

You can directly use `main` to check it out :slightly_smiling_face: like this in your pubspec:
```  livekit_client:
    git:
      url: <https://github.com/livekit/client-sdk-flutter>
      ref: main```
main is using flutter_webrtc 0.7.0 already.

I just tried the main branch with the example, and it crashed on connect
<https://hastebin.com/ifetahozuw.yaml>
tomorrow with more time, I'll diff the changes from master to previous version, and patch just what is needed for the events bug, to see if same thing happens

Is that android? does it happen consistently on your device?

it crashes consistently on Android Emulator, it works fine on a physical android device... but it used to work on Android Emulator on previous version.

Is this when you publish local video ? It might be related to `flutter_webrtc`, I will take a look.

exactly, if I don't publish the local video, it doesn't crash.

just wanted to confirm, are you using ARM ver of android emulator ?

I'm using x86, I'll try arm64 to see what happens

there's an issue reported on flutter_webrtc about this same crash: <https://github.com/flutter-webrtc/flutter-webrtc/issues/734>

Hi Alex, we are looking into this with <@U02GAC58A79>, it seems to be related to the hardware encoder. <@U01NZKTNE1H> told me we might need to update from WebRTC M92. Since there is better null checking when hardware  encoder is not available. <https://webrtc.googlesource.com/src/+/refs/heads/main/media/engine/simulcast_encoder_adapter.cc#725>

Hopefully we will have this fixed soon. Thanks to <@U01NZKTNE1H> <https://github.com/webrtc-sdk/webrtc/pull/11> :man-bowing:

Livekit is amazing! I was using Intel Collaboration Suite for WebRTC, and it was a living nightmare! I'm so glad I found this framework!

Hi I checked the pull request comment today. Starting today, i  will start working on the key provider. ( <https://github.com/livekit/livekit-server/pull/99> )

are you using react or something else? I've used this before: <https://github.com/roderickhsiao/react-aspect-ratio>

<@U01KZ61GV5Z> oh well, im using plain javascript / jquery :smile:

I already make it work how I need with some simple calculation

you'd want `setSubscribed` on the `RemoteTrackPublication`: <https://docs.livekit.io/client-sdk-js/classes/remotetrackpublication.html#setsubscribed>

<@U01KZ61GV5Z> there doesn't seems to be an equivalent function like that on client-sdk-flutter

ah yes, for flutter <https://docs.livekit.io/client-sdk-flutter/livekit_client/RemoteTrackPublication/subscribed.html|subscribed> is the right setting. we'll improve the docs there

example code here for other folks finding the thread: <https://docs.livekit.io/guides/room/receive#client-side-selective-subscription>

I'm going through the same challenges, I found out when the publication is not subscribed, the track is null. So that's why it checks for it I guess

The question from my standpoint is why there are some subscriptions with no associated notification (ie, callback invocation), when in the the documentation it says the track subscribed event will "always fire" (unless tracks aren't ready for use). It's not clear from the documentation *or* example code how you're actually supposed to use livekit.

Aren't you getting TrackSubscribed events on some subscriptions?

You won't get TrackPublished events for already published tracks when joining the room, but you should get the TrackSubscribed events for those.

My experience was that the behavior was different for chrome users than Safari users.

(perhaps due to differences in webRTC implementations)

I'm experiencing a similar issue on Flutter android, TrackSubscribed events are not getting fired, but the same code fires that event when running it on flutter chrome.. maybe a bug?

Hey <@U02HS7BES0H>, I appreciate the feedback.

for track subscribed, I don't think the line in the sample code is needed. as the docs indicate, TrackSubscribed should be fired on all tracks, regardless of if the source participant is in the room or not.

<@U02HN8GPUBY> I see your issue with flutter &amp; android. will address that separately

Hey <@U01KZ61GV5Z>, thank you for your reply. I just verified that what I mentioned is true - for me, the line in the sample code is absolutely required for Safari (desktop) users if there is anyone already in a room when they join. Otherwise, they never receive subscription events for tracks corresponding to people already in the room. Why there is a platform difference, I have no idea, but it's completely reproducible for me. It doesn't matter if the clients already in the room are chrome or safari ... the new user won't get those notifications unless I do it myself.

Some of the (generated) documentation for events is also incorrect in terms of the arguments given. For example, this says there's no track argument for `TrackUnsubscribed` events:

<https://docs.livekit.io/client-sdk-js/enums/roomevent.html#trackunsubscribed>

It also appears the requests made by the webhooks don't set the content-type to `application/json` ... It would be great if this could be added as well.

I am also seeing a behavior where upon calling `.disconnect`, it will deliver additional unsubscribe messages for tracks that have already been unsubscribed (and the user disconnected already - ie that user isn't even on the server anymore).

The unsubscribes have identical `sid` values, so I think they really are repeats.

For now I'm simply ignoring the duplicated unsubscribe messages, but it is very confusing.

hey <@U02HS7BES0H> you are right this is an issue in Safari! This is due to the TrackSubscribed prior to `connect` is able to return. The timing issue exists because you are probably publishing a track during connect and causing Safari to pops up a permission dialog. we'll open an issue to track and fix

also, if you are seeing things that should improved, please open issues and/or PRs! LiveKit is an open source project and we'd love for folks to help us make it better

issue filed: <https://github.com/livekit/client-sdk-js/issues/57> it should be fixed in the next release

Screen Shot 2021-10-21 at 5.40.16 PM.png

I'm not seeing the doc mismatch. it indicates track will be indeed passed in

<@U01KZ61GV5Z> It looks like the docs fixed happened 2 days ago actually: <https://github.com/livekit/client-sdk-js/commit/c6dee574e50b282fd33fb24448eba1800374a578>

Thank you for looking at the issues I mentioned.

I will see if I can create issues or PRs where possible.

In some of these cases, like the multiple unsub track messages, I'm not what a minimal reproducible case is.

lol looks like I corrected the docs then and had forgotten about it. glad to hear people are reading them 

Welcome Duan! Your work on flutter-webrtc is much appreciated!

how do you guys handle click events on an RTCVideoRenderer when building it with flutter for chrome/web ? it seems the RTCVideoRenderer absorbs all the clicks even if you wrap it with a GestureDetector.  But It works fine when building flutter for Android and iOS, it seems to be a specific issue on the web renderers

is that something that needs to be changed in the flutter-webrtc repository?  btw, I just found a workaround: using a Stack with two chidlren: RTCVideoRenderer and GestureDetector (in that order). :grin:

<@U02GAC58A79> does it make sense to fix this in flutter-webrtc? Would be ideal if it just works across platforms.

I guess it is caused by the difference in the implementation of the video renderer. The texture widget used on android/iOS can be wrapped or stacked by other controls, but flutter web uses an html5 video element (PatformView), so I think the click event should be from the video Export in element, or use absolute layout to overlay GestureDetector on top of video element.

hey y’all, <@U02BBTRGF98> wrote a quick guide on setting up LiveKit server with AWS using cloud-init: <https://blog.livekit.io/running-livekit-on-aws-7e86849d4a40?gi=d33516e6a654|https://blog.livekit.io/running-livekit-on-aws-7e86849d4a40?gi=d33516e6a654>

Let us know if you run into any issues! :pray::skin-tone-4: 

Thanks <@U02HN8GPUBY> for bringing this to my attention. I will try this on a few devices.

<@U029G6PA4D7> it also fails on iOS (I've tried the example on iphone 6s, iOS 14.7.1) I have no error messages on it though, it just freezes the video when I click the flip camera icon

:face_palm: May I know what device &amp; OS you use ? It doesn't happen with my device :thinking_face:

Although I can't reproduce the exception on my device. I changed the `restartTrack` sequence used to switch cam. <https://github.com/livekit/client-sdk-flutter/compare/6240457333db33ae530d10c0b3bab37da70e883a...4779b3306c87351a1d94d6daa8cbd745b849fbfd>

I'm not sure this will fix it but it would be great if you can try it out.
It's in the `main` branch.

Thanks <@U029G6PA4D7> this commit fixed the issue: (dispose previous stream) <https://github.com/livekit/client-sdk-flutter/commit/4779b3306c87351a1d94d6daa8cbd745b849fbfd>

at least partially, now the camera switch works, but only the remote participant can see the change, on the local participant side, the track is frozen on the last frame, no matter if I switch back and forth again, it keeps stucked for the local Participant.

same behaviour on both devices I tested:

• Samsung Galaxy S7 (SM-G930F) on Android 8.0.0
• Iphone 6S on iOS 14.7.1

Ok I think I know why, it might be a event thing.
Thanks Alex for all the reports :man-bowing:

on the contrary, Thanks for fixing them :smile:.. I come from intel collaboration suite for webrtc, where you report a bug, and several months and releases later, they don't even acknowledge or fix it

<@U02HN8GPUBY> The local participant view not updating should be fixed in the `main` , Android issues will take more time :slightly_smiling_face:

<@U029G6PA4D7> that's great, I'm pulling to test, is there a reason why the TrackPublishedEvent is using a TrackPublication instead of RemoteTrackPublication on this new update?

Yes that's because `TrackPublishedEvent` is emitted for both LocalTracks and RemoteTracks. But the fact that you most likely need to cast all the time isn't so great.

<@U029G6PA4D7> it works now! :partying_face:

but sometimes It fails if you flip camera too fast

there are some messages: "camera device could not be opened because there are too many other open camera devices", even when it can switch cameras fine, it seems to retry 2 or 3, but if it fails, it doesn't throw an exception on _createStream()

I've tried to set a 1 sec delay between dispose and _createStream, and that seems to reduce the problem a lot...

weird thing there's an await for dispose().. maybe it's returning earlier than actually disposing the camera at a lower level. (just guessing)

You might be right, `flutter_webrtc` might be returning `dispose()` too early. Even I'm awaiting and correct sequence may cause issues. We definitely should make this perfect since in real world apps users may fiddle with it and cause crashes :grimacing:.

スクリーンショット 2021-10-28 3.13.02.png

スクリーンショット 2021-10-28 3.13.31.png

Digging into `flutter_webrtc` 's code I think we need to call `stopCaptureWithCompletionHandler` instead and return only when the completion handler is called. I will make a PR later.

<@U02GAC58A79> <@U01KZ61GV5Z> This PR should fix it <https://github.com/flutter-webrtc/flutter-webrtc/pull/748/files> I will test it now.

impressive!!! I'll give it a try later :partying_face:

We need to wait until it's merged in `flutter_webrtc` and it's tagged. I'll let you know when it's ready :slightly_smiling_face:

Hey Mustafa, are you setting the domain and attaching the cert/key files as well? Can you share the output of your `<chrome://webrtc-internals>` ?

Screen Shot 2021-10-26 at 3.58.05 PM.png

Screen Shot 2021-10-27 at 5.56.37 PM.png

Hi David, please find the screenshot below. Additionally,  I’ve pasted the turn configuration section from livekit’s `config.yaml`. Appreciate your help!

```turn:
  enabled: true
  udp_port: 3478
  tcp_port: 443
  domain: <http://turn.robotbhai.com|turn.robotbhai.com>
  cert_file: /opt/livekit-server/ssl/turn.crt
  key_file: /opt/livekit-server/ssl/turn.key```

instead of `tcp_port`, specify `tls_port`

see <https://docs.livekit.io/deploy#turntls>

Argh! Sorry about that, I thought I had pasted it from the sample config — must have updated it at some point! Thank you, connection-tester shows all is well with turn now.

Hi I managed to get a LiveKit instance setup and connect with the react sample app. I'd like to run a load test. I compiled the load tester cli. But when I start it up I always get a room closed message

This is the closest I could find to instructions:
<https://github.com/livekit/livekit-cli>

But if these aren't sufficient, we should definitely add some more documentation to the official docs.

Maybe <@U01UB0LHTTQ> can help unblock here?

./livekit-load-tester —url wss://&lt;my url for LiveKit server&gt; —run-all —room load-test

Get the following msg 

could not connect after timeout

Do you have your API key and secret set in env vars?
That command works for me

Yes I do I set them before the command but let me try again

the load tester isn’t connecting to your server - do you see any traces of connection in your livekit-server logs?

Seems my turn server config doesn't work. I tried using udp. But the connection tester app showed me it doesn't work

But if I turn the turn server off I get farther but the load-test still doesn't complete 

I guess we need turn on tls to get it to work? <@U01UB0LHTTQ> 

Some firewalls will block everything except turn/tls, but if you can connect from the react app that's probably not the issue. 
<https://docs.livekit.io/deploy|https://docs.livekit.io/deploy> for more info on turn

I have the same problem，I test with livekit-cli in intranet ,everything is ok, but when i test on my vm on cloud,  it failed with msg could not connect after timeout, I have test with <https://livekit.io/connection-test> , everything is ok,  Connecting to signal connection via WebSocket✓
Connected to server, version 0.13.7-SNAPSHOT.
Establishing WebRTC connection✓
Can connect via TURN✓
Can publish audio✓
published 137 audio packets
Can publish video✓
published 393 video packets
Resuming connection after interruption✓
TEST COMPLETE

And I found panic issue when run loadtest ,
```if len(payload)&gt;8 {
   sentAt := int64(binary.LittleEndian.Uint64(payload[len(payload)-8:]))```
sometime  the payload is less than 8 ,so it will broken , and some error mesage pc ERROR: 2021/11/01 14:49:36 Incoming unhandled RTP ssrc(92223750), OnTrack will not be fired. mid RTP Extensions required for Simulcast

<@U02CD6JKTSM> the load tester should always be sending payloads &gt; 8
```func (p *LoadTestProvider) NextSample() (media.Sample, error) {
	ts := make([]byte, 8)
	binary.LittleEndian.PutUint64(ts, uint64(time.Now().UnixNano()))
	packet := append(make([]byte, p.BytesPerSample-8), ts...)

	return media.Sample{
		Data:     packet,
		Duration: p.SampleDuration,
	}, nil
}```
do you have a way to reproduce the panic?

<@U01UB0LHTTQ>  sure, I can send my test env to you ,

<@U01UB0LHTTQ> it took awhile but I got a cert no the connection-tester shows everything succeeds and it seems a bit intermittent but got it to work. Sometimes it succeeds other times I get a timeout 

Is there a recommended server size? I'm using a t3.medium on aws just for testing purposes. Just wondering if that is part of the problem 

For simple testing just to get it connecting, a t3.medium should be sufficient

If you're doing a full load test using `--run-all` your server will want 8 cores - we ran that on a c2-standard-8. It also can't be running in single port mode, or the port becomes a bottleneck when forwarding thousands of tracks.

TURN server is a pure relay, i. e. it relays traffic between two peers. While doing P2P, it is possible that two clients cannot directly talk to each other for media traffic exchange (due to NAT/Firewall). In those cases, they will relay the media traffic via a TURN server. TURN server knows which ip/port from which packets will come in and which ip/port it needs to relay those packets to. It does not have any notion of a conference or a room.

SFU (Selective Forwarding Unit) have the notion of a room. A room has a bunch of participants. Each participant publishes their media to the SFU. SFU will broadcast (fan out) that media to all the other participants in the room. It is also `Selective` in the sense that it may choose to forward video of some publisher and may not forward video of some other publisher (based on things like bandwidth constraints of the receiving participant, business rules, etc).

Awesome explanation! Thanks a lot!
So TURN server looks like more generic version of relay functionality. SFU sounds more appropriate for Group Calls which has rooms where as TURN mainly for broadcasting is it?

You are welcome.

SFU is for broadcasting.

TURN is one-to-one. Basically, you can imagine machine A and machine B want to talk directly using P2P. But, they cannot because of NAT/Firewall. So, machine A and machine B will use the TURN server to relay traffic between them. The route will be machine A &lt;-&gt; TURN server &lt;-&gt; machine B. TURN server knows that it will get packets from machine A and it has to be send to machine B and from machine B to machine A. That's it. No broadcasting.

Got it! Thanks once again. I will read further on this as now I have a basic understanding!

<@U02GWB761DK> loosely related, but if you ever want to go deep on NAT traversal, this is a wonderful post on that topic: <https://tailscale.com/blog/how-nat-traversal-works/>

different behavior for platform might be a `flutter_webrtc`  issue, I'll let you know If I find something!

<@U029G6PA4D7> it might be related to this flutter-webrtc issue, I just added some comments on it, hoping some flutter-webrtc developers catches it
<https://github.com/flutter-webrtc/flutter-webrtc/issues/653>