LiveKit Community

hey Jake, you can run it on any machine that you'd like.

increasing file handles would be recommended regardless of the size of the machine used.

Hi everyone! There is some way to get over the limitation of 1 room, 1 node? we need to scale 1 room where the sum of the bitrate of the different sources * number of active participants is much bigger than what can be handled in 1 node

That is LiveKit takes care of the whole picture for you.

absolutely! We deal with webrtc annoyances so not everyone has to :wink:

I spoke with Tsahi after the post and provided a few clarifications.

I think Livekit should be on top! The best option by far :heart:

Hi Amir!

Our protocol is based on protobufs and technically can be sent across other messaging systems. with that said, our client SDKs all use WebSocket today, so if it'll take a bit of effort on your side to fork + maintain clients.

<@U0427KZNM7A> could you only post your question in a single channel (ideally #helpdesk), please? We look at all the channels, so you don't have to worry about us missing it.

I'm going to remove this copy, since I see you've also posted it in #helpdesk.

Sure <@U01KZ0SCDKL>
Thank you for reply .

Hi there! I'm glad you found us :slightly_smiling_face:

There are definitely trade-offs in the approach between different providers. For example, if you are looking for a low-code solution with existing templates, some providers could meet those needs better.

Our goal is focused on making the best RTC stack accessible to all. Whether it's self-hosted, or we can help with hosting it.

Got it. Thanks David! Also, another question if you don't mind. I've noticed that the other paid services price based on the number of minutes.

But since WebRTC is peer-to-peer, doesn't it just require a server for the initial handshake? So if I'm hosting LiveKit using a DO droplet or AWS, my hosting costs remain static for the number of minutes.

*I'm kind of new to this, so my understanding might be wrong :sweat_smile:

There's definitely a lot of confusion around this. I'll write a blog post about it someday.

WebRTC *can* be peer to peer, but
• it doesn't scale beyond 2-3 people in a room
• signaling state can be complicated/error prone
For those reasons, LiveKit (and others) adopt a SFU-model, where each user connects to the server. This means traffic does go through to the server and it will incur cost for the provider hosting the server.

I'd appreciate a blog post on this for sure! But if it's 1 to 1 then it can be peer to peer? And therefore not incur server costs?

My application is strictly 1 to 1 so if other providers are making me pay by the minute that's a game changer.

that is possible, but because of the complexities (and many livekit features rely on server-side support), we do not support peer to peer.

you could see how much bandwidth your application would use in a month, on DO, you get some TBs of bandwidth each month with every droplet

That's good to know. In that case, I'll look at LiveKit’s features to see if it's worth ditching peer to peer. It's a pretty minor detail, but having p2p gives some peace of mind for scaling up down the road.

Hi Shubham, lots of helpful folks here :slightly_smiling_face:

hi <@U043351G48Y>, i see you posted in <#C01KVTJH6BX|>, so I’m going to delete these duplicate messages. :pray:

someone will respond in <#C01KVTJH6BX|> :slightly_smiling_face:

busy, but we do read and respond — just takes a bit of time

Hey <@U03S00K6BD5>,

That makes sense to use that information. If you are interested in creating a PR, we'll gladly accept it.

It could definitely get smarter about how to place rooms, especially if there was information about relative capacity of the nodes they are running on.

what do you mean by putting room metadata in kubernetes?

Great, we'll put something together. The basic thing we are trying to accomplish is a minimal set of changes to LiveKit to allow it to integrate a little tighter with Kubernetes so that Kubernetes can be responsible for scaling the nodes up and down with something other than basic CPU scheduling.

The optimal pathway would be for kubernetes to schedule a pod per room and run the livekit service inside the scheduled pod. This is probably worth exploring, but it's a bit of a departure from the current architecture so we feel it might not be easiest way to accomplish what we want.

The solution we have arrived at as a starting point is:

1) Change the livekit service to request zero resources, but be required on every node in the cluster.

2) At room creation time, schedule a placeholder pod in kubernetes, annotated with the room name, and CPU/mem requests suitable for hosting a room with the desired constraints (number of participants, codecs, size of video streams, etc). Inside the placeholder pod, run a simple process that monitors the status of the room and terminates automatically when the room is destroyed on the livekit side.

3) Modify the node selectors to have an additional argument with the room data. Use this to a node selector which simply talks to the kubernetes API to locate the node annotated with the room name.

In this way, node selection will be offloaded to kubernetes, which will make sure that nodes are provisioned in an optimal fashion using the room sizing information, and automatically scale the cluster up and down as the request limits reach the required threshold.

Since a large percentage of our usage is scheduled meetings with known attendees and we know quite a bit about what is likely to happen during the meeting, it also should allow us to estimate much more accurately how much capacity we will need to have provisioned at any given time.

Does that make sense?

Note that there is also a clear pathway from this to pod per room if it makes sense to pursue that in the future (it would allow for CPU/mem limiting which could be useful to make sure a room doesn't use more than it is supposed to for example), since that would just require moving the livekit service inside the placeholder pod rather than the pod just being a dummy process.

thank you for sharing your use case!

If you are able to scale up on demand, you could look at the manual node allocation path. If you specify a nodeID when creating a room, it'll allocate the room onto that specific instance (<https://github.com/livekit/protocol/blob/main/livekit_room.proto#L57>)

The flow I'm thinking of is:
• start a pod while assigning the node ID to the instance (this doesn't exist yet, but would be simple to add)
• call CreateRoom with that nodeID
• when participants join, they'll be connected to that node
with this approach, you'd just need a single livekit server instance to handle the API calls, but as you said, it doesn't have to be configured with a lot of resources

Pod per room makes sense if the livekit server is running inside each pod, but since each pod doesn't get its own external IP, what magic routing would be required to get the peer connection into the pod from the outside? I thought we were stuck with a single livekit instance per node because of the single IP address per node.

What would it take to have the node pass the same ID as the kubernetes node it is running on when it registers in the kubernetes mode? I think that's all we'd really need and we could avoid changing the node selectors to include room info if we can manually assign the node at room creation time. We could still use the placeholder pods approach and lean 100% on kubernetes for scaling and allocation that way, without enhancing the selectors to know anything about what's actually going on.

That said, I still think it's strange that the selectors can't use room info to make intelligent decisions about node placement based on metadata in the room, but maybe the right answer is just throwing out the selectors entirely when you want to do smart things with the scaling and placement of rooms and punting it to something like kubernetes anyway.

<https://github.com/livekit/livekit/blob/master/pkg/routing/node.go#L15:6>

Looks like we could maybe extend the config object to support passing a node ID as part of the config, and if the node ID is specified use it instead of generating a new one. Then, set it based on the cli flags:

<https://github.com/livekit/livekit/blob/33f782a99be1ee7fe5611244e5e6a9d5cfc22560/pkg/config/config.go#L339>

and add a new cli flag in main.go.

Then we can tell kubernetes to put its ID in the env variable the cli is reading the flag from.
<https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/>

For the pod / port issues, you could look at using different ports for each pod so they don't conflict. With using UDP/TCP muxes, it could run by using only 3 ports (HTTP, UDP media, TCP media).

and yeah, it's possible to have a config option so that it will use an environment var for node-id

Hey Ti Xu :wave:

That is just an example for the benchmark. You can put as much speakers as you need, while your server support it :smile:

Oh wow <@U03Q1K8NXE3> that's really great to hear. Currently we are using Zoom. Our big rooms have ~700+ users in there (~60% camera off, 40% camera on). Are you saying this is possible to do with LiveKit as well? :smiley:

<@U042YHETGM7> how many folks are on-screen at one time?

Hi <@U01KZ0SCDKL> -- we're using Zoom at the moment so maximum is 25 on screen at one time (grid of 5x5)!

I would say is possible to handle that number, you can make stress tests using livekit-cli to double check. You will probably need pagination on the client side and a big server.

The checks work like this:
• Connecting to signal connection - checks if <https://docs.livekit.io/client-sdk-js/enums/RoomEvent.html#SignalConnected|SignalConnected> is emitted
• Establishing WebRTC connection - if Room.connect is successful
• Connect via TURN - attempts to connect with `iceTransportPolicy: relay` in RTCConfiguration (a <https://docs.livekit.io/client-sdk-js/interfaces/RoomConnectOptions.html|connect parameter>)

• Resuming connection - calls <https://github.com/livekit/client-sdk-js/blob/main/src/room/Room.ts#L412|simulateScenario> with `signal-reconnect`

are there any video tutorials for setting up server and using livekit like there are for agora

Thanks so much for the kind words, Jesse. :hearts: Knowing that folks are getting value out of LiveKit is really energizing for us! :zap: 

you would check <https://docs.livekit.io/client-sdk-js/classes/remoteparticipant.html#iscameraenabled|RemoteParticipant.isCameraEnabled>

Hey Jesse,

Which platform are you building for? muting a video track should turn off the camera indicator.

Though unpublish/republish should be supported too. Do you mind filing an issue for this under the repo where you are seeing this?

Hi <@U03S00K6BD5>,
to add to this, one thing to be aware of is that js client will assume that in case you provide mediastreamtracks yourself (via getMedia and `new LocalTrack`) you are in charge of releasing and reacquiring media streams. In contrast, if you create the track via `createLocalTracks` , the device release will be managed within the sdk for you.

This means on the other hand that muting a track will not release the device if you created a `LocalTrack` by yourself by default. One workaround would be to explicitly signal to the client sdk that you still want that track to be managed by the sdk, by passing in an optional constructor parameter `new LocalVideoTrack(mediastreamtrack, constraints, false)` that last parameter (which is true by default) will indicate whether the track will be treated as userProvided (i.e. unmanaged) within the sdk.

This is using the Web SDK.

Thanks, setting userProvided = false did solve the problem with camera indicator getting turned on and off properly. I'll put together a bug report about publish / unpublish / publish not appearing to work for cams.

Probably would be good to have some details in the SDK docs added to the LocalVideoTrack / LocalAudioTrack constructor explaining what userProvidedTrack does. I wouldn't have guessed this changes the mute / unmute behavior. On that note, it's a little unintuitive to call mute on a video track rather than enable / disable or something.

appreciate the feedback. we'll add a note about this in the docs.

you are totally right it's non-obvious to use mute on video tracks. We ultimately went with the same interfaces as audio tracks to be consistent

Here's an example that uses the server SDK for webhooks: <https://github.com/livekit/server-sdk-js/tree/main/examples/webhooks-http>

you could use the other server APIs in a similar fashion.

Currently I don't think there's a way to RTMP In, this said, If I remember correctly, its on the roadmap of the LiveKit Team. One way to work it out, from my test is to OBS out to the Virtual Cam, then use the browser to send the feed to livekit. Its a workaround but works.

Yeah, i guess this option will have to do for now, thanks for the tip. 

But platform support for RTMP sources would be awesome, and really round off livekit’s magical steaming powers!

:wave: Matt

We are working on universal ingress which will support RTMP (among other input formats)! It should be ready for folks to play with in the next few weeks :slightly_smiling_face:

Oh that is awesome news, and great timing. Thanks <@U01KZ61GV5Z> 

<@U01KZ61GV5Z> any updates on universal ingress? 
P.s Just saw the cloud announcement, and this looks perfect for my project, will be switching over next week :mostly_sunny:

yes! it's almost ready for a developer preview. updating the server SDKs at the moment :slightly_smiling_face:

I see that `IngressClient` is now in the docs :tada:
Is it available on the cloud service yet?

Hi Ray! We don't believe p2p is the way forward, especially with our goal of reliability and providing an end to end solution. There are other libraries mentioned in <https://github.com/livekit/client-sdk-js/issues/270|this issue> that might help.

with the Go SDK, `RoomCallback.OnParticipantConnected` will fire for every participant that joins the room. It doesn't need another RoomID parameter since you can handle that in your callback object

Hey Jesse, these are both good ideas. If you want to create issues <https://github.com/livekit/egress/issues|here>, you can track them and I’ll talk to the team about putting them in.

If you need solutions asap - we send the all recording info with the egress finished webhook, you could listen to that and upload the info to the same location. For generated names, if you leave `filepath` empty or if it ends with a `/` we will generate a filename using the room name or track id

Thanks, I added a few issues along with a third which is that in order for both of these to be useful, you really need the ability to request all the tracks to be saved as individual streams, rather than a composite (this is actually the only currently supported egress mode on twilio programmable video unless you use their newish livestreaming feature which has something similar to the chromium based compositor).

We did add the ability to record all tracks individually, which will be in the next livekit-server release (it’s a new option in the CreateRoom request)

:wave: Hi everyone!
I checked the unity demo following link (<https://github.com/livekit/client-unity-demo>).
But I don’t run this demo like as <http://unity.livekit.io|unity.livekit.io>.
what is the difference between <http://unity.livekit.io|unity.livekit.io>?

Screen Shot 2022-09-29 at 3.40.14 PM.png

Hey <@U045816UEBS>, our Unity SDK only supports WebGL at the moment, we are currently working on implementing native platforms.

You can't see the result directly in the Unity editor, you have to compile for WebGL to see the SDK working

yes, I built as webGL, but have same issue.

can you tell me about steps how can I see the result in webGL?
do you think need to add other source code or SDK build besides Github(<https://github.com/livekit/client-unity-demo>)?

Screen Shot 2022-10-02 at 9.09.13 AM.png

but fbx import error occurred and other errors.

Screen Shot 2022-10-02 at 9.12.13 AM.png

Screen Shot 2022-10-02 at 11.47.02 PM.png

<https://simmer.io/@PROCW000/test888>
I uploaded this app to <http://simmer.io|simmer.io>, can you check it?

I'm not sure about what is happening in your link

yeah, can you tell me about *demo-api, protocol* holder?
I just only open the *unity-project* in unity2021.

<https://github.com/livekit/client-sdk-unity-web/releases/tag/v1.3.0>
need to import the sdk additionally?

demo_api &amp; the protocol are used to generate join tokens.

<@U031GJTM8F2> Looks like a flutter package import issue, please try importing the logging package with the following settings

pubspec.yaml
```dependencies:
  logging: ^1.0.2```
lib/main.dart
```import 'package:logging/logging.dart';

void main() async {
  final format = DateFormat('HH:mm:ss');
  // configure logs for debugging
  Logger.root.level = Level.FINE;
  Logger.root.onRecord.listen((record) {
    print('${format.format(record.time)}: ${record.message}');
  });
...
}```


[ANDOIRD] I need help.  I can't hear and talk when App go to background mode

Hey Mark, at the moment, a specific room is hosted in the region where it was originally created. All participants connecting to it will connect to that region.

Understood. Thank you! I'll keep an eye on new released :slightly_smiling_face:

Are you able to share what you’re building?

Heh I can dream up quite a bit. I have quite a bit I'd like to build however I'm not sure public forum is the place to document that quite yet :slightly_smiling_face: . my team has been quite impressed with Pion/LiveKit however, and hope to contribute back in the near future :smile:

In this case I was asking in terms of geographical dispersed meetings :slightly_smiling_face:

<@U02EU2ZPTQU> has started to think about/explore a framework for LiveKit processors/plugins. He may have some thoughts here :slightly_smiling_face:

today, you could also perform 1 directly upon the MediaStreamTrack objects acquired on the browser side. LiveKit's <https://docs.livekit.io/guides/room/publish/#advanced-track-management|advanced publish APIs> can take in a MediaStreamTrack, so is compatible with the w3c spec.

I haven’t experimented with scenario 2, yet.
Could you share a bit more about your usecase and how you would imagine this to work? Are you imagining an additional hop in between client and sfu that only does a mediatrack transform?

<@U02EU2ZPTQU> That's correct, I imagine effects would be applied after the VideoFrame leaves the client browser and before the SFU performs its more advanced functionalities like adaptive stream, and dynamic broadcast. Possibly deployed, to reduce transcoding overhead, at the SFU.
Include relevant parameters in the participant's metadata, or within the track datachannel.
A few approaches that could exist:
• twirp based api extension
    ◦ Configure where your processor is running and have the api pass metadata and frames (audio chunks as well) to it
• webhooks based
    ◦ Similar to above, however I believe this is primarily used for client apis
• TrackBased
    ◦ Define an interface similar to mediacapturetransform for tracks.
    ◦ Deploy as part of a stripped down sfu or as a client track relay.

Twirp comm channel is likely going in the wrong direction. It and webhooks have the benefit of possible reduction of transcode overhead.
Track based seems more flexible to me but leads to more encode/decode.

Building as a plug-in may be reasonable but I am not familiar how to best do that in go/what headaches would come from multi language/platform support.

From what you describe, I think the easiest way to achieve something along those lines today would be to deploy your own TURN server and “misuse” it to modify packages.
<https://github.com/pion/turn/blob/master/examples/turn-server/add-software-attribute/main.go>

Thanks, that does look like a way to do it.

<https://www.w3.org/TR/mediacapture-transform/|mediacapture-transform >does also seem to be supported browser side (as an experimental feature in some browsers). Do you have an opinion on whether it should be included in the server sdks, or as some separate system, as well?

<@U02EU2ZPTQU> at that point, when the message is intercepted at the TURN server, the video frame would be encoded still right? I'm under the impression a lot of the functionality in the SFU, with respect to determining encoding type and state, would need to be re-implemented there.
Do you think intercepting at the sfu receiver layer might be easier?

yeah, I think generally it’s quite an undertaking to achieve what you described, especially if you want to manipulate frames on a pixel level.

I’m not an expert on this, but AFAIK SFUs generally don’t decode video frames, the architecture that does this kind of processing would be a MCU. And because a MCU also handles decoding and re-encoding, it is generally a lot more resource hungry than a SFU.

We do have plans to expose an easy to use interface for stream transforms on the client side, but there are currently no plans regarding deployable transform nodes in front of a LiveKit server. But it’s an interesting thought for sure!

The reason I suggested using a TURN server as a starting point was mainly  because at least the relaying part to the actual SFU wouldn’t have to be reimplemented, but I don’t think it’s a perfect solution either.

Great, thanks again for all of the feedback!

So multiple quality streams are published directly from the client. I think I missed that the SFU wasn't resizing to the various stream qualities itself. That would further increase the amount of processing that would need to happen at the mcu/sfu level.

exactly, that is if the publisher has `simulcast` enabled. <@U01KZ61GV5Z> has written a great blog post about this topic <https://blog.livekit.io/an-introduction-to-webrtc-simulcast-6c5f1f6402eb/>

Yeah, I think I knew this but somehow glossed over it.
I'm now considering whether the client could go through a Track (or MediastreamTrack) proxy. Ignoring considerations like node/region selection for now. I'm going to look at the Track code in more depth to see if that seems feasible.

`identity` should be unique (like a session id or some other uid), there is a separate `name` field which can encode a user’s display name.

i see that we note it <https://docs.livekit.io/guides/core-concepts/#participant|here>, but it likely bears repeating on the <https://docs.livekit.io/guides/access-tokens/|access token> page.

also the docs show a seperate command for load testing when its now a flag on the cli command :slightly_smiling_face:

ah, yeah, the CLI improvements are moving a bit faster than our docs. :man-facepalming:

this is always the tricky part with docs — our old docs used to be OSS, and getting the new ones in a public repo is still on our list of TODOs. Hopefully once those are out there, folks can submit PRs for updates we miss

Sounds like a hint to me :smile: Ill try to get some of that done this week. its the very least i can do

lol, no no, not a hint at all — just me trying to be transparent. :slightly_smiling_face:

very much appreciate you calling out the docs mismatch, thanks so much Mark! :pray:

haha I was only kidding. We've been sandboxing Janus, Jitsi etc - trying to decide what projects we want to go for first and which stack would be best suited.  Loving the project so far!

If you want, you can simply add a random string at the end of the identity.

yep! thats what we just did and it worked fine :slightly_smiling_face:

const at = new AccessToken('foo', 'bar', {
        identity: participantName+'_'+r
    });

Would love to hear what use case has 10k people. That’s amazing! :D

virtual events platform leveraging unreal engine and webrtc

:wave: Ryan, are you looking for 10k users on a single stream? or 10k users across multiple sessions?

The latter is quite easy to scale up with LiveKit, we've got at least a few folks using it in production with that type of scale.

To handle 10k users on a single stream, it would require a more involved orchestration + deployment scheme. Our upcoming cloud product might be a better fit for that type of use cases.

initial usecase would be 1 publisher and 10k subscribers

do you have an idea of timescale for when the cloud product is available?

I've just joined to the Community to ask the same question about scaling :)

One of the idea for scaling 1 to many web-rtc is to send signal from source to `origin` node which broadcasts it to many `edge` nodes. Do you have a plan or interest for moving in this direction with LiveKit?

<@U043EN6NPSB> Cloud is running in closed beta right now. If you have a moment, fill this out and <@U02BH3ZHJ0Y> will get in touch:

<https://livekit.typeform.com/cloud-beta|https://livekit.typeform.com/cloud-beta>

<@U04585JLQR0> it's definitely possible to scale by fanning out. It does get really complex to deploy + manage. You can bet that we intend to tackle the scale challenge though.