LiveKit Community

this isn't possible today.. I believe if participant disappears (without clean disconnect), it'll take 15s for them to be removed from the server.

Hi, i was curious about scalability of LiveKit.
It can  support 3000 users in one to many scenario?

are you using Track.attach? adaptive stream requires attaching to tracks instead of using .mediaStreamTrack directly.

otherwise it will not function, and you should leave it off

Yep. If i back to livekit tab videoElement became visible and video starts. But i need it works without switching to livekit tab. AdaptiveStream:false works fine. But it will be heavy for users. Maybe i could emit track somehow. Also i try on TrackEvent.unmuted .detach and .attach track. It's not help)

ah I see what you mean.. that's probably due to <https://github.com/livekit/client-sdk-js/blob/c610860570bec3c90bebe6d8101334790afa7432/src/room/track/RemoteVideoTrack.ts#L192|this check>. When your app is backgrounded, remote video are invisible and adaptive stream will temporarily pause them.

are you saying that you are not getting the `TrackSubscribed` event while backgrounded?

which platform are you running on? is this web or mobile?

I have already subscribed on track. Getting TrackUnmuted as normal. But it's not start because is not visible at that moment. I think it's not issue and normal at adaptiveStream:true. Video is not visible and not streaming))) But i need start it. It's client-sdk-js

Other videoTracks from other clients works fine in background if they already unmuted

But if remoteTrack mute and unmute then videoTrack not starting until i open the tab with livekit client

I'm still not understanding what you mean. can you list out the steps to reproduce and what is the desired behavior?

1. Connect to room with adaptiveStream: true
2. Subscribe to all participants
3. Attach all track by.attach
4. Click on another tab in my browser
5. Some of participant turn off and turn on camera  by method setCameraEnabled
I expect that videoTrack start streaming and not waiting until i returned to the tab with livekit.

which browser are you using? just checking on this behavior

ok I think the current behavior is working as designed. Are you asking for the option to not pause videos when the tab is backgrounded (but elements visible on that page) ?

Yes. Or maybe, how can i do it at current version, if it posible

it's not possible today. adaptive stream pauses all invisible videos, including from background tabs.. but I think you raise a good question if that should be the behavior.

I have specific task. I use canvas to output streams to video element and open it in pictureInPicture. In background tracks works fine with adaptiveStream until someone turn off- on camera. Disabling adaptive streaming increase CPU work. Thats why i ask to add some option to show streaming after it will be visible after adding to scene or maybe how can i trigger start stream manual.

understood.. that's a good point, we'll discuss this with the team.

Good day. Have you already disscuss this with the team?

hey yes, we did chat about this and would like to make it work for this use case. However, it's still not clear how we could reproduce it.. if possible, could you fork and modify the <https://github.com/livekit/client-sdk-js/tree/main/example|sample app> to demonstrate how to get it to happen?

your server had restarted, and looks like Flutter client isn't able to reconnect during this sequence. The correct behavior is to initiate a full reconnect; so this is likely a bug

thanks, where should I look through in server code

oh I see, client need to start whole new session right

oh, it seems it was ice not connected, so can we fix it by `iceRestart: false` ?

Lol! If LiveKit was really being used for this, I could die happy. :joy: 

<@U01KZ0SCDKL> dream big bro, lol :smile:

These are for static video files (or HLS), so they wouldn't work with LiveKit. Are there any specific features of wistia that you are looking for?

hmmm, just the aesthetics of the player, the buttons for mute, changing quality… etc i know you can’t change speed for live video but does the rest make sense?

the answer is "it depends" :slightly_smiling_face: how many users is each subscribing to? and what is the publishing bitrate. check out <https://docs.livekit.io/deploy/benchmark/|the docs> for the framework for thinking about it.

I think you can try a 8 core server for ~800mbps of throughput.

Hello I can’t reproduce the issue, can you try `flutter pub upgrade` to make sure dependencies are latest.

Also can you paste the logs ?

Thanks for the fast response, Hiroshi  :smile: It was actually my bad, I was sending data in invalid format.

Sounds good, I’ll try to make the error message more informative.

there is `0.18.6`  now, better upgrade to the latest right?

Hi there, there’s an egress service that’s coming soon. That will serve all LiveKit recording needs, including uploading to S3 and such. Stay tuned!

it's a new capability we are launching next week :slightly_smiling_face: think of it as giving you all sorts of ways of exporting your tracks.

by the way, would you consider open sourcing your python sdk? we'd love to link to it. look like someone already <https://github.com/tradablebits/livekit-server-sdk-python|published one here>.

sure! our implementation is only a day old but is pure python and doesnt require go as a dependency. would you mind if i dm’d you to ask a few more q’s?

At the risk of becoming a fanboy, I'll make a Twitter account to boost.

I've been working hard since June of last year, when I really started self-learning full stack development to build a video conference app. Muddled through Amazon Chime, Jitsi, and OpenVidu. All of them were a pain to self-teach and work with for various reasons.

Between culmination of experience and LK's straightforward setup, it's clicking now. Got a stateful app with participant and room metadata updates taking shape. Never got this far before.

So, thank you for writing the library in Typescript. Thank you for publishing a functional React library that isn't 3+ years old. Thank you for using plain English and not ambiguous words like prosody or jingle. Thanks because, legit, it's happening. There's plenty of work and challenges ahead for my project and implementations, but I appreciate this opportunity.

<@U03D1HEGHSN> this message made my day. Thank you for sharing more about the journey you’ve been on. I’m excited for continued progress with your project, and please let us know if there’s ever anything we can improve or help with along the way.

WebCodecs + WebTransport could be an alternative to WebRTC for getting media to browsers. it gives you finer grained control over the entire pipeline. though my guess is it'll take at least 3 more years before it reaches the point of maturity that WebRTC has.

i guess SIMD + WASM falls under same umbrella <https://youtu.be/5b4YcLB4DVI?list=PLNYkxOF6rcIDKuCU73tmdRN_-mI3tKFPD&amp;t=656>

SIMD + WASM might be sooner. Amazon Chime noise suppression called Voice Focus already uses it. I think Chrome had it as experimental since version 84 or so. I am guessing it is in the mainline now.

Happy Launch Week everyone! :tada: :tada: :tada:

Day 1 announcement: Unity WebGL SDK

<https://twitter.com/livekitted/status/1526252482778738691>

image.png

cc: <@U01KZ0SCDKL> :point_up::skin-tone-4:

ok it’s resolved now — thanks <@U02G87YSENA>!

Screen Shot 2022-05-16 at 5.55.16 PM.png

I'm good dude :) looks like y'all are making big moves!

Screen Shot 2022-05-17 at 7.07.51 PM.png

Congratulations <@U01KZ61GV5Z> and livekit team. This will help many organization.

Congrats.  Network congestion is the hardest problem to solve.  Great work livekit team

I'm not totally familiar with recording and <@U01KZ61GV5Z> should be able to help you better with this, but I know the team are working to release a new egress service which might be easier for you. Im sure one of the team members will be able to provide more info :slightly_smiling_face:

appreciate it so much, it means a lot to me

:tada: :tada: thank you <@U02SGSET1RT> !!

Cool, what happens with timeouts and reconnects? :thinking_face: . Assuming configurable?

If it times out on the initial connection it'll fail (I haven't seen this happen except with a bad URL), but if it times out mid-stream it will continue retrying. Timeouts/retries are not currently configurable

I'm excited about this! Does the egress service have any concept of region? It didn't look like it from an initial glance. This would be pretty necessary in a multi-region cluster to avoid track data going across the globe if a close egress server isn't picked.

Also, in region traffic is often cheaper or free on cloud providers :slightly_smiling_face:

It doesn't currently, for now in our own multi-region setup we only have it deployed to a single region. Definitely something we'll be thinking about though

<@U01UB0LHTTQ> Gotcha. I'd say the logic that the livekit-server uses for node selection would be good for this as well. The egress servers would just need region specification and the same math used to find close servers could be used for egress selection. I'll go ahead and open an issue for the feature request :slightly_smiling_face:

there's always room for improvement :slightly_smiling_face:

Yeah! I think it's a great feature. I'm really hoping to get some time soon to play with it. I had built live transcription into my Jitsi based FoundryVTT module previously, and had done code to make Jitsi use Azure's speech to text platform (which oddly worked much better than google's). When I get time, I'll try porting both of those things over for this :slightly_smiling_face:

amazing, I just asked about recorder yesterday, what a beautiful day

`Each Egress instance is a worker that observes Redis for incoming requests. A worker may decide to fulfill a request based on its current load. For example, track egress doesn't involve transcoding and is consequently lighter-weight.` that's exactly what I'm looking for, can't wait, I'm trying it right now

this is really interesting, does this support mixing streams and output 1 as an MCU?

yeah! it's sort of like an MCU in that way.. except the work isn't done by the single server, so it's easier to scale.

hi, <https://docs.livekit.io/deploy/egress/> it seems there is only doc for deployment in docker way, which asks `livekit-server must be run using --node-ip set to the above IP` , the IP as Docker sees it, for instance `172.17.0.1` , but at the same time `node-ip`  for `livekit-server` should be the address as public client sees it, ends up with conflict, did I missed something

I think we will not face such issue, if we deploy it in native way. without docker

I don’t think that’s recommended. You’ll need to install a bunch of libraries such as GStreamer and headless Chrome and ensure their compatibilities. Afaik, the egress can run without being exposed to the public (it should run in the background) if you run it in service mode via Redis. I’ll defer to <@U01UB0LHTTQ>  and <@U01KZ61GV5Z> for advice.

alright, I think I have made some misunderstanding, let me check it again

It's totally fine to run livekit-server locally, and without node IP.. that comment is out of date, so I'll take it out.

Egress itself should be ran from within Docker, mainly due to the number of GStreamer dependencies and that Room composite requires Chrome/X11/Pulse.

Great work! I deployed in our environment, wonderful experience.

Thanks for the callout, it’s been a real joy to get involved and I’m planning to stick around. This welcoming community is the first reason for me to contribute, the second being it is a game-changing project :fire:

Thanks for the callout :slightly_smiling_face: I didn't realize that I was the first one using a multi-region deployment :joy:

And thanks for the project, it's been great to work with :blue_heart:

Thanks everyone, this is an amazing community, thanks <@U01KZ0SCDKL> <@U01KZ61GV5Z> <@U01UB0LHTTQ> and the rest of folks. thanks a lot <@U02SGSET1RT> for the egress, it’s been a tremendous help for us :pray:

I wanna say the Flutter SDK works on web…? Maybe I'm totally wrong. <@U01KZ61GV5Z> 

Livekit -&gt; Flutter -&gt; canvas -&gt; web -&gt; :exploding_head:

yeah it does! Supported features are documented <https://github.com/livekit/client-sdk-flutter#current-supported-features|here>

Nice everything is supported, now imagine all room activities happens on the pixel level on a canvas not on html elements :slightly_smiling_face:

Was fun! Thanks for sharing it out :tada:

hey Jon, we'd like to have one eventually, but unsure of exact timeframe.

I'm curious, what is your use case with a Rust client? Are you looking for publish or receive media

Both. We’re using LiveKit for our WebRTC infrastructure for our <https://www.littlecinema.net/|digital experiential platform>. We’re looking to solve two main goals:

1. Have a way to easily pull in remote participants into broadcasting pipelines. We currently use a custom webpage that a broadcast tech will screen capture. This requires a seperate computer for each remote participant for isolating video and audio. We’d like to build a way that allows us to send each participant’s A/V to an SDI output on a PCI-E card.
2. Have a way to easily connect the output of our broadcast video pipeline to our WebRTC infrastructure for streaming. We currently use a Decklink -&gt; OBS -&gt; OBS Virtual Camera -&gt; Web pipeline, but we’d like a plug-and-play solution with dedicated hardware.
We’ve been currently experimenting with <https://github.com/webrtc-rs/webrtc>. Given it’s also based on Pion’s WebRTC stack, perhaps there’s a way to integrate?

(Also, want to take the opportunity to thank you for the work on LiveKit, it was a game changer for our webrtc stack.)

You can have a look at livekit egress repo for your use case livekit/egress: Export and recording of WebRTC sessions and tracks (<http://github.com|github.com><https://github.com/livekit/egress|)>

Maybe wrapping libwebrtc with rust is an option, this project compiles google webrtc into a separate dll, only needs to wrap it into rust, and then use the rust version of livekit signaling, I think this will be a viable production solution.
<https://github.com/webrtc-sdk/libwebrtc>

Another  solution is to use webrtc-rs + rust gstreamer, I think it is also feasible, because pion + gstreamer already has a case

Both ways should be able to get raw data from audio/video track, then you can send raw data to mixer, compositor, filter

<@U036TMWNHM5> that's an incredible use case! thank you for sharing!

For the first use case, as Jie mentioned above, egress sounds like it could help with this. What do you think?

For the second use case, we are actually in the middle of designing an ingress service that'll let you pull in any stream from existing sources (including OBS/RTMP)

I need to spend some time digging into the details of the universal egress, but it seems promising.

If you can add RTP to that rather than just RTMP that’d be a big unlock for us and anyone else working in big broadcast infrastructures.

<@U01KZ61GV5Z> what is Ingress Service? I already see references to in the master branch, and it looks that we are going to have new service which is able to accept RTMP stream, and put it into WebRTC participant?

Any place where it is possible to read about it more? Find a beta version, etc?

<@U03UHEVPUHW> As you have guessed, Ingress is for ingesting non-livekit streams/files, including RTMP. You can take a look at the APIs we've planned: <https://github.com/livekit/protocol/blob/main/livekit_ingress.proto>

we are working on it at the moment. if you'd like an early look, could you tell me a bit about your use case?

(feel free to DM) :slightly_smiling_face:

Hi Guillaume!  We plan to have a full Unity client at some point in the future, but it’s not on our immediate roadmap.

What’s your use case? Are you trying to add voice/video chat to a game?

It's not a game, it's an industrial application for professional a bit like a Remote Assistance app. I work for a big company, do you offer commercial services (maybe that could fast forward the full Unity client) ?
Thanks.

Hey <@U03GJ2984G5>, which targets does your application run on? I’ll DM you and we can chat more.

Unity and we target Android, Windows Desktop and WebGL

+1 for me.  We're looking at a native application for AR/VR use.

+1 from us also. Looking to do a native app with features we can't do on webgl.

+1 for Windows desktop so I can test multiplayer from the Unity editor instead of building for WebGL.

Is it impossible to test webgl from Unity editor ?

+1 I’m new in the community. I’m working with Unity for AR/VR under android, ios and Microsoft Hololens

we are not handling node to node relays at the moment. it'll be part of a future offering though!

All audio output is currently stereo. Multi-channel audio should be possible for the other egress types (track and track composite), but they don’t support it currently