LiveKit Community

<https://github.com/livekit/client-sdk-js/blob/main/src/connectionHelper/ConnectionCheck.ts>

Bildschirmfoto 2023-01-18 um 19.22.59.png

btw. in my local tests (with a local server), the “medium” and “high” quality produced in the same video width and height.

high: 1280x720
medium: 1280x720
low: 320x180

I don’t really know how to interpret this now, or what it means.

Hi Dennis! Looks like a pretty cool app you guys have there!
To answer your questions:
1. yup, adaptive stream requires simulcast (enabled by default)
2. that's correct, adaptiveStream does not currently return dimensions post-transformation. I did a bit of research here, and it seems that using <https://developer.mozilla.org/en-US/docs/Web/API/Element/getBoundingClientRect|getBoundingClientRect> would take transformation into account. so that might be a solution we would pursue!
3. these corresponds to different layers. Layers are determined by the publisher, so from the subscriber side, it'd be best to use `setVideoDimensions` . Our SFU would look through the layers from the publisher side and send back the one that matches dimensions the best.

Thanks a lot and thanks to you it will be even cooler  :slightly_smiling_face: :rocket:

2.1 what do you mean by “adaptiveStream does not currently RETURN dimensions post-transformation” ? You mean that adaptiveStream can’t handle css “transform” so css based “dimensions post-transformation”, right?
2.2 And what do you mean by “so that might be a solution we would pursue”. Are you saying that you are considering implementing this on your end to make it work? If so, I think that will be a problem, because the ResizeObserver just doesn’t give you an event. So you have no trigger to react, see <https://jsfiddle.net/doc95s6j/15/>. I think I have to do it in our application, because I know when the client has zoomed.

3. ok, thanks. I will then always use `setVideoDimensions` after a zoom in our application. I have two more questions about this :
3.1 Can a publisher influence the layers it publishes? Or is it done automatically by you in a way?
3.2 Can you try to explain to me what the setting VideoQuality does? Does it set a defined dimension and fps or how are these 3 values related?

When we have LiveKit integrated, I can invite you to a test session if you want.
Then you can see what we do and how it works.

2. ah ha, I misunderstood you initially. yes you are right we won't be able to pick up the resized events. though we may be able to offer an interface for users to notify us of the change.

3. you can customize the simulcast layers. docs on that <https://docs.livekit.io/client/publish/#advanced-track-management|here>.

Setting video quality would influence the layers the subscriber would receive (0, 1, 2). However, that's focusing on mechanics over the desired outcome. We will track the layers internally, but mapping that to dimensions for the subscriber.

For sure! let me know when it's integrated, would love to try it

I'm curious, what are you guys using for video today? (and how did you hear about us?)

2. Ah okay, yes that is indeed a possibility. Thanks!
But I think putting the VideoDimensions on each TrackPublication after zooming is kind of the same and/or just as good first.

3. All right, got it!

We are currently using LiveSwitch, but I’m not that satisfied. If you like, happy to talk more in a private chat.
I did a lot of googling and reading up and that’s how I came across you guys :slightly_smiling_face:

True, that's essentially the same thing :slightly_smiling_face:

Thanks for sharing, I'll definitely hit you up and would love to hear your experience with the switch.

Hey Rahul, it’s difficult to say what the cost difference would be primarily due to the unknown staffing/operational costs of maintaining 99.99% uptime with your self-hosted LiveKit instance.

We operate LiveKit Cloud cost-efficiently and pass along these savings (via economies of scale) to you.

Out of curiosity, do you already have a product in the market doing 10K concurrent broadcasters?

I run a company called <http://www.appscrip.com|www.appscrip.com> , where we have a bunch of products that have an integrated live stream , so we have clients hitting reasonably good scale there , so we have built out SDK's on our end now that can power their streams and we were exploring moving our engine over from AWS for RTMP and Agora for WebRTC into this .  So we do have an operations team that can manage infra etc on our end but we were wondering if we should do it in a way where we first try the livekit cloud with certain clients and if its working right we thinking of running this on our own infra , and hence I was evaluating the cost differential between self hosted infra and the cloud infra that Livekit already offers.

Ah I see! Thanks for sharing that additional info. You can definitely run on Cloud to start and switch to self-hosted any time. There’s no API/code changes between the two, just different websocket URL and API keys.

yeah module replacement should work ok.

I'm not sure if it'll be an issue here. I know several companies that use livekit and/or pion in products that also serve the government. We have not heard that come up as an issue yet.

For now this is a “what if” scenario, but that is good  to know

Are you looking to playback your own audio? Or only to analyze captured audio? I think echo will be a major issue in playing back your own audio.

However, if you would to attach the track locally, it's possible by calling `.Attach()` just like a remote audio track

I'm interested in playback of local audio for testing microphone quality. Can you subscribe to your own track?

you do not need to subscribe to it, (it's already available locally). Calling `Attach` on the track like mentioned above would start the playback of that audio track

Fired up :fire:!! Congrats on this launch!! This is looking really fantastic! :partying_face::star-struck:

This looks great! I've been building LiveKit components very much like this for Vue, taking inspiration from the excellent <https://headlessui.com/|Headless UI> by the Tailwind team. So far ours are not public but we're considering opening the source after we get our product out of the door :slightly_smiling_face:

<@U04D8MUUX6U> that sounds very interesting!
Would be great to get your feedback on the `components-core` package as that’s designed to make component package implementations with other frameworks a lot more straight forward!

For vue our idea was to (eventually) leverage composables and `provide()` in a similar way to how we are using hooks and context for react today!

<@U01KZ0SCDKL> cool stuff. Any plan for Vue release too once React’s is stable?

Hello. Screen share does not work. In Chrome it does nothing. In Safari it asks for permission to share and then nothing happens.

Question: Will it work in a self-hosted LiveKit or just in the cloud?

Hmm, where are you seeing screenshare not work <@U03M86BE4H0>? With Meet or when you’re playing with components? I did a screenshare in a call yesterday (from Chrome) using Meet, and it was fine. cc <@U02EU2ZPTQU> 

<@U04A1328AJY> yup, Vue and Svelte are planned after we’ve got the core APIs set (in React)!

<@U03M86BE4H0> it all will work in self-hosted LiveKit. :blush:  

<@U01KZ0SCDKL> Correction: It works in Google Chrome! Two computers. Connected to different networks. Thanks

Love it - Think of a solution involving video meetings. I can see a direct application. But I need a vanilla JS solution. Actually, I am using Alpine.js with Laravel. But it is like vanilla JS. So a vanilla JS SDK would be most appreciated.

<@U03M86BE4H0> I believe support for <https://developer.mozilla.org/en-US/docs/Web/Web_Components|Web Components> is also planned.

Great. I believe this will fit my timeline. Working on the skeleton into which the video meetings will figure.

wrt Web Components: We hope that a Web Components implementation could be supported as a byproduct of some other framework implementation (e.g. solidjs and svelte allow for web components as a compilation target). Even though this works in theory, we’ll need to verify that this is possible without compromising on the general developer experience around LiveKit components. We also outlined our current thinking around web component support in the repo <https://github.com/livekit/components-js#did-you-consider-using-custom-elements|readme>.
<@U03M86BE4H0> it should also be rather straight forward to embed _any_ framework implementation within your vanilla-js project and load it basically as a “widget” if your use case allows to treat the video conference part as an independent module of your website/app.

I would prefer to look at source code and make it in vanilla, rather than embedding a React. 

Embedding give it an IFRAME flavor that complicates the whole. 

The best would be a vanilla JS. Not even web components. I do not know how web components will fit. But probably better than embedding React.

I really like the way <https://github.com/livekit/components-js/tree/main/packages/core> is using Observables! Very similar the the wrapper I put around the `client-sdk` :+1:

In general, I believe many companies today start with vanilla JS SDK, and build any framework SDK on top of it, as a wrapper. As <@U046B0LNLD7> point to on the core. If I got it right.

<@U03M86BE4H0> I think conceptually, you can think of <https://github.com/livekit/client-sdk-js|https://github.com/livekit/client-sdk-js> as the vanilla JS SDK. Components is a higher-level abstraction which sits on top and makes it simpler to build common calling and live streaming experiences. As a developer, you’re always free to drop into the core layer if you need direct access to the raw JS primitives. AFAIK, reactive UI/bindings and state management are not well-defined or standardized in plain JS (outside of WebComponents).

Got it. Thanks . This completes the picture for me. This is useful guidance. 

<!channel> :point_up: this is a big one for us!

haha, probably not until 1.0 — thank you so much Max!! :heart:

really awesome what you are doing with this project

1. Yes, you can fix the video dimensions, those are determined by your app
2. Livekit does not provide this but you can add a watermark yourself
3. (same, you can add your own dynamic watermark)
4. “Admin status” as such is not built in to the LiveKit SDK, but your application can mute/unmute a given track using the server SDKs, as seen here in our <https://docs.livekit.io/server-sdk-js/modules.html#ParticipantPermission|server JS SDK> (not sure what you’re using on the server side)
If you have any more questions, please post them in the <#C01KVTJH6BX|> channel, thanks :pray:

when it was first developed, it was left default off by default since it was a new feature. We typically let users opt into new features to avoid any unexpected behavior.

it's gotten a lot more robust since then, but we wouldn't want to change the defaults until the next major version. It probably makes sense to enable both adaptive stream and dynacast by default then.

Hey everyone :wave:
 
Did somebody implement successfully blurred backgrounds using livekit? If so, how and what did you use?

Are their any new information besides the following conversation using the service “mediapipe” for chromium based browsers:

<https://livekit-users.slack.com/archives/C01KVTJH6BX/p1665737702532479?thread_ts=1665730213.377889&amp;channel=C01KVTJH6BX&amp;message_ts=1665737702.532479|https://livekit-users.slack.com/archives/C01KVTJH6BX/p1665737702532479?thread_ts=1665730213.377889&amp;channel=C01KVTJH6BX&amp;message_ts=1665737702.532479>

<https://google.github.io/mediapipe/solutions/selfie_segmentation|https://google.github.io/mediapipe/solutions/selfie_segmentation>

Hey <@U0491G6RB5F> I’m working on a spatial example app using the new components-js library (<https://github.com/livekit/components-js>)

The gist of how you do this:
• Subscribe to a participant’s audio tracks. Instead attaching the track directly to an audio element, you’ll use WebAudio’s `MediaStreamAudioSourceNode`  (<https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamAudioSourceNode/mediaStream>)
• Attach that source node to a `PannerNode` (<https://developer.mozilla.org/en-US/docs/Web/API/PannerNode>)
• Using data channels (or your transport of choice) - tell other participants the position (x,y,z) and orientation (x,y,z) of yourself
• Use those values to update the PannerNode’s values
:warning: WebAudio can be finicky sometimes. You may need to attach tracks to muted `&lt;audio&gt;` elements to get the audio to process

Thank you <@U04F14MLUJE> :slightly_smiling_face: 

I will try it out and let you know if I was able to make it happen.

You should be able to run the server and clients on your local network. (This is how we develop LiveKit.) But it’s difficult to know what your problem is here without knowing more details. Are you certain your client can reach the server? (you could test this with ping)

her is what i do
• download the server and run it in my machine
• create room and token for two participates
• run client appliction in the same machine the url given to the client is "localhost:7880"
• the two user connecting successfully and start streaming
• disable the internet access (turn off wifi )    and refresh the client so it will reconnect with the server it give me this massage "error could not connect PeerConnection after timeout"
• if i turn on wifi it will connect and start streaming 
• if i turn off wifi during streaming the streaming will not stop

Ah, okay. Currently the way the client/server connection doesn’t actually work over the loopback interface, it only works if you’re actually connected to a switch.  I forget the details of that, but basically, LiveKit will not work if your computer’s wifi is off, but it WILL work if your wifi access point is on but not connected to the Internet.

Will that work for you? What are you trying to do with LiveKit without being connected to the Internet?

I think it should work on loopback.. but you'd have to pass in --node-ip while starting the server to ensure the server can advertise the correct address to clients.

try launching the server with `livekit-server --dev --node-ip 127.0.0.1`

Hey Michal, we haven't benchmarked it that yet. While I'm curious on the end to end timing, it's not really something we are optimizing for. IMHO anything that's &lt;1ms and can perform consistently without introducing additional jitter would meet our design goals.

Initially I thought that ~1ms is pretty long but actually it would allow to handle about 1000 packets/s. Given that 1.5Mbps produces about 150 packets/s, there is still much room for higher bitrates. Thank you!

I see what you mean, that makes sense. Though throughput is what you'd be optimizing for, instead of end-to-end forward latency.

You'd share the CPU core with other forwarding work. (not to mention being able to leverage multiple cores).

During some throughput testing, LiveKit should be able to do more than 1Gbps on a decently sized machine (8 or 16 cores).

By 1Gbps you mean overall throughput (e.g. 10 publishers, each streaming 100Mbps and one subscriber) or for the one participant (one publisher and one subscriber)?

overall throughput. either meetings (lots of publishers and subscribers) or livestreaming (single publisher, lots of subscribers)

Unfortunately we don’t have that as a feature inside of LiveKit yet :disappointed:

hi <@U04MSQT7KBJ>
that’s a restriction of iOS/Safari. unfortunately there’s nothing we can do about this from the livekit side.
if screen sharing on mobile is important to your app, you would have to use one of the sdks that can be installed as a native app (e.g. swift/android sdk or flutter).
here’s an overview of the underlying API for screen sharing and its browser support: <https://caniuse.com/mdn-api_mediadevices_getdisplaymedia>

HI android mobile also have the same issue.

yeah, exactly. if you check from the link above you can see that mobile browsers currently do not have the ability to screen share

Hey <@U04MSQT7KBJ> are you using https to serve your application? Because typically mobile browsers block access to audio and video on non https website.

Haven't tried livekit on mobile yet so I am not entirely sure about the support, but this could solve your problem.

Huh, we don’t have this built-in, but I like the idea a lot!

For what it’s worth, you could implement this yourself using our server-side SDKs, listening for room data events. You’ve probably already seen this section of the docs: <https://docs.livekit.io/client/data-messages/#data-messages>

We’ll keep this idea in mind for the future, though!

actually, I should ask: how exactly do you want the room’s real-time chat to be included in the recording?

If it’s ok for the chat to be rendered in the video, you could use Room Composite recording and have the Room Composite rendered video include the rendered chat messages

Thank you for the quick response. Listening for room data events seems very doable with the server sdk, I'm leaning towards that.

I am also considering just including the chat in the video but I gotta check with my teammates. I appreciate the help very much and I hope you do end up implementing this functionality some day!

Great! Also I have been reminded by my teammates that this is in fact on the roadmap, but we won’t get to it for several months at least

Hm unfortunately our only fully-featured iOS example app is <https://github.com/livekit/client-example-swift|a video chat app>. We’re currently working on more examples, though, including a live streaming example.

What are you imagining — an app like Instagram Live or Periscope, that records from one phone and broadcasts to many viewer phones? Or do you have something different you want to build, and a streaming app is just the closest existing thing?

<@U02MC5V3C2H> something like IG LIVE, yes

Hey <@U04MUK57FC1>, yes, this is absolutely possible. You can rig this together pretty quickly using the new LK Components.

<https://livekit-users.slack.com/archives/C01KZ61HZRR/p1674158758259689|https://livekit-users.slack.com/archives/C01KZ61HZRR/p1674158758259689>

&gt;  I don't want the the PARTICIPANTS camera to switch on or be visible in the stream, is this possible using livekit?
yup, you can set `canPublish: false` in permissions for subscribers. <https://docs.livekit.io/concepts/authentication/#example:-subscribe-only-token|docs here>

Thanks for the prompt reply, I'll take a look at those suggestions

image.png

Hi, where exactly are you seeing this error? are you using LiveKit Cloud or is this self-hosted instance?

Just to make sure I understand: you’re using the JS server sdk, and when you run your server on your local machine you successfully get a token, but when you run your server inside a docker container you get an Invalid Token error? Are you certain that your container’s clock is set correctly?

both VM and container are in UTC time zone

here is the sample code <https://github.com/kmamtora/livekitstest> using js server sdk

it works in both browser if the app was created using cra, but fails in firefox if it is created in vitejs

Hi! 
could you provide a reproduction for this so that we can take a look?
also have a look at the new react components (and hooks): <https://github.com/livekit/components-js|https://github.com/livekit/components-js> 

<@U02EU2ZPTQU> I opened an issue just now with an example and reproduction steps:
<https://github.com/livekit/livekit-react/issues/120>

thanks! strange, the error does not sound related to the hooks. can you post the results of <https://livekit.io/connection-test> when run from your firefox instance?

also: is the error only present in dev mode or also in a production build?

Didn't test reproduction but it only present in vitejs

`Connecting to signal connection via WebSocket`
`Connected to server, version 1.3.2.`
`Warning: Server is insecure, clients may block connections to it`

`Establishing WebRTC connection`

`SKIPPED: Can connect via TURN`
`Warning: No TURN servers configured.`

`Can publish audio`
`published 143 audio packets`

`Can publish video`
`published 261 video packets`

`Resuming connection after interruption`
`Pass`

<@U02EU2ZPTQU> Just tested production build of vite project and it fails just the same

<@U02EU2ZPTQU> I is very likely that the issue is not in the component/hooks themselves but in the under the hood  livekit client sdk

which versions are you using for vite, livekit-client, livekit-server, firefox ?

@livekit/react-core: 1.1.0
react: 18.2.0
react-dom: 18.2.0
livekit-client: 1.6.4
livekit-server:1.3.2
firefox: 109.0
OS: macos on MacBook air 2020 m1

If i set
`{`
          `rtcConfig: { iceTransportPolicy: "relay" },`
`}`

in the cra app it fails as well in firefox

But if i set the opposite in the vite app (i.e `iceTransportPolicy: "all"` )
it still fails

So i just know how to fail the app in cra and not solve in vite :slightly_smiling_face:

So did some diggint, and it seems it is caused by the livekit client and not the react hooks themselves which I suspected from the start

the following is a vanilla typescrip example to reproduce
`import { Room } from "livekit-client";`
`import { setLogLevel } from "livekit-client";`


`export async function connect() {`
  `setLogLevel("debug");`

  `const room = new Room();`
  `await room.connect('<ws://localhost:7880>', '{YoutTokenHere}');`

`}`

`document.querySelector&lt;HTMLDivElement&gt;('#app')!.innerHTML = ``
  `&lt;div&gt;`
    `&lt;button id='connect'&gt;connect&lt;/button&gt;`
  `&lt;/div&gt;`
```
`document.querySelector&lt;HTMLButtonElement&gt;('#connect')!.addEventListener('click', () =&gt; connect())`

Another interesting fact: If i build the app for production it also fails when run in `vite preview`

But works in some other server like `apache2`

which is strange since webrtc runs entirely in the browser so why does a server which is only used to serve the files is interfering

hm. that’s strange indeed. maybe previewing vite with ssl does the trick on firefox? <https://github.com/vitejs/vite-plugin-basic-ssl>

Even if ssl is the solution,why does it work with CRA and apache without ssl?

It's failing because you did not configure TURN:

```Warning: No TURN servers configured.```

<https://docs.livekit.io/oss/deployment/#improving-connectivity-with-turn>

<@U01KZ61GV5Z> Well, the server is running on localhost on the machine when I run my test, so I shouldn't need a turn server to start with,also given that it works fine in chrome and fails in firefox it means it is not really a connectivity issue

That's correct if you are trying to test localhost. The reason why it fails with `rtcConfig: { iceTransportPolicy: "relay" }` is because `relay` means to use a TURN server (which you have not set up).

How are you running your server? and what other web servers is between your client and livekit-server?