LiveKit Community

CC: <@U02EU2ZPTQU> <@U03RM7G8Q6A>

I think the right behavior is if the Room isn't connected, we should catch this and ignore. wdyt?

Sounds reasonable to me. Do you see any problems with this approach, <@U02EU2ZPTQU>?

<https://github.com/livekit/components-js/pull/501|PR>

I had one other related question on documentation: the `LiveKitRoom` component says the `defaultValue` for `audio` and `video` is `true` <https://github.com/livekit/components-js/blob/main/packages/react/src/components/LiveKitRoom.tsx#L32|here> and <https://github.com/livekit/components-js/blob/main/packages/react/src/components/LiveKitRoom.tsx#L38|here> respectively.

But the `defaultRoomProps` has both of those defaulting to `false` <https://github.com/livekit/components-js/blob/main/packages/react/src/components/LiveKitRoom.tsx#L90-L91|here>. Working with the component, they seem to be defaulting to false, is there something else that should be defaulting those values to true?

thanks! the docs are wrong! defaults are false, we’ll fix that

Hi. I want to refresh this issue since it's becoming a problem again. We've fixed the problem for iOS devices by migrating music player to backed (ingress or bot participant with custom audio), but now we want to notify participants about some events during the call with some sounds and the problem arised once again. Can someone look into this problem and propose a solution or a workaround?
<@U01KZ0SCDKL> <@U01KZ61GV5Z> <@U02GAC58A79>

Can you try to put in a slight delay before sending that message? I'm wondering if it's some sort of timing issue of some sort.

I added 1 second sleep at first line of  OnParticipantConnected function and now it become slightly better but after for example 3-4 user connection still it does not send the data that Im trying to send in that callback
It seems a little deadlock in livekit to me :thinking_face:

can you share the code you are using? LiveKit regularly handles 10k+ participant rooms (in Cloud). it's highly unlikely that you would hit a deadlock with 3-4 users...

`var RoomClient client`

`type client struct {`
    `*lksdk.RoomServiceClient`
`}`

`func InitLiveKit() {`
    `RoomClient = client{lksdk.NewRoomServiceClient(fmt.Sprintf("<http://%s:%s>", os.Getenv("LIVEKIT_HOST"), os.Getenv("LIVEKIT_PORT")), os.Getenv("LIVEKIT_API_KEY"), os.Getenv("LIVEKIT_SECRET_KEY"))}`
`}`

`func (c *client) ConnectToRoom(roomName, identity string) (*Session, error) {`
    `r, err := lksdk.ConnectToRoom(fmt.Sprintf("<http://%s:%s>", os.Getenv("LIVEKIT_HOST"), os.Getenv("LIVEKIT_PORT")), lksdk.ConnectInfo{`
        `APIKey:              os.Getenv("LIVEKIT_API_KEY"),`
        `APISecret:           os.Getenv("LIVEKIT_SECRET_KEY"),`
        `RoomName:            roomName,`
        `ParticipantIdentity: identity,`
    `}, &amp;lksdk.RoomCallback{`
        `OnParticipantConnected:    onUserConnected,`
        `OnParticipantDisconnected: onUserDisconnected,`
        `ParticipantCallback: lksdk.ParticipantCallback{`
            `OnDataReceived: onDataReceived,`
        `},`
    `})`
    `if err != nil {`
        `return nil, err`
    `}`
    `return &amp;Session{`
        `Room:                 r,`
        `Key:                  roomName,`
    `}, nil`
`}`

`func (s *Session) SendMessage(msg any, sid ...string) error {`
    `bytes, err := json.Marshal(msg)`
    `if err != nil {`
        `return err`
    `}`
    `return s.LocalParticipant.PublishData(bytes, livekit.DataPacket_RELIABLE, sid)`
`}`

`func onUserConnected(rp *lksdk.RemoteParticipant) {`
    `if rp.Identity() == "server" || rp.Metadata() == "" {`
        `return`
    `}`
    `log.Println("user connected", rp.Identity(), rp.Metadata())`

    `time.Sleep(1 * time.Second)`

    _`//do on user connects stuff`_

    _`//will be immediately printed but clients receives this message randomly`_
    `log.Println("room data sent")`
`}`

`func onUserDisconnected(rp *lksdk.RemoteParticipant) {`
    `log.Println("user disconnected")`
`}`

`func onDataReceived(data []byte, rp *lksdk.RemoteParticipant) {`
    `log.Println("data received:", string(data), rp.Name())`
`}`

Where do you send data? I only see the line `log.Println("room data sent")`, without any calls to `SendMessage`

can you upload an example that can reproduce this as a gist? that'd make it easier for us to run

<https://gist.github.com/justCodeLife/069107fa984d5cd0f30d97d32b370042>

The problem was in ingress config. I set wrong ws_url. The url was set as <wss://livekit.example.com/>. The correct url is <wss://livekit.example.com>. Slash it the end of url brakes auth. May be it can be useful for some one else.

thank you for the note. that seems like something we should address in Ingress. I've filed issue <https://github.com/livekit/ingress/issues/112|here>.

Hey Ashan, that wouldn't be possible. the CLI is built to efficiently simulate the workload. It will not able to load any UI.

You can look into simulating Chrome load testers if you are looking for it to load your custom UI.

image.png

Hi Xiaoxin, do you mind sending me a DM with your Server URL?

I’m not at my computer atm but interestingly the token generated from the cloud dashboard worked.

Oof, thanks for letting us know, I’ll look into this. If you can send me your project’s server URL when you have the chance, I’d appreciate it!

Hey, with the same exact url and token can you try connecting using :
<https://meet.livekit.io/?tab=custom>

Are you able to connect ?

I can reproduce this error, it looks like the token generated in the sample code is incorrect somehow.

it's not possible today. there's an issue <https://github.com/livekit/client-sdk-flutter/issues/201|here>. tracking it.

any media ingested via Ingress would appear as normal Tracks within LiveKit. so yeah, it'll trigger a `TrackPublished` and `TrackSubscribed` event for clients

since this stream contains both audio and video, you would expect two separate tracks to appear.

One more question, <@U01KZ61GV5Z> is this method converting rtmp to webRTC?

Yes, LiveKit Ingress handles conversion from RTMP/WHIP to LiveKit WebRTC

they should work fine side by side. With that said, it's always a good idea to test the upgrades on a test cluster first.

are you running a single instance node? or is this part of a cluster?

hi john!
we’re planning on releasing something that should simplify this this week!

Hello <@U02EU2ZPTQU> - is it going to be announced somewhere?

yeah, We will post an update here in this thread and also in a message in <#C01KZ61HZRR|general>!

we just made <https://github.com/livekit/track-processors-js> public where you can test this out!

we’ll be adding support for this to components-react directly in the coming weeks

Hi <@U02EU2ZPTQU>,

Livekit is indeed amazing product so far. Love it.

Do you have any update on thee direct support of using blur and virtual background processor in components-react?

<@U05D8952VPC> <https://github.com/livekit/track-processors-js>

Hi <@U048B2EFGNN>,

Yes, I know this is the track processor but needs to manually publish the track. We are using react-component so it would be good to get it integrated.

hey,
the work for this can be tracked in <https://github.com/livekit/components-js/pull/534>
it’s not quite finished yet

Hi <@U02EU2ZPTQU>, thanks for making track processor public. :raised_hands:  This makes it really easy to use. But compared to teams or google meet   there is almost always some "margin" around the cut-out person.  With enabled background image replacement, this results in seeing a "glow" of original background around the person.
Is there a ways around or to improve it?

the quality difference wrt google meet and teams could be due to the fact that they have better models than the OSS version that mediapipe (google) provides.

the only way you can adjust some of that glow on the background transformer is by playing with <https://github.com/livekit/track-processors-js/blob/main/src/transformers/BackgroundTransformer.ts#L117|this value>. `3px` seemed like a good compromise, but if you find out that other values work better for most cases, then we can consider updating this value

Thanks, I'll take a look :slightly_smiling_face:

Hey! Just jumping in on this thread. Really exciting to see that background blur and virtual backgrounds are being worked on, with a view to them being fully supported - this is great news! :raised_hands:

Having tried the pull request in `components-js` the integration with the components framework looks like it’s progressing well too - thanks for all the hard work on this one.

A couple of questions:

1. Is there any likelihood of the pull request being resumed / reviewed / merged any time soon?
2. The results when testing the VirtualBackground are a little mixed - quite a lot of the actual background is bleeding through the background image. If I understand correctly, this is due to the quality of the mediapipe models. Is it likely that we’ll see some improvement in this space?

hi Tomos,

1. you should be able to use it already today within components-react, the PR that’s pending is only for the `meet` example. We’ll get back to that sometime later.
2. Unfortunately we don’t have a lot of influence on that. Those are the models that google provides for segmentation.

Background blur and virtual backgrounds are working nicely on Chrome. Thanks! However, it looks like there is no support on Firefox and Safari. Does anyone know if they will get support in the near future?

The underlying functionality that this relies on is currently only implemented in Chrome as `MediaStreamTrackGenerator`. However there are efforts across browsers to support this behaviour in a slightly adjusted way under the name `VideoTrackGenerator`. However, as of now, no browser implements this new interface without any special flags enabled.

<@U02EU2ZPTQU> <@U01KZ61GV5Z> I wonder if you could create some announcements channel on Slack to inform people about new features or releases - I currently find it hard to to catch up on new stuff. Or maybe there is something that I am not aware of?

General channel looks like another helpdesk channel to me.

Hey <@U03C8KQDJMA>, 2 things here
• This `pipeline frozen` issue is an issue I’ve been working on, related to latencies, synchronization, and timestamping. There are a lot of moving pieces, so I already have some updates written that are still unreleased, but I’m hoping to have it all solidified this week.
• `duration`, `size`, and `location` are empty because it failed to close out the file. The resulting file is just an empty mp4. However, that is a bug that the status is `ENDING` instead of `FAILED` with no `endedAt` set. I’ll add that to my backlog to look at after the timestamping issues.

Thanks for your effort <@U01UB0LHTTQ>!
It's good to hear that it's a known issue that is taken care of.

If there is anything right now we can already adjust in a k8s setup to prevent such an issue, let me know please.

Hi <@U01UB0LHTTQ>

I'm wondering if there is a related Github ticket to the *'pipeline frozen'* issue?

Am I right, that your recent commits and PRs (<https://github.com/livekit/egress/branches/active>) could hopefully improve the situation?

Thanks again for your help!

There isn't a GitHub issue for it, but my past 5 or so PRs to egress and server-sdk-go have been addressing it. I'll be releasing a new egress version tomorrow and I'm hoping the issue is fully resolved

We are so excited to test the new Egress release 1.7.2 :partying_face:
I'm sure, we will give a feedback soon.

Hey Kevin, it *should* work in theory. But we are a small team, and only has resources to support the latest version of the software. I would suggest upgrading and following the <https://docs.livekit.io/oss/deployment/|deployment docs> to ensure TURN is configured properly.

<@U01KZ61GV5Z> I also tried vp8 and removed max-bundle and rtcpMuxPolicy properties
till now I set name of camera tracks to someting fix like "camera" and now I removed setting names too because I thought maybe it effects on this problem

which app are you able to reproduce these behaviors? can you reproduce with <http://meet.livekit.io|meet.livekit.io>?

This type of functionality isn’t pre-built into the chat component, but you could certainly extend/modify it to do so. There are room/participant events you can bind to in order to obtain information like when someone joins.

<@U04PF52R5M0> mobile is not supported for this demo. <@U022KB72NFJ> may have some thoughts on mobile web support for the Unity WebGL target.

<@U04PF52R5M0> If it helps I have got mobile to work using Unity2022.2 and upgrading Text Mesh Pro to v3.2.0-pre4 . It still won't show software keyboard in full screen but works well enough if the app is using 100% of the browser window

I updated pods to latest version. Video quality should be noticeably better.

do y’all have any recommendations <@U01UEMQ8C3F> <@U02EU2ZPTQU> <@U03RM7G8Q6A>?

hey! this depends on how far you want to take it.
the easiest way is to override the css variables we define <https://github.com/livekit/components-js/blob/main/packages/styles/scss/themes/default.scss#L13-L63|here> in your own css file and just import them _after_ importing the styles package. Just be aware that we add a prefix to those, so e.g. `--bg` becomes `--lk-bg` in the actual styles package
if you want to take it even further than that, all our (non prefab) components accept `class` and `style` as react props that you can use to write your own css modules for each component.

Screenshot from 2023-06-01 00-40-26.png

oof, sorry, that’s a UI bug (“get current time, render the month name in the current timezone”). Thanks for pointing it out to us, we’ll fix the UI.

As for your project bandwidth, you’ll have to wait until June 1 UTC time to get your free June quota :slightly_smiling_face:

Hi, are there instructions to build livekit server in bazel?

1). There are two ways to `sendData` from a `RoomServerClient` <https://github.com/livekit/server-sdk-js/blob/main/src/RoomServiceClient.ts#L276-L302|here>, but I only see one of them <https://docs.livekit.io/server/room-management/#senddata|referenced in the docs>, are they both supported?

I'm trying to send a data channel message with a `topic` via the `options` argument, but I'm not seeing the topic come through client side, it's always undefined. Let me know if code snippets would be helpful.

Screenshot 2023-05-31 at 4.22.05 PM.png

2). I'm sending data channel messages (without a `topic` if that matters because of the above issue I'm having), and any time I send one, it results in an empty chat message being added to the chat prefab looking like this:

topics are not yet supported. we've added them to the protocol but server is not currently implementing them.

but yes, both forms of `sendData` are supported. It's either sending it to a select set of recipients, or everyone in the room

if you are using the chat prefab as is, then it will listen to a specific topic, which is `lk-chat-topic` defined <https://github.com/livekit/components-js/blob/main/packages/core/src/observables/dataChannel.ts#L8|here>. you can customize that of course by creating your own chat with a different (or no) topic.
The chat messages for the chat prefab also satisfy a chat message interface (<https://github.com/livekit/components-js/blob/main/packages/core/src/components/chat.ts#L7-L10|here>), so that the chat prefab understands them.

AFAIK topics should (and do) work without a problem already from client to client. just server-sdk to client is not working as of now

I’ll look into why we would be adding empty chat messages on unrelated data messages, that looks like a bug!

&gt; AFAIK topics should (and do) work without a problem already from client to client
Yeah I'm sending some data messages client to client and that seems to be working as expected

Good to know on topics not being supported server side, do you have an idea of when they will be supported?

Also thanks for looking into those empty chat messages!

<@U050W7KDGCT> there was a bug in the message handling that would lead to topics that were `undefined` to be interpreted as chat messages. This is fixed in `@livekit/components-react@1.0.5`.

Cloud server support for topics should be available sometime this week!

Yes, you can omit cloud storage settings to store locally. For example you can mount a drive to `/out` and then add `local_directory: /out/recordings/` to your config, and then all files would go into the `recordings` dir on your mounted drive. I’ve never used EFS but I imagine it’s very similar

There is the hidden property you can set on a participant’s join token:
<https://github.com/livekit/server-sdk-js/blob/7d7b2d0f459909d942d423bb1d39945afb3e04d0/src/grants.ts#L56|https://github.com/livekit/server-sdk-js/blob/7d7b2d0f459909d942d423bb1d39945afb3e04d0/src/grants.ts#L56>

But this means that only this particular participant is invisible, right? Want I want to achieve is to hide other participants unless I’m allowed to see them.

so I understand clearly: you need to actually hide them, in data, versus simply hiding them in the UI?

That’s correct, I also don’t want to allow to read chat messages (data channel) until the permission is granted.

from a permissions perspective, I don’t think this is currently supported. cc <@U01KZ61GV5Z> <@U02EU2ZPTQU> for visibility

the closest we have right now for this is the permissions API <https://docs.livekit.io/client-sdk-js/classes/LocalParticipant.html#setTrackSubscriptionPermissions>
this does not cover data messages however.

I just wondered whether that’s possible in LiveKit directly. So probably the only way to achieve that I want is to not allow my participant to connect to the room (don’t give him JWT).

Yeah, I think not allowing them to connect would be the path forward today.

I am curious though, what is your use case here? Are you trying to allow the participant to connect as a publish-only participant?

<@U01KZ61GV5Z> I want to let people into the room only when I’m ready to talk to them. But I want to know that they are waiting.

ah got it. so you are thinking of updating their permissions afterwards?

is it an issue if they could see the other participants.. but are not able to subscribe to their tracks?

I think it's not uncommon for meeting apps to show you who's already in the meeting on the waiting page

That’s exactly the issue. There may be people that I don’t want to allow to see others until the others left.

Turning off sounds from the perspective of _all_ participants in a meeting or for a single participant in one?

for a single participant. The user should be able to mute all sounds for himself. Other users can listen if they want. <@U01KZ0SCDKL>

Sounds like `room.stopPlayback` for local sound card Or `room.stopSubscribeRemoteAudioTracks`   So, which one do you prefer?

1.png

After muting, the participant should be able to unmute again.
I couldn't find either function.

I mean you can describe the functionality you need so that I can add the corresponding methods in later versions.:joy:

a) A room method for mute/unmute local audio output (the RTP packet is still being received, and the reply playback will be  fast)?
Or b) a room method that subscribes/unsubscribes to all remoteTracks (Turn on/off subscription to all RemoteTracks at once through UpdateSubscription signaling)?

Method A makes sense to me. Thank you :slightly_smiling_face:

<@U02GAC58A79> any progress? It is important feature for us.

hey,  yeah, I should be able to fix it within a week

I believe so. cc <@U01NZKTNE1H> to be sure

<@U01NZKTNE1H> I tried but not all functions were available to use in java , I think only instances can be passed but functions can't be used in a java file.

sorry, you’ll have to use kotlin due to the coroutine functions we expose

<@U01NZKTNE1H> Can I write a wrapper in kotlin and then use that in my java code, would that work?
It's a large code base that I'm working on (has an MVVM architecture). Manager classes are invoked then a connector instance is created which then creates the room object.

sure, kotlin can live alongside java code. alternatively, there may be coroutine converters available to change it to the async library of your choice.

for example, there’s a converter for rxjava: <https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-rx2/>

I think it'll be about 2 weeks.. it should be working as is.. Feel free to run against the branch directly!

can you post the server and client logs upon connecting? cc <@U02EU2ZPTQU>

'<ws://127.0.0.1:7880/rtc?access_token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuYW1lIjoiYWFhIiwidmlkZW8iOnsicm9vbSI6ImJtcGMtejlhcCIsInJvb21Kb2luIjp0cnVlLCJjYW5QdWJsaXNoIjp0cnVlLCJjYW5QdWJsaXNoRGF0YSI6dHJ1ZSwiY2FuU3Vic2NyaWJlIjp0cnVlfSwiaWF0IjoxNjg1NjkzNTI2LCJuYmYiOjE2ODU2OTM1MjYsImV4cCI6MTY4NTY5MzgyNiwiaXNzIjoiZGV2a2V5Iiwic3ViIjoiYWFhIiwianRpIjoiYWFhIn0.QJlhvU6V0t6Gj_cyIkvcrbqgSPA6ldbdM26gF9NkZ4U&amp;auto_subscribe=1&amp;sdk=js&amp;version=1.9.7&amp;protocol=9&amp;adaptive_stream=1>' failed:

In this api call, it was stuck to get the response.

hi, can you post the browser client logs too?

looks like the ws connection never establishes

I've run .\livekit-server --dev in command window

rtc?access_token=xxx api call remains status "pending"

probably, web socket is not working in windows.

Web socket connection is not working in Windows 11

<@U04CNESF2ER> also writes (pls :thread:related messages):

```# 1. Copy this file and rename it to .env.local
# 2. Update the enviroment variables below.
# API key and secret. If you use LiveKit Cloud this can be generated via the cloud dashboard.
LIVEKIT_API_KEY=devkey
LIVEKIT_API_SECRET=secret
# URL pointing to the LiveKit server.
LIVEKIT_URL=<ws://127.0.0.1:7880>
## PUBLIC
NEXT_PUBLIC_LK_TOKEN_ENDPOINT=/api/token```
It's my .env.local file

<@U04CNESF2ER> I think the issue here is a regress preventing the server from working correctly on Windows.

If you are able to pull the latest code from repo and build there, it should work correctly on Windows

<@U01KZ61GV5Z> I installed WSL ubuntu and installed livekit there, and I was able to make it working.

fantastic.. the next version 1.4.3 should work natively on Windows