LiveKit Community

Enabling virtual camera:
```2023-01-23T18:56:32.180Z    INFO    livekit sfu/streamtrackermanager.go:405 available layers changed - layer seen   {"room": "6384d9c293d85d83ec64b000", "roomID": "RM_WBVnfZVSLST3", "participant": "IItb9dmGS7fquX1ciz0Gq", "pID": "PA_KbZVAvYra9Mi", "remote": false, "trackID": "TR_VCZ2842dd5JR9K", "relayed": false, "mime": "video/vp8", "added": 0, "availableLayers": [0], "exemptedLayers": []}
2023-01-23T18:56:32.229Z    INFO    livekit sfu/forwarder.go:1488   locking to target layer {"room": "6384d9c293d85d83ec64b000", "roomID": "RM_WBVnfZVSLST3", "participant": "qMyou85mf76YJcHCIOYSu", "pID": "PA_bYhxEKj7kKR6", "remote": false, "trackID": "TR_VCZ2842dd5JR9K", "relayed": false, "current": "VideoLayer{s: -1, t: -1}", "target": "VideoLayer{s: 0, t: 1}"}```
Disabling virtual camera:
```2023-01-23T18:56:38.181Z    INFO    livekit sfu/streamtrackermanager.go:482 available layers changed - layer gone   {"room": "6384d9c293d85d83ec64b000", "roomID": "RM_WBVnfZVSLST3", "participant": "IItb9dmGS7fquX1ciz0Gq", "pID": "PA_KbZVAvYra9Mi", "remote": false, "trackID": "TR_VCZ2842dd5JR9K", "relayed": false, "mime": "video/vp8", "removed": 0, "availableLayers": [], "exeptedLayers": []}```

cphachdwz9ndvk907kk0gnp000tcdso2.png

After pausing the virtual camera, local participant is left:

bagw90ncrbr6q2u3jbdiyndvjplbahg6.png

When a participant joins on the “pause screen”:

to be clear, the “pause screen” is the white “test” text on the black background, which the local participant can see but the remote participant cannot see?

Ah sorry for not being clear. The OBS no-camera “pause” screen (left-most camera) is the local user in both screenshots with the right-most camera being the remote view of the same participant.  The right-most camera _should_ be displaying the OBS in both cases (this is two instances of the livekit example app side by side)
• In the first screenshot, the local user can correctly see the OBS no-camera screen, but in the other users session this is seen as the last frame from when the virtual camera was enabled (aka the white “test”)
• In the second screenshot, the local can correctly see the OBS no-camera screen, but the remote session is receiving nothing/black

Screen Recording 2023-01-23 at 11.16.50 pm.mov

oh, hm, okay.  I know the OBS pause screen doesn’t send keyframes after the initial pause screen frame, so your second bullet point is known behavior. Not sure why the first bullet point is happening though.

I’m not an OBS expert but if you could configure it to re-send the keyframe periodically, I think that would fix the issue.

I know <@U02EU2ZPTQU> has dealt with the OBS pause screen a bit, he might have some commentary when he’s awake.

thanks for the mention!
did you observe that change when upgrading server version or client version?
v1.2.0 sounds like it was a server upgrade?

correct — I’m noticing this when upgrading the server

v1.2.0 is the last version that works as expected, as soon as I update to and beyond v1.2.1 the behaviour changes and we run into the above

(the clientside SDK running on our app didn’t change throughout upgrading and I tested above with the hosted React example app to rule out our own implementation)

<@U04KMLHBWR0> This is caused by OBS probably sending very low frame rate. When paused, does OBS send data at all?

There is a module to detect start/stop of video layers. This is needed as clients do not signal if it is going to stop a simulcast layer due to some client side constraint. So, server has to use some heuristics to detect potential stop of a video layer and switch to a different available layer. The code is written in a generic way so that we can apply the same code path even if simulcast is not enabled.

Back in 1.2.0, we had the concept of exempt layers to specifically work around these kind of issues where a layer (specifically layer 0 or the lowest resolution layer) is declared always available even if the above mentioned heuristics module detects a stop. That was limited to only screen share tracks starting in 1.2.1 as we had not seen very low frame rates with camera sources. The virtual cameras (e. g. OBS) present a challenge there.

So, we have made the tracker module above configurable. With one of the newer releases, you can set the tracker config like below
```video:
    stream_tracker:
      video:
         exempted_layers:
            - 0```
to set layer 0 as always exempt for video (camera) streams. Can you give that a try please?

Also, we are working on some improvements to make this more resilient and hope to catch more cases where this does not trip. Remains to be seen how much resilient we can make that.

Thanks for the detailed response, this is super helpful — I suspect this is quirks of OBS’s virtual camera driver, anecdotally from what I’ve seen it only impacts the macOS version of OBS . E.G based on <http://example.livekit.io|example.livekit.io> stats, on the pause screen windows drops to 640x360 @ 160kbps where macOS drops to 320x180 @ 1kbps. The windows version currently works fine.

Either way, after making the configuration change to the latest version of LiveKit server, the virtual camera does indeed show up fine (and no longer gets stuck when the virtual camera is disabled/pause screen visible). Is there anything to be concerned with regards to this config change in a production deployment? Presumably this just means the lowest quality may be incorrectly seen as active and be visible to remote participants as such? We’re upgrading from a pre-1.0 build of livekit, so sounds like there would be no material change in behaviour for us.

Thank you Luke for the details about differences between Windows and MacOS. Very helpful.

Like you mentioned, exempting the layer should not cause anything adverse to a large extent. There are extreme cases where browsers like Firefox seem to stop lower layers while streaming higher resolution layer. Unclear under what conditions they do that. To deal with that, newer versions of the server allows overshoot when there is no congestion, i. e. even if the subscribed layer maximum is layer 0, server will stream a higher layer if for some reason layer 0 has stopped.

With exempted layer, that layer will not be declared stopped. So in a condition like the above, the overshoot will not kick in. I cannot remember, but overshoot was probably not there in pre-1.0 server. So, I think there should not be any material difference. But I am being cautious in saying that as so many things have changed since pre-1.0 :blush: 

hi ben
that’s expected. a `hidden` participant is invisible to other participants in the room, that’s why no events are fired.

thanks, are there any different behaviors for hidden other than events not being fired?

can you give an example of what you’re thinking of that might be different?

I am not sure, just asking in case I run into those differences in the future

is adaptive streaming turned off as well?

what livekit-server version are you using?

i reported the same for the js client which was fixed before by LK team and pushed.

Found the same issue in react-native as well

can you post your Room.create and Room.connect options?

<@U0444GJ7CLC> what happens when you introduce a delay between subscribe and setting video quality?

<@U01NZKTNE1H> then it works, I see in the logs that react-native-webrtc renegotiates tracks and then if I set the quality then it works

ok, this seems like of a react-native issue vs a timing issue with two successive calls. we are working on improvements in this area that should ship in about a week

hi liam!
this is something that is being actively worked on at <https://github.com/livekit/client-sdk-rust#do-you-plan-to-offer-a-cc-sdk>

Screenshot 2023-01-24 at 15.51.01.png

Could you say a bit more about your setup Alex? cc <@U022KB72NFJ> 

I'm using cloud setup at <wss://ansoftware.livekit.cloud> and trying to connect with RustSDK and simple_room. Below are the logs
`2023-01-24T15:40:36.298660Z  INFO wgpu_core::instance: Adapter Metal AdapterInfo { name: "AMD Radeon Pro 5500 XT", vendor: 0, device: 0, device_type: DiscreteGpu, driver: "", driver_info: "", backend: Metal }`    
`2023-01-24T15:40:36.303393Z  INFO wgpu_core::device: Created buffer Valid((0, 1, Metal)) with BufferDescriptor { label: Some("egui_uniform_buffer"), size: 16, usage: COPY_DST | UNIFORM, mapped_at_creation: true }`    
`2023-01-24T15:40:36.308235Z  INFO wgpu_core::device: Created render pipeline Valid((0, 1, Metal)) with RenderPipelineDescriptor { label: Some("egui_pipeline"), layout: Some((0, 1, Metal)), vertex: VertexState { stage: ProgrammableStageDescriptor { module: (0, 1, Metal), entry_point: "vs_main" }, buffers: [VertexBufferLayout { array_stride: 20, step_mode: Vertex, attributes: [VertexAttribute { format: Float32x2, offset: 0, shader_location: 0 }, VertexAttribute { format: Float32x2, offset: 8, shader_location: 1 }, VertexAttribute { format: Uint32, offset: 16, shader_location: 2 }] }] }, primitive: PrimitiveState { topology: TriangleList, strip_index_format: None, front_face: Ccw, cull_mode: None, unclipped_depth: false, polygon_mode: Fill, conservative: false }, depth_stencil: Some(DepthStencilState { format: Depth32Float, depth_write_enabled: false, depth_compare: Always, stencil: StencilState { front: StencilFaceState { compare: Always, fail_op: Keep, depth_fail_op: Keep, pass_op: Keep }, back: StencilFaceState { compare: Always, fail_op: Keep, depth_fail_op: Keep, pass_op: Keep }, read_mask: 0, write_mask: 0 }, bias: DepthBiasState { constant: 0, slope_scale: 0.0, clamp: 0.0 } }), multisample: MultisampleState { count: 1, mask: 18446744073709551615, alpha_to_coverage_enabled: false }, fragment: Some(FragmentState { stage: ProgrammableStageDescriptor { module: (0, 1, Metal), entry_point: "fs_main_gamma_framebuffer" }, targets: [Some(ColorTargetState { format: Bgra8Unorm, blend: Some(BlendState { color: BlendComponent { src_factor: One, dst_factor: OneMinusSrcAlpha, operation: Add }, alpha: BlendComponent { src_factor: OneMinusDstAlpha, dst_factor: One, operation: Add } }), write_mask: RED | GREEN | BLUE | ALPHA | COLOR | ALL })] }), multiview: None }`    
`2023-01-24T15:40:36.308384Z  INFO wgpu_core::device: Created buffer Valid((1, 1, Metal)) with BufferDescriptor { label: Some("egui_vertex_buffer"), size: 20480, usage: COPY_DST | VERTEX, mapped_at_creation: false }`    
`2023-01-24T15:40:36.308454Z  INFO wgpu_core::device: Created buffer Valid((2, 1, Metal)) with BufferDescriptor { label: Some("egui_index_buffer"), size: 12288, usage: COPY_DST | INDEX, mapped_at_creation: false }`    
`2023-01-24T15:40:36.308545Z  INFO wgpu_core::device: configuring surface with SurfaceConfiguration { usage: RENDER_ATTACHMENT, format: Bgra8Unorm, width: 1600, height: 1200, present_mode: AutoVsync, alpha_mode: Auto }`    
`2023-01-24T15:40:36.308576Z  INFO wgpu_core::device: Automatically choosing presentation mode by rule AutoVsync. Chose Fifo`    
`2023-01-24T15:40:36.308593Z  INFO wgpu_core::device: Automatically choosing alpha mode by rule Auto. Chose Opaque`    
`2023-01-24T15:40:36.308604Z  INFO wgpu_hal::metal::surface: build swapchain SurfaceConfiguration { swap_chain_size: 3, present_mode: Fifo, composite_alpha_mode: Opaque, format: Bgra8Unorm, extent: Extent3d { width: 1600, height: 1200, depth_or_array_layers: 1 }, usage: COLOR_TARGET }`    
`2023-01-24T15:40:36.308822Z  INFO wgpu_core::device: Created texture Valid((0, 1, Metal)) with TextureDescriptor { label: Some("egui_depth_texture"), size: Extent3d { width: 1600, height: 1200, depth_or_array_layers: 1 }, mip_level_count: 1, sample_count: 1, dimension: D2, format: Depth32Float, usage: TEXTURE_BINDING | RENDER_ATTACHMENT }`    
`2023-01-24T15:40:36.338666Z  INFO wgpu_core::device: configuring surface with SurfaceConfiguration { usage: RENDER_ATTACHMENT, format: Bgra8Unorm, width: 1600, height: 1200, present_mode: AutoVsync, alpha_mode: Auto }`    
`2023-01-24T15:40:36.338726Z  INFO wgpu_core::device: Automatically choosing presentation mode by rule AutoVsync. Chose Fifo`    
`2023-01-24T15:40:36.338738Z  INFO wgpu_core::device: Automatically choosing alpha mode by rule Auto. Chose Opaque`    
`2023-01-24T15:40:36.338747Z  INFO wgpu_hal::metal::surface: build swapchain SurfaceConfiguration { swap_chain_size: 3, present_mode: Fifo, composite_alpha_mode: Opaque, format: Bgra8Unorm, extent: Extent3d { width: 1600, height: 1200, depth_or_array_layers: 1 }, usage: COLOR_TARGET }`    
`2023-01-24T15:40:36.338957Z  INFO wgpu_core::device: Created texture Valid((1, 1, Metal)) with TextureDescriptor { label: Some("egui_depth_texture"), size: Extent3d { width: 1600, height: 1200, depth_or_array_layers: 1 }, mip_level_count: 1, sample_count: 1, dimension: D2, format: Depth32Float, usage: TEXTURE_BINDING | RENDER_ATTACHMENT }`    
`2023-01-24T15:40:36.399638Z  INFO wgpu_core::device: Created texture Valid((3, 1, Metal)) with TextureDescriptor { label: Some("egui_texid_Managed(0)"), size: Extent3d { width: 2048, height: 256, depth_or_array_layers: 1 }, mip_level_count: 1, sample_count: 1, dimension: D2, format: Rgba8UnormSrgb, usage: COPY_DST | TEXTURE_BINDING }`    
`2023-01-24T15:40:36.401993Z  INFO wgpu_core::device: Created buffer Valid((3, 1, Metal)) with BufferDescriptor { label: Some("egui_vertex_buffer"), size: 40960, usage: COPY_DST | VERTEX, mapped_at_creation: false }`    
`2023-01-24T15:40:43.336492Z  INFO wgpu_core::device: Created buffer Valid((1, 2, Metal)) with BufferDescriptor { label: Some("egui_index_buffer"), size: 24576, usage: COPY_DST | INDEX, mapped_at_creation: false }`    
`2023-01-24T15:40:47.717130Z  INFO connect{self=RTCEngine { inner: EngineInner { lk_runtime: LKRuntime, session_info: Mutex { data: None }, running_handle: RwLock { mr: 536870911, s: Semaphore { permits: 536870911 }, c: UnsafeCell { .. } }, opened: false, engine_emitter: Sender { chan: Tx { inner: Chan { tx: Tx { block_tail: 0x7fc5f0055200, tail_position: 0 }, semaphore: Semaphore { semaphore: Semaphore { permits: 8 }, bound: 8 }, rx_waker: AtomicWaker, tx_count: 1, rx_fields: "..." } } }, reconnecting: false, full_reconnect: false, reconnect_interval: Mutex { data: Interval { delay: Sleep { inner: Inner { deadline: Instant { t: 803519739204844 } }, entry: TimerEntry { driver: MultiThread(multi_thread::Handle { ... }), inner: UnsafeCell { .. }, initial_deadline: Some(Instant { t: 803519739204844 }), _m: PhantomPinned } }, period: 5s, missed_tick_behavior: Burst } } } } url="<wss://ansoftware.livekit.cloud>" token="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NzQ2NjE3NzgsImlzcyI6IkFQSUVBWEhWazhrWlh1UCIsIm5iZiI6MTY3NDU3MTc3OCwic3ViIjoiYW50ZXN0MSIsInZpZGVvIjp7ImNhblB1Ymxpc2giOnRydWUsImNhblB1Ymxpc2hEYXRhIjp0cnVlLCJjYW5TdWJzY3JpYmUiOnRydWUsInJvb20iOiJydXN0X3Rlc3QiLCJyb29tSm9pbiI6dHJ1ZX19.dk1XDEIm13lLyJ3XkfzdZ7wKOIkACMD0azxwkuFfyJ8" options=SignalOptions { reconnect: false, sid: "", auto_subscribe: true, adaptive_stream: false }}: livekit::signal_client::signal_stream: connecting to SignalClient: <wss://ansoftware.livekit.cloud/rtc?access_token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NzQ2NjE3NzgsImlzcyI6IkFQSUVBWEhWazhrWlh1UCIsIm5iZiI6MTY3NDU3MTc3OCwic3ViIjoiYW50ZXN0MSIsInZpZGVvIjp7ImNhblB1Ymxpc2giOnRydWUsImNhblB1Ymxpc2hEYXRhIjp0cnVlLCJjYW5TdWJzY3JpYmUiOnRydWUsInJvb20iOiJydXN0X3Rlc3QiLCJyb29tSm9pbiI6dHJ1ZX19.dk1XDEIm13lLyJ3XkfzdZ7wKOIkACMD0azxwkuFfyJ8&amp;protocol=8&amp;reconnect=0&amp;sid=&amp;auto_subscribe=1&amp;adaptive_stream=0>`
`2023-01-24T15:40:49.154649Z  INFO wgpu_core::device: Created buffer Valid((2, 2, Metal)) with BufferDescriptor { label: Some("egui_vertex_buffer"), size: 81920, usage: COPY_DST | VERTEX, mapped_at_creation: false }`    
`2023-01-24T15:40:52.131734Z  INFO livekit::room::room_session: test`
`2023-01-24T15:41:28.904113Z ERROR livekit::rtc_engine::rtc_events: ICE candidate error (Subscriber): address: 169.254.123.x - port: 64446 - url: stun:143.42.16.135:3478 - error_code: 701 - error_text: STUN binding request timed out.`
`2023-01-24T15:41:28.904157Z ERROR livekit::rtc_engine::rtc_events: ICE candidate error (Subscriber): address: 169.254.175.x - port: 49787 - url: stun:143.42.16.135:3478 - error_code: 701 - error_text: STUN binding request timed out.`
`2023-01-24T15:41:28.904171Z ERROR livekit::rtc_engine::rtc_events: ICE candidate error (Subscriber): address: 169.254.88.x - port: 59905 - url: stun:143.42.16.135:3478 - error_code: 701 - error_text: STUN binding request timed out.`
`2023-01-24T15:41:28.904183Z ERROR livekit::rtc_engine::rtc_events: ICE candidate error (Subscriber): address: 169.254.85.x - port: 65390 - url: stun:143.42.16.135:3478 - error_code: 701 - error_text: STUN binding request timed out.`
`2023-01-24T15:41:58.129433Z  INFO livekit::room::room_session: test`
`2023-01-24T15:41:58.129472Z  INFO livekit::room::room_session: Participant connected: PA_8jJNu8pRTeQ8`

It looks like no valid ICE candidates `ICE candidate error (Subscriber): address: 169.254.123.x - port: 64446 - url: stun:143.42.16.135:3478 - error_code: 701 - error_text: STUN binding request timed out.`

Hey <@U04KM5CRHHP>, how are you publishing videos?
The Rust SDK doesn't support publishing yet (We're actively working on it).
If you want to receive videos on your Rust app. You can either use the livekit-cli or use the JS example app to publish videos

Is there an SDK allowing publishing videos in a native(Win and Mac) app?

Here’s (probably) the fastest path Alex:
<https://docs.livekit.io/getting-started/cli-setup/#simulating-another-publisher|https://docs.livekit.io/getting-started/cli-setup/#simulating-another-publisher>

Thanks, but if I want to publish live video from camera I probably should take the Flutter route?

Yes, if you want to use your webcam, it is a good way

The fastest way to do that is probably to use this (if you’re just trying to test)?
<https://example.livekit.io/>

Yes, I saw your cool React components and I already ran Unity WebGL example but I need to run a native app.

If I mute the microphone after starting the egress, then the mp4 file is saved.

I expect a silent mp4 video to be saved, containing the camera recording

this is up next on my backlog - <https://github.com/livekit/egress/issues/203>

It’s good to hear you have a repro. cc <@U01NZKTNE1H> for follow up questions.

<@U049R9F507Q> do you have logs from these users? Can you send us a couple of example session URLs?

<@U01NZKTNE1H> thanks for the reply, and apologies for the delay. here’s an example session url: <https://cloud.livekit.io/projects/p_1eeobbhm3tg/sessions/RM_AxjF3UizeFy8>
On the client side, I’m seeing “could not connect PeerConnection after timeout”

Here’s another example: <https://cloud.livekit.io/projects/p_1eeobbhm3tg/sessions/RM_YZDhQFjpwjaq>
In this one, user `KXHf5uukd3SKEXRVSD1URwKwTPP2` is the one with the issue. At the end they were able to connect when they switched back to wifi, but they were unable to connect until then when they were on mobile data.

thanks for sharing Sruli, just to confirm: can the user connect from our web client? I.e. if they can connect from <https://meet.livekit.io>

Thanks <@U01NZKTNE1H> both users are in the US and are not online at the moment. I will check today and get back to you. Thanks :pray:

<@U01KZ61GV5Z> I tried this out with an affected user. He was able to see himself. However, if I joined first, he was unable to see me when he joined. If he joined first, he was able to see me when I joined. This was reproduce-able consistently as we were testing.

to confirm, this user is connecting on mobile / cellular network?

I think him not able to see you might be a bug with the application on mobile. As long as he's able to see himself, that means he's indeed able to connect to LiveKit Cloud from cellular networks. The issue seems to be with react-native somewhere

we'll investigate.. we may need to add additional logging on the server side to track this down a bit better.

okay, cool. Thank you. Let me know if you need anything from me.

<@U01KZ61GV5Z> I’ve confirmed that this happens when there are two sim cards on the device. Both affected users have two sim cards on their device. We tested with them removing one of them and they were able to connect to the session.

Hey Sruli! I really appreciate the update! Wow.. this is bizarre.. I guess this may be a difference between web browser vs native webrtc on how it handles dual sim devices. we should test this use case with our RN client.

filed: <https://github.com/livekit/client-sdk-react-native/issues/47>

Hey <@U01KZ61GV5Z>. Do you know if there’s been any movement on this? I’m learning about more users affected by this issue, and there must be others I do not know about. It’s the main issue we’re experiencing with Livekit. To the extent that when someone reports that they are unable to connect to calls, I immediately ask if they are (a) on a dual sim device and (b) on data. And that’s usually it. Thanks :pray:

hey <@U049R9F507Q> sorry we haven't been able to track this down. It's on our list of high priority issues tho. We'll spend some time on it this week and well keep you posted.

<@U049R9F507Q> we’re working on getting some dual sim devices to test this against, would you happen to have any examples of known devices it fails on? Just want to make sure we don’t run into a rare dual sim device where it does somehow work

Hey <@U01NZKTNE1H> I just reached out to some users. I’ll let you know when I hear back from them. Thanks :pray:

<@U01NZKTNE1H> just heard back from one user: iPhone 13 / iOS 16.2

hmmmm if it’s iOS only specifically, looks like it’s this issue: <https://bugs.chromium.org/p/webrtc/issues/detail?id=10966>

there’s a field trial that seems to have fixed this in 2021 but they haven’t turned it on by default yet

can you try putting this in your AppDelegate.m:

```
#import &lt;WebRTC/RTCFieldTrials.h&gt;

- (BOOL)application:(UIApplication *)application
    didFinishLaunchingWithOptions:(NSDictionary *)launchOptions {
  NSDictionary *fieldTrials = @{kRTCFieldTrialUseNWPathMonitor : kRTCFieldTrialEnabledValue};
  RTCInitFieldTrialDictionary(fieldTrials);
  /* ... */
}```

yes it seems to be iOS specific. I’m not aware of any instances on Android.

I just tested this out with one user and it seems to be working. Going to be doing a bit more testing today. Thanks so much :pray:

<@U01NZKTNE1H> <@U01KZ61GV5Z> tests seem to be holding up. Just released a new version with the fix. Thanks for your help.

Thank you <@U049R9F507Q> for the report and validating the fix! It would have been really difficult for us to track this one down without your help

yes, if they are not expected to broadcast anymore, you should unpublish their tracks.

also, if you <https://docs.livekit.io/concepts/authentication/#updating-permissions|update their permissions> to not be able to publish, we'd automatically unpublish all of their tracks.

If the meeting host wishes, they can broadcast again. But it's not always the case that the host asks again.

In case they broadcast again, would it be enough to set Camera Enabled(true) and setMicrophoneEnabled(true)?

if you don't take away their permission to publish, then yes

image.png

For `LiveKit URL` you should put `<wss://distancesales-preprod.livekit.cloud>` :slightly_smiling_face:

Sure, myserver was write just to hide the url, but after we decide to edit the posrt and put correct url. The test was done with correct url ;-)

Hey <@U04CW432GJV>, looks like certain room names were not able to connect to livekit for the last 3 hours in EU. it should be resolved now. We'll be posting an incident report on <http://status.livekit.io|status.livekit.io> shortly

bumping this for review/reproduction since this is blocking us from launching the product :slightly_smiling_face:

Hey, is it something you see only see on our demo?
Or the issue is coming from our SDK and affect your product.

It came in our product, we were luckily able to reproduce it in your demo as well

Do you see any error on the console? Or it is only the behaviour that is wrong?

No error on the console, only behavior is wrong

Do you know which event is not fired correctly ?

I just released a new version of client-sdk-unity-web, can you give it a try ?

&gt; Do you know which event is not fired correctly ?
We are not sure

&gt; I just released a new version of client-sdk-unity-web, can you give it a try ?
Taking a look

Would the new client changes also be live in <http://unity.livekit.io/> so that we can test it out quickly?

to understand: does the issue occur only after a certain number of participants are in the room? 
does it go away if you set `expWebAudioMix : true` un your room settings?

when we were trying it internally with 3-5 ppl it was ok, in production streams with 7+ people it was noticeable and it seemed to get worse with time.
We’ll try the option you’re suggesting and I’ll post feedback here.

there are some restrictions by browsers concerning the maximum number of simultaneously playing audio elements. I think chrome mixes only the three loudest together IIRC.

and is there some recommendation regarding bitrates?
we’re now using `32_000` (`AudioPresets.music.maxBitrate`) for everything because we had troubles getting microphone volume on-par with the music and increasing the bitrate (from `AudioPresets.speech.maxBitrate`) helped…

32kbps is more than enough for mono. If you are playing music in stereo, you could try to increase that even further. Folks have reported a noticeable quality increase for stereo content

<@U02EU2ZPTQU> So far we’ve increased bitrates for instructor to:
• music track - `96kbps`
• microphone track - `48kbps`
decreased bitrate for participants to `20kbps`

plus enabled `expWebAudioMix` and disabled `dtx` for instructor and it seems to have made a big difference. We’ll try combinations of `expWebAudioMix` and `dtx` ON/OFF to see what really helps.

HandleSpeakersChanged.png

Hey Vicente, do you know what is the content of the array? ( At the end of the message )

I reproduced the error and made screenshots of the array content:
Hope it helps! Thank you very much.

<@U022KB72NFJ> Any ideas about what might be happening? or at least a workaround to avoid the crash? please the success of a big project depends on it

Hey Vicente, I’m going to take a look, do I have do to something to reproduce the issue?
Also what is your Unity version?

I don't know how to reproduce the issue, it ramdomly happens to some connected users, and sometimes don't

Do you use Participant.IsSpeakingChanged?

also, it never happens when we test it deploying in localhost, only when we use the server

Can you try to reproduce the issue with the `LK_DEBUG` define added to the build system?
It enables internal logs

ok! I'll try it and I'll paste the log, thank you so much!

<@U022KB72NFJ> I added LK_DEBUG to the script compilation symbols, but I am not sure if the info provided changed

I can’t figure out the order of the messages with the screenshots? Can you mb send a log file?

I have just realized that there are a couple "StopAllCoroutines" at our code. Can these also affect the coroutines of livekit and mess it up??
EDIT: Nevermind, I removed them and the error still happens

maybe the unity build is outdated and conflicts with the last server releases? our server build is from last friday while the last unity sdk is from september (maybe I am saying silly things, just trying to figure out a reason :sweat_smile:)

Hey, I’m going to update the JS SDK version today, I’ll investigate the above issue

<@U022KB72NFJ> Thank you! Meanwhile I keep trying things

I guess the unity SDK is based on the JS one? will it be updated as well?

Hey, I just updated the Unity SDK to JS SDK v1.6.3

Same error, it looks that eventually the IsSpeakingChanged C# callback is unreachable :cry:

So weird, so you’re not using IsSpeakingChanged and ActiveSpeakersChanged?
It’s weird that nothing happen on <http://unity.livekit.io|unity.livekit.io>

So there is no particular steps to reproduce the issue?

no, we just need to keep speaking for a while, and eventually one of the IsSpeakingChanged fails

just guessing things: might it be GC related? if you register a dummy event handler for the speaker updates and thus keep it alive, does that help?

In the last tries I suscribed to the ActiveSpeakersChanged event

<@U022KB72NFJ> We see that right before the error, there are always those "SetKeepAlive" to false which seem to be causing the next callbacks to not be found? We can't figure out what may cause that some resources are realeased/killed. The weird thing is that after the error the next calls to the speakers are fine again and everything keeps working (just slower due to some broken JS behavior or something like that? No idea).
Sorry to annoy you again, we are getting crazy here :laughing:

<@U02EU2ZPTQU> I totally disabled the garbage collector to check if it might be related, and the error still happens :worried:

the GC was just a random guess, sorry to hear that it doesn’t seem to fix things.

Can you enable Full StackTrace ( even for info logs ) ?
So we can know where the SetKeepAlive(false) come from

I can’t reproduce that on my side, it’s why I’m asking you to do it :slightly_smiling_face:

Also it’s weird, I always thought it was a GC issue, maybe the references are still broken even if the GC is disabled

Good news! It looks like the error is solved! This is what we did:
We have several Player instances in the game. Even though a new player is created every time a participant joins the room, they are not related in any other way. So we had independent participants and players.
Even though it did not seem a GC issue because I disabled it, I have tried to keep a reference to the Participant in every Player we create, so it wouldn't be disposed while the player exists. I tested it for 10 minutes and the error did not happen again! (the error used to happen in less than 1 minute).
I can't thank you enough for all your help and time dedicated to this! I would have never guessed the cause without your help!

Ohhhh gotcha, we’re already keeping a reference of the Participant, so something is wrong on our side, I’ll verify that the references are valid and publish a fix if this is that.
Thanks for investigating :slightly_smiling_face:

When you reproduce the issue, does a participant disconnect in the meantime?

Nope, they kept connected, their cam worked and they could speak normally. The game became very slow though

if you mean if anybody else left the room, no, the participants never changed before or after the issue

Is it possible that you reproduce the issue with full StackTrace, also for the Info logs? ( In the build settings )
So I can see where the SetKeepAlive(false) come from

If we enable the full stack trace, there happens an error right after entering the room and it fails

it is related to a null track, despite we added a try-catch arround the tracks adding

<@U022KB72NFJ> We can't find a donations link. Is there a way to get you an small tip to thank your support? For some well deserved beers :wink:

Hey, thank you very much for offering but we don’t need it :blue_heart:

this is something we will have to improve in our docs, but the third value in your array actually doesn't have any effect. in order to get the highest layer as 540, you will have to acquire the camera track with these settings (the highest layer is always automatically the original one acquired from the device)

and I guess adaptive streaming always starts with HIGH simulcast layer and then SFU will dictate what is the highest requested resolution and participant’s publisher will adapt to it…

the `videoQuality` you are reading from currently doesn’t update when adaptive stream is being used. Agreed that this is a bit misleading, will open an issue about it. Either we need to clarify that the reading is only reflecting the correct state if setting the quality manually or update it also for adaptive stream changes!

<https://github.com/livekit/client-sdk-js/issues/560>

No incident visible on our side. Can you please share your details of an attempt where things failed? `project id` and `participant identity` is good. You can DM me.

It was a flaky connectivity issue from LK side. awaiting a better experience in detecting soon. Thanks

It's delayed. I was trying to start an egress on the `tack_published` webhook, but the participants tracks that `listParticipants()` returns doesn't include the recently published tracks. It's not an issue, but unexpected. I think this should be mentioned somewhere in the docs.

Hey <@U04L7AJ26P7> are you using Cloud or self-hosted?

Self hosted. I think it's caused by a network delay between livekit and redis, because delaying `listParticiapants()` by 2 seconds(just to be safe) on a `track_published` webhook solved the issue. So livekit doesn't wait until the data is persisted in redis and sends the webhook.

Got it, you are right.. the order of operation is incorrect here.

filed <https://github.com/livekit/livekit/issues/1339> to track. we'll fix this one in the next release

huh, that sounds like a bug! `useToken` has no business of importing the `tlds` package, which is only used for `chatMessageFormatter`

• which version of components are you using? 
• does the error change between version 0.2.1 and 0.2.3? 
• what bundler/builder are you using?

I'm using components-react: `0.2.3`  in a next.js `12` app.

thanks, are you using a nodejs version &gt; 17.5 by chance?

from what I see.. its being used here: <https://github.com/livekit/components-js/blob/0e02fcab0e4247f70ce318837403fa465a89d80a/packages/core/src/helper/urlRegex.ts>

This said, you are right, it is as soon as I use any of the components to create a room.. such as:
```import {
  useToken,
  LiveKitRoom,
  VideoConference,
} from "@livekit/components-react";```
Not specific to `useToken`.

yup, this is connected to import assertions being required in nodejs v17.5 and above for JSON imports

I guess nodejs simply complains because the import exists in the file.

isn't the fix to only do something like:

```import tlds from 'tlds' assert {type: 'json'};```


yeah, that’s the fix, I’m just trying to figure out if this breaks things with older node versions

```Node JS : v17.1.0, v16.14.0	
Add support for import assertions.```
I feel this should be fine.

LiveKit is planning on supporting up to which older version?

it’s not set in stone, for components specifically we won’t strive to support versions that have reached EOL already, but v14 is still in maintenance mode, so if nothing speaks against that, that would be the earliest version we would strive to support

in some local tests things seem to work fine with v14 too though

Well, that's great news. Maybe they've backported support or its simply being ignored in earlier version since its only adding more context :man-shrugging:

yeah not sure what, but opted to just not use JSON imports - just to be safe :smile:
here’s the <https://github.com/livekit/components-js/pull/274|fix>, we’ll have a high release cadence during our preview phase anyways so I’ll rollout a release in a bit!

Yeah I was following the PR :wink: Great work <@U02EU2ZPTQU>! And thanks a lot for the quick fix! Much appreciated.

Ill keep an eye out for the new release, and I can let you know if everything works :slightly_smiling_face:

This is what the <http://example.livekit.io|example.livekit.io> receives.

Logically speaking, that `test-identity` participant shouldn't be shown, if I am correct.

hm hidden participant would not be sent to any clients. so they should not be visible at all. if they are showing up, that seems like a bug on the server side.

are you saying the components work the same way as the existing example app?

No, its different. From my tests anyways.

I'll debug this some more and provide more context if i can.

what is the difference? from the looks of it, the existing example app is also showing the participant with `test-identity`

OKay, So I did get the chance to look into this some more...
```{
  "sub": "internal-user3",
  "jti": "internal-user3",
  "exp": 1674801787,
  "nbf": 1674787387,
  "iat": 1674787387,
  "iss": "APIHDb4NpGLEDNG",
  "name": "Internal Test User 3",
  "video": {
    "roomCreate": false,
    "roomJoin": true,
    "room": "testRoom",
    "canSubscribe": true,
    "hidden": true
  }
}```
I think this is happening through JS SDK.

I've ran this on 4 different windows... and I can see all the users on each.

You can test this through <http://example.livekit.io|example.livekit.io>.

Providing some test context...
1 isnt hidden, but all 3 are showing.

I'm rather curious if this is a bug or something.

Ill let it to the team to test and see if this is only happening on my end or not.

When testing as a Go Hidden Participant, that participant is never shown, kinda strange that the behaviors are different.

Pat, can you share a hidden participant's token with me?

Hey <@U01KZ61GV5Z>, sorry for the delay, here's a token `eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuYW1lIjoiSmFjayIsInZpZGVvIjp7InJvb20iOiJmNGVkNzlkZC03OGRjLTRjOTYtOGNjZC1iOTY3ZGJmYmI3NzEiLCJyb29tSm9pbiI6dHJ1ZSwiY2FuUHVibGlzaERhdGEiOnRydWUsImNhblN1YnNjcmliZSI6dHJ1ZSwiaGlkZGVuIjp0cnVlfSwiaWF0IjoxNjc1MDkyNDQ4LCJuYmYiOjE2NzUwOTI0NDgsImV4cCI6MTY3NTExNDA0OCwiaXNzIjoiQVBJSERiNE5wR0xFRE5HIiwic3ViIjoicC1qYWNrLWxkaXlzbzZ2IiwianRpIjoicC1qYWNrLWxkaXlzbzZ2In0.1bU7Xs3QkEaLj763i74oDJkJu6QsUROd8hIAov8J6Gs`

bitrate-problem.mp4

Hello everyone,
First of all, I would like to thank all the friends who contributed to LiveKit, which I liked very much as I used it.

My problem is that the video of everyone in the video conference is at 1080p or 720p. There will be at most 3 people in the conference. And these people have high speed internet.

If you watch the video, you will see that, for some reason I don't understand, the bitrate values start at high values when publishing starts and then drop rapidly. In this case, the quality of the stream to the other people in the conference also decreases.

I have played with many settings such as Adaptive Stream, Simulcast, Dynacast and VideoQuality but no change. I can't understand why this is happening.

On the other hand, when I check cpu load avarage and network with iftop, there is almost no load or anything strange.

If anyone has any suggestions, I would appreciate it.

Best

Screenshot 2023-01-27 at 11.58.04 AM.png

Hello <@U03M7581MNF>, it is impossible to join multiple participants with the same identity in a same room. The first participant will be disconnected as soon as the second participant joins.

to extend on what Pat said:
`identity` is treated as a unique user id within each room, if you want to set a display name there’s also the `name` that you can set when generating a token

but I want to be able to be connected with the same identity from many devices: let’s say from the phone and from the computer

and be able to write texts from both devices

imagine how it is on all chat apps: you can switch from one device to another in the same chat

you should use a unique identity instead. i.e. `user-deviceId`

video isn't like chat, since we have to attribute each published video or audio track to a single `Participant` construct. for more information, please refer to <https://docs.livekit.io/concepts/api-primitives/#participant|our docs>

to expand on what David said: you could do this in your app, eg, imagine all your participant identities are like `andy:1`, `andy:2`, `andy:3`, etc, when connected from different devices, but your app could just render the UI for all of them with the label `andy`

(or you could have some in-memory data structure on your server managing the mapping between user-facing labels and participant identities)

Thank you for the suggestions David and Noah.
I’ll do that.

Any suggestion on how I could connect to the same room from many devices?

react-native SDK does not support simulcast correctly yet. this is still being worked on.

Hi, I was wondering if it is in the roadmap to implement simulcast in react native and if that is the case, when.  I ask because I want to start a project in react native with this library. Thanks in advance :slightly_smiling_face:

It does support simulcast now! just <https://github.com/livekit/client-sdk-react-native/releases/tag/v1.0.0|released v1.0.0> today

Hey LK Team, After reconnecting to Livekit room after a long time, the tracks dont resume and i see these error
```cannot send signal request before connected, type: offer
cannot send signal request before connected, type: trickle```

The only requirement is for something that could generate tokens using the api key &amp; secret. Because those keys give a user full access to your livekit instance, they should not be exposed to the user in the client code.

That is the reason why we recommend using our <https://docs.livekit.io/references/server-sdks/|server SDKs> or CLI to <https://docs.livekit.io/concepts/authentication/|generate a token> (which is time-limited and scoped to a particular user).

Thank you I was able to figure it out and make it work :D

Hey Rossam,
1. We have data centers around the globe. Currently singapore is the only data center in Asia. Japan and south korea are coming online in the near future. There shouldn't be any latency issues connecting from japan to singapore though. We typically see &lt;80ms latency.
2. Yes, our Unity WebGL SDK wraps around our JS SDK, which is optimized for mobile.
3. I don't believe so
4. 100k with LiveKit Cloud
5. We do not use MOS as it's not a great fit for RTC. but send down a similar connection quality metric.
6. we use the standard noise &amp; echo canceling features that's present in WebRTC

<@U01KZ61GV5Z>
aww, thank you so much!
your answers are very helpful for me :+1:
I’d appreciate it if you’d allow me to continue asking my question a little longer

how far have ur team verified LiveKit with regard to mobile browser support?
In our product, we would like to implement screen sharing, streaming video from a smartphone camera and comments from viewers on it on the screen, is this possible using LiveKit?

I’d also like to know if ur team know the procedure for checking my test project using LiveKit in a mobile browser environment…

<@U01KZ61GV5Z>
could u help me with the above?

Hey, we've tested mobile browsers comprehensively. Some have been completed on a physical device, others via BrowserStack. We'll be publishing a the test results to the SDK readme soon.

Thank you for answering!
I’m looking forward to it!

I might be missing something, but could it be possible that no `beforeunload` event is triggered with this codebase to disconnect from a room ?