This message was deleted.

Enabling virtual camera:

2023-01-23T18:56:32.180Z    INFO    livekit sfu/streamtrackermanager.go:405 available layers changed - layer seen   {"room": "6384d9c293d85d83ec64b000", "roomID": "RM_WBVnfZVSLST3", "participant": "IItb9dmGS7fquX1ciz0Gq", "pID": "PA_KbZVAvYra9Mi", "remote": false, "trackID": "TR_VCZ2842dd5JR9K", "relayed": false, "mime": "video/vp8", "added": 0, "availableLayers": [0], "exemptedLayers": []}
2023-01-23T18:56:32.229Z    INFO    livekit sfu/forwarder.go:1488   locking to target layer {"room": "6384d9c293d85d83ec64b000", "roomID": "RM_WBVnfZVSLST3", "participant": "qMyou85mf76YJcHCIOYSu", "pID": "PA_bYhxEKj7kKR6", "remote": false, "trackID": "TR_VCZ2842dd5JR9K", "relayed": false, "current": "VideoLayer{s: -1, t: -1}", "target": "VideoLayer{s: 0, t: 1}"}

Disabling virtual camera:

2023-01-23T18:56:38.181Z    INFO    livekit sfu/streamtrackermanager.go:482 available layers changed - layer gone   {"room": "6384d9c293d85d83ec64b000", "roomID": "RM_WBVnfZVSLST3", "participant": "IItb9dmGS7fquX1ciz0Gq", "pID": "PA_KbZVAvYra9Mi", "remote": false, "trackID": "TR_VCZ2842dd5JR9K", "relayed": false, "mime": "video/vp8", "removed": 0, "availableLayers": [], "exeptedLayers": []}

to be clear, the “pause screen” is the white “test” text on the black background, which the local participant can see but the remote participant cannot see?

Ah sorry for not being clear. The OBS no-camera “pause” screen (left-most camera) is the local user in both screenshots with the right-most camera being the remote view of the same participant. The right-most camera should be displaying the OBS in both cases (this is two instances of the livekit example app side by side) • In the first screenshot, the local user can correctly see the OBS no-camera screen, but in the other users session this is seen as the last frame from when the virtual camera was enabled (aka the white “test”) • In the second screenshot, the local can correctly see the OBS no-camera screen, but the remote session is receiving nothing/black

oh, hm, okay. I know the OBS pause screen doesn’t send keyframes after the initial pause screen frame, so your second bullet point is known behavior. Not sure why the first bullet point is happening though. I’m not an OBS expert but if you could configure it to re-send the keyframe periodically, I think that would fix the issue. I know @polite-kilobyte-67570 has dealt with the OBS pause screen a bit, he might have some commentary when he’s awake.

thanks for the mention! did you observe that change when upgrading server version or client version? v1.2.0 sounds like it was a server upgrade?

correct — I’m noticing this when upgrading the server v1.2.0 is the last version that works as expected, as soon as I update to and beyond v1.2.1 the behaviour changes and we run into the above (the clientside SDK running on our app didn’t change throughout upgrading and I tested above with the hosted React example app to rule out our own implementation)

@orange-angle-53929 This is caused by OBS probably sending very low frame rate. When paused, does OBS send data at all? There is a module to detect start/stop of video layers. This is needed as clients do not signal if it is going to stop a simulcast layer due to some client side constraint. So, server has to use some heuristics to detect potential stop of a video layer and switch to a different available layer. The code is written in a generic way so that we can apply the same code path even if simulcast is not enabled. Back in 1.2.0, we had the concept of exempt layers to specifically work around these kind of issues where a layer (specifically layer 0 or the lowest resolution layer) is declared always available even if the above mentioned heuristics module detects a stop. That was limited to only screen share tracks starting in 1.2.1 as we had not seen very low frame rates with camera sources. The virtual cameras (e. g. OBS) present a challenge there. So, we have made the tracker module above configurable. With one of the newer releases, you can set the tracker config like below

video:
    stream_tracker:
      video:
         exempted_layers:
            - 0

to set layer 0 as always exempt for video (camera) streams. Can you give that a try please? Also, we are working on some improvements to make this more resilient and hope to catch more cases where this does not trip. Remains to be seen how much resilient we can make that.

Thanks for the detailed response, this is super helpful — I suspect this is quirks of OBS’s virtual camera driver, anecdotally from what I’ve seen it only impacts the macOS version of OBS . E.G based on example.livekit.io stats, on the pause screen windows drops to 640x360 @ 160kbps where macOS drops to 320x180 @ 1kbps. The windows version currently works fine. Either way, after making the configuration change to the latest version of LiveKit server, the virtual camera does indeed show up fine (and no longer gets stuck when the virtual camera is disabled/pause screen visible). Is there anything to be concerned with regards to this config change in a production deployment? Presumably this just means the lowest quality may be incorrectly seen as active and be visible to remote participants as such? We’re upgrading from a pre-1.0 build of livekit, so sounds like there would be no material change in behaviour for us.

Thank you Luke for the details about differences between Windows and MacOS. Very helpful. Like you mentioned, exempting the layer should not cause anything adverse to a large extent. There are extreme cases where browsers like Firefox seem to stop lower layers while streaming higher resolution layer. Unclear under what conditions they do that. To deal with that, newer versions of the server allows overshoot when there is no congestion, i. e. even if the subscribed layer maximum is layer 0, server will stream a higher layer if for some reason layer 0 has stopped. With exempted layer, that layer will not be declared stopped. So in a condition like the above, the overshoot will not kick in. I cannot remember, but overshoot was probably not there in pre-1.0 server. So, I think there should not be any material difference. But I am being cautious in saying that as so many things have changed since pre-1.0 😊