LiveKit Community

I'm using livekit cloud

when I'm trying to test it via <http://example.livekit.io|example.livekit.io> and flutter example program

I got permission denied error

I really dont know what to do anymore

image.png

the server response

I create token with:

```.\livekit-cli.exe create-token --admin --room "room1" --identity rubi  --api-key xxx --api-secret xxxx```

thanks <@U032TUAL4KT>

I wonder if there is a place to request a livekit *cloud feature?

Thanks for jumping in, <@U032TUAL4KT>!

<@U03NBU0MGA1> right here’s the place for feature requests (I work at LiveKit)

The LiveKit team is very active. Feel free to post your feature request, I'm sure they'll see it.

My Feature Request:

1. An API to get Live Thumbnail (<https://developers.cloudflare.com/stream/viewing-videos/displaying-thumbnails/>)
2. An API to get live viewer count (<https://developers.cloudflare.com/stream/getting-analytics/live-viewer-count/>)

FYI: #2 is totally feasible with the existing SDKs.

yeaa, but as a cloud version, having Web API would be much better n easier :smiley:

anyway, thanks guys, let me continue with my code again :pray::raised_hands:

this is the expected behavior depending on the OS, it's not within our control

I'm on Windows and other apps don't do it: It seems like there is something going wrong, discord for example doesn't do it on Windows (but they have a setting for enabling/disabling, it's called attenuation) as well as WebRTC on the web (which uses libwebrtc too).

Okay, so I did some more research and as it turns out the call seems to count as a Voice-over-IP call which has lowered volume by default. Is there any way you could implement something to prevent this, otherwise I'll just have to tell users to change their settings which is also fine ig

WebRTC in browsers would go through a diff path. On Windows, I think there might be ways of overriding this behavior.

<@U02GAC58A79> do you know of a way we could do this?

This may be related to the adjustment of the Windows volume slider by the hardware volume controller on audio device core win. I will try to dig into the code for Windows to find out the reason.

<@U052BJTEV2N> When you find that the volume is lowered, can you check the position of the windows volume slider?

The volume in the volume slider is at the same position as before, but the audio seems to count as communication: I even found a way to disable it by going to Control Panel (the old one) -&gt; Hardware and Sound -&gt; Sound -&gt; Communication and then setting that to "Do nothing", it is on "Reduce by 80% by default" which is kinda weird. Is there any way we could prevent it from counting as communication audio?

Screenshot 2023-04-12 at 12.21.36.png

can you share what you are doing to record them? If using Egress, is it Cloud or OSS (and what version of Egress)

SESSION RM_bNDFVQgLNnBz
1cb3842b-dd4c-4d09-ab1c-e464e1fedadd

<@U01KZ61GV5Z> is this information enough?

yeah, we've managed to reproduce. It seems like it shows up with our default template. looking at why

<@U04T10D1C1K> the scrollbars have been removed

hello <@U051LG9AM6V>, I'm rather curious, are you using the component-js library? I'm asking because a lot of those states are already handled for you. Ref: <https://github.com/livekit/components-js>

track.detach is not supposed to release camera.. that detaches the track from UI, but if it's a local track, it would continue to be published.

you'd want to mute or unpublish the LocalVideoTrack if you are looking to stop it

Also in the LiveKit Unity SDK example project on github, in <https://github.com/livekit/client-sdk-unity-web/blob/main/Samples%7E/ExampleRoom/ExampleRoom.cs|ExampleRoom.cs> I see this comment:

``` IEnumerator Start()
    {
        // New Room must be called when WebGL assembly is loaded
        m_Room = new Room();```
What does this comment actually mean? Does this have something to do with why it isn't working in my project?

hey yeah it def sounds related to the issue you are seeing.. do you have your room creation in an `IEnumerator Start()` block as well?

No, I don't have it in an `IEnumerator Start()` block but rather a different coroutine that is executed later on. Is that a problem?

hah I don't know Unity, but was just reading the comments and thought it was relevant to your error. will defer to <@U022KB72NFJ> on this one.

Hey <@U04DUL22NF5>, are you trying to run your scene inside the editor?

Yes.. This is currently not supported, at the moment, the WebGL SDK needs a browser context to work.
We’re currently working on a Unity Native SDK, this will allow to run LiveKit on the editor and many other platforms

Oh, dang. :confused: That makes implementation difficult. Will the native SDK you mentioned be released soon? In the meantime, what's the recommended way to implement/test/iterate and have things not throw errors when using the app in the Unity Editor?

You can use compiler directive (`UNITY_WEBGL`). I agree that it slows down the development, my priority is to release the Unity NativeSDK, some blockers must be fixed and it should happen soon

When the Native Unity SDK is released, with there be only a single Unity package that includes both the Unity native and Unity web SDKs and uses compiler directives to execute the correct SDK? (Native SDK in Unity Editor &amp; web SDK in a WebGL build?)

As my eventual goal would be to use LiveKit as a cross-platform voice/video solution for any apps I create with Unity, it would be great to use a single package, write the C# code for LiveKit once, and have it work cross-platform (for example, a Unity WebGL user, a Unity Windows user, a native Unity Quest VR (Android) user could all communicate with each other in a cross-platform app)

So, just curious about the roadmap there.

Correct, the goal is to have a single Unity package with the same APIs that works across native and web.

The plan is roughly the following:
1. release Unity for native (there may be some minor API differences between the current WebGL SDK)
2. merge WebGL SDK into it (all with the same API)

That sounds great! Could you give a rough estimate for when this will be released?

So when Phase 1 is complete, would it be possible to have both packages in the same project and rig up some compiler definitions to allow LiveKit to function both in the editor and in a WebGL build? (And allowing WebGL builds to connect with the Unity Editor for voice/video?)

Ok, thanks <@U01KZ61GV5Z> and <@U022KB72NFJ>! By the way, I really appreciate that you and your team are working towards a full Unity integration. I expressed some disappointment earlier about LiveKit not working in the Unity Editor, however I understand where you are at and what your goals are and I am very eager and excited for when the native Unity SDK is released. This is super important for our team since we do a lot of multiplayer testing and iteration inside the Unity Editor and when things don't function the same way they would in a build, it makes things pretty difficult to work with.

We're currently using Agora's Unity WebGL plugin for voice chat, which does work both in the Editor and in a WebGL build. However, their plugin is in beta and I've found it to be rather buggy, and updates to it are fairly infrequent. I would really like to move *everything* over to LiveKit, but I can't do that until LiveKit has a native Unity integration released. That being said, so long as the native Unity SDK is in-progress and releasing fairly soon, I think I am still going to attempt to implement _only_ screen share with LiveKit and see how that goes. In the short-term I can try to write some code to handle things "gracefully" when running in the Unity editor - disabling incoming and outgoing screenshare functionality when using the Unity Editor. It's a bit of a challenge since I'm not just running LiveKit - it'll be tied into Photon PUN2 which is coordinating most of our multiplayer networking. But once I wrap my head around how LiveKit does things I'm hopeful I can come up with a good solution to handle things differently inside the Unity Editor than in a build.

Once the native Unity integration is released, I could then update my code to have screen share also work inside the Unity Editor, and then transition voice chat and any other services over from Agora to LiveKit.

Anyways, I just wanted to type this out because it gives you guys a picture of an actual use case! I've enjoyed reading your blog posts and I admire LiveKit's goals and mission. I hope to use LiveKit in all of my Unity projects once the native Unity SDK is released!

Thank you Matt. Your support means a lot!

We are anxious to release full Unity support and hope we won't disappoint!

I’m digging deeper:

```tt.util is undefined```

hi <@U052ZGLM46N>,
can you give us more context about your build pipeline?

Also: when does this error occur?

and can you enable source maps?

Build pipeline is tricky, yes :slightly_smiling_face:

It’s clojurescript with shadow-cljs. It’s working with all npm UMD packages that I tried so far… I’m trying to figure out what’s root problem here, but without success so far…

&gt; Also: when does this error occur?
during “loading” of module (it uses Google Closure for this)

if you are interested in an investigation (I understand that it’s not very popular combination of tools :slightly_smiling_face: ) I can make a minimal reproducer.

sourcemaps would be the most valuable help right now in order to see where the error actually originates from.

is this a _runtime_ error or does it happen during your _build_ ?

it’s runtime error during dynamic module loading .

I’m trying to get line with applied sourcemaps, again without success so far (I guess because there is “two levels” of source mapping in my environment).

But looks like it’s _m0.util.Long references

aha! the two libraries that impact that are
```import Long from "long";
import _m0 from "protobufjs/minimal";```
it does sound like `_m0.util` is undefined however.. no idea how that would happen.
fyi we do have a rollup script that removes all occurrences of `eval` in the source code and replaces them with `undefined` .
maybe that helps you with finding the culprit!

Thank you. So far I tried to switch to esm, it didn’t help, I’m still getting “TypeError: _m0.util is undefined”

&gt; fyi we do have a rollup script that removes all occurrences of `eval` in the source code and replaces them with `undefined`
thank you for the clue!

rollup (?) generates

```var _m0 = /*@__PURE__*/getDefaultExportFromCjs(minimalExports);```
minimalExports is empty object at this point

maybe you know what is in charge of all this getDefaultExportFromCjs build magic? ) thank you!

<https://github.com/rollup/plugins/blob/master/packages/commonjs/src/proxies.js#L63>

Ok, looks like the issue was fixed here <https://github.com/rollup/plugins/pull/1455>

Updating @rollup/plugin-commonjs to 24.1.0  fixed my issue

Any chance to merge this update into the lib?

sure, we’ll get this in as part of the next release!

thank you! I’m happy so far with my fork with bumped dependency.

BTW, have you deliberately designed client-sdk-js API after Twilio API? Anyway, it helps a lot to switch from Twilio :wink:

you don't need any ports to be opened for Egress

<@U01KZ61GV5Z> I only allowed ssh port as inbound connection in firewall and
Egress stopped sending rtmp recording.

I undo the operation. Now the records show as status 1 in the egress list, when I try to stop egress, I get the "twirp error unavailable: no response from servers" response.

There is no problem in recording new broadcasts, but I cannot cancel old broadcasts even though they are not registered. Should I restart the livekit kubernetes?

Egress communicates with livekit-server via redis. your API calls are reaching livekit server and not Egress instances

Publishers can stream now, I don't think the livekit has a connection problem with redis.

old publishers seem to be deleted from the egress list when they close their broadcasts.

<@U01KZ61GV5Z> I have restarted livekit kubernetes and still old egress log are there.

Screenshot 2023-04-13 at 5.07.19 PM.png

Hey can anyone off any assistance with this error? Thanks

Isaac, see here for a <https://docs.livekit.io/server/egress/room-composite/#stream-to-hls|HLS output example>. and <https://docs.livekit.io/server/egress/#filenames|filename templating>

I’m in the middle of checking generated code, but according to Google Protobuf docs it should be `addFileOutputs` , no?

so yes, setFileOutputs uses util.List.set method in generated code. it means that it should be used only to *replace* file outputs, not add them.

it means that this example <https://docs.livekit.io/server/egress/room-composite/#starting-a-roomcomposite>

can’t work for Java/Kotlin/JVM

In my opinion it’s a bug. Should I report it via github issues or prepare PR?

I think the same is applied to all other outputs.

<@U052ZGLM46N> thanks for bringing this to our attention, just merged a fix and released in 0.5.1 (should be up on maven central in an hour or so)

Thank you. It started to send requests, but now I have a few of other issues… Could someone help me? I’m evaluating LiveKit to switch to it from Twilio, but it seems so far that it isn’t production ready… I understand that it’s new software and we are ready to try it, but looks like it would require help from your side… What do you think?

Issues that I have now:

- .startRoomCompositeEgress.execute.body returns (most of the time) null so I can’t get egress ID. As I understand it returns null if there already exists egress for room? or?
- for some (!) requests it actually creates egress. but livekit-cli list-egress prints `bucket location was nil` as an error and I don’t understand what it means. I set all required fields in Egress requests, including `bucket` and `endpoint`
- livekit-cli list-egress gets/prints “STARTED AT” broken time (`1754-08-30 23:37:09.12865484 +0053 LMT`)

Ok, looks like it doesn’t create new egresses because of Concurrent egress requests limit during trial?

I upgraded a project to paid subscription, but it still doesn’t create egresses…

It looks like “limit 1” is still in effect for our project (<wss://fliptalk-freenhoy.livekit.cloud>)

Hey Andrew, we'll take a look at your project.

I think it probably took a bit of time for the caches to update after you upgraded. Are you still seeing issues? Feel free to DM

It would be helpful to see the Egress ID that you are issuing

so I need to change it to <https://github.com/livekit/protocol.git> before build

I’m also getting bunch of failed tests:

```&gt; Task :test FAILED

EgressServiceClientTest &gt; listEgress() FAILED
    java.lang.IllegalArgumentException at EgressServiceClientTest.kt:25

IngressServiceClientTest &gt; listIngress() FAILED
    java.lang.IllegalArgumentException at IngressServiceClientTest.kt:25

RoomServiceClientTest &gt; listRooms() FAILED
    java.lang.IllegalArgumentException at RoomServiceClientTest.kt:40

RoomServiceClientTest &gt; updateRoomMetadata() FAILED
    java.lang.IllegalArgumentException at RoomServiceClientTest.kt:64

RoomServiceClientTest &gt; getParticipant() FAILED
    java.lang.IllegalArgumentException at RoomServiceClientTest.kt:85

RoomServiceClientTest &gt; listParticipants() FAILED
    java.lang.IllegalArgumentException at RoomServiceClientTest.kt:75

RoomServiceClientTest &gt; sendData() FAILED
    java.lang.IllegalArgumentException at RoomServiceClientTest.kt:90

RoomServiceClientTest &gt; createRoom() FAILED
    java.lang.IllegalArgumentException at RoomServiceClientTest.kt:27

RoomServiceClientTest &gt; deleteRoom() FAILED
    java.lang.IllegalArgumentException at RoomServiceClientTest.kt:51```

that shouldn't be a problem on LiveKit Cloud. total end to end latency should be below 400ms even for distant users. all of our servers are on low-latency private backbones.

Hey, there isn't a way to limit from livekits side. However you can enforce this within your application in publish options.

My application is going to be open-source, the problem here is that people would then be able to use a higher bitrate and suck my Cloud traffic empty. It would be good to have an option to prevent this, is there maybe some webhook I could listen for that tells me the bitrate of their screenshares or cameras?

Are you saying that a user should be free to modify your client code, but at the same time be able to use your keys to connect to LiveKit Cloud?

That doesn't sound like a good idea. TBH

To answer your question, we are not able to surface bitrate of other clients in real time. This is because it purely depends on what the webrtc client decides to do since bitrate is dynamic in webrtc.

i.e. it's perfect valid for a client to start sending 200kbps.. but then to boost bitrate to 4mbps.. there is no way for us to prevent that.

Alright, I didnt know it was that complicated, sorry for asking for something almost impossible. For your question, my Frontend in Flutter is open source while the backend is closed source. I use this approach to be a lot more transparent about what really goes on in the Frontend, so people know what the app they're using does, so there is no security risk there.

Got it, thanks for clarifying. I'm always paranoid about api key/secret leaking to clients :sweat_smile:

Hey <@U053JNGRYRZ> are you using a layout? I think there's a recent change in behavior on our side that if no layout is provided, it doesn't select a default

Oh, I didn't provide layout parameter, I read that it's not required. Thanks, I'll try to provide one and let you know if there is an error. By the way, where can I see a list of all layouts that I can pass as a parameter? I saw only one in examples called 'grid-light' and no more

This is a regression on our part. we changed the default behavior by accident. This will be fixed later today.

we have a few <https://docs.livekit.io/server/egress/room-composite/#default-layouts|default layouts>, and you can always <https://docs.livekit.io/server/egress/custom-template/|create your own>!

Thanks a lot! Providing grid layout parameter fixed black screen :partying_face:

Glad to hear it! sorry for the regression regarding default behavior

hi SOAM Team, does the published resolution look low (can’t reach 1080p)?

I think I found the reason. when simulcast is disabled, the videoEncoding parameter is not passed correctly, so the sending bit rate cannot be as expected.
Please try this branch, <https://github.com/livekit/client-sdk-flutter/pull/270>, I have submitted a fix, it should be able to publish 1080p 30fps correctly, and use the preset bitrate of 3mpbs.

Commits for fixes.
<https://github.com/livekit/client-sdk-flutter/pull/270/commits/589a3ad4afd43b5cb600b1f8ac4f0ea46ac9550d>

Hi <@U02GAC58A79> ,

The problem described below :

Android (publisher) -&gt; any (subscriber) the video quality is excellent.

iPhone X (publisher) -&gt; any (subscriber) the video quality is low.

Android in general works as expected (publisher/subscriber) but iPhone is not as (publisher)

Yes, this is related to the preset encoding maxBitRate not being set correctly(I reproduced it locally, set h1080p_169, but the published video is blurry, after fixing it will work as expected), can you try this branch and tell me the test results

I will <@U02GAC58A79> and thanks for your immediate action!

Hi <@U02GAC58A79>,

The setup dependencies are below:

  `#livekit_client: ^1.2.1`
  `livekit_client:`
    `git:`
      `url: <https://github.com/livekit/client-sdk-flutter.git>`
      `ref: chore/screen-share-capture-options`

We still getting blurry video quality from the iPhone X

hey, can this issue be reproduced with client-sdk-flutter/example? i have an iphone x

953D40A6-178F-4EDB-93EC-9F14BF69D35E.jpeg

E240B1A9-97CC-4583-9A1C-2515D0405C4B.jpeg

We were viewing the video from the web browser

The clear one from the Android (Nexus 6) and the blurry one from iOS
(iPhone X)

oh no,  This resolution is indeed very low, it looks like 192x144

The odd thing that Nexus 6 released on 2014 and doing better
than iPhone X :slightly_smiling_face:

Does the video get clearer when your phone camera is still for a while?

Can you try to test with client-sdk-flutter/example? This is the video I posted with iPhone X and viewed with the macOS client

I should try that out now and let you know

<@U02GAC58A79> What is your camera settings in the iPhone X?

These steps ensure that the correct branch is updated
```flutter clean
rm -rf ios/Pods
rm -rf ios/Podfile.lock
flutter pub get
cd ios &amp;&amp; pod repo update```


I just changed the default resolution to h1080_169

defaultVideoPublishOptions: VideoPublishOptions(
    videoEncoding: VideoParametersPresets.h1080_169.encoding,
    simulcast: false,
),
defaultCameraCaptureOptions: CameraCaptureOptions(
    cameraPosition: CameraPosition.back,
    params: VideoParametersPresets.h1080_169,
 ),
*Our configuration options*

Hi <@U02GAC58A79>

We have tested the client-sdk-flutter/example and still getting blurry video quality.

ah, weird, can you provide more details,  what is the downlink bitrate for the viewer when the video is blurry?

We are testing in local network using dev mode for the the livekit server and connecting to the server via wireless network (firewall disabled).

The token is auto generated via the flutter app for testing.

We will test using other iOS device and let you know about the results

Hi <@U02GAC58A79>

We have just tested publishing from iOS (iPhone 14 Pro Max) with your <https://github.com/livekit/client-sdk-flutter/pull/270|PR> changes.

The result was *excellent*  :+1: and hopefully your <https://github.com/livekit/client-sdk-flutter/pull/270|PR> will be merged soon!

It seems the blurry video quality issue with iPhone X in general or the one that we are using during testing.

Thanks for your help!

2023-04-17_LiveKit_Screenshare_Washed_Out_Firefox.png

2023-04-17_LiveKit_Screenshare_Black_Chrome.png

Also, if it matters, I am using Unity version 2021.3.21f1 with the Universal Rendering Pipeline (URP).

And also, the "washed out" issue may not be exclusive to Firefox - it might be app-wide regardless of the web browser used. But I am not sure yet since I haven't seen it work in Chrome quite yet.

I'm still very early in my implementation, so there's a chance I may not be doing something right. However I would expect to see something in Chrome if it's working in Firefox.

For 1.  the colors to the left appear to be much brighter. are you screen sharing this from another LiveKit client? if so, which one?

For 2. that's rather unexpected. do you have any logs in the console that doesn't look right? i.e. any errors or warnings.

For #1, everything is done in a Unity WebGL build. One Unity WebGL app is screen sharing to another app instance. To me this looks like something could be wrong with the "color space" of the video texture in Unity (or the "format"), but I am not sure as I don't have a full understanding of the internals.

For #2 I don't see anything suspect. There are no errors or warnings in the console log.

I put in a console log for when the video texture is updated. The console log reports the resolution of the video texture and its format. When screen share begins, looks like a video texture is being created 3 times in a row (Chrome):

In Firefox (which works), looks like it's being created 2 times in a row, and maybe these warnings might mean something?

In this test I am sharing my entire screen (a 1920x1080 monitor).

this is helpful. thanks! we'll look through and see if we can figure out the Chrome issue. To be clear, you are screen sharing from FireFox, and it's showing up as blank in Chrome as the viewer?

Thank you, I really appreciate it!

Specifically I am screen sharing from Chrome (using very similar code from the example documentation and code samples), and the video shows up as a black screen in other receiving app instances also on Chrome. The video works in Firefox. Audio works fine on both.

And <@U022KB72NFJ> if there's anything specific that I can do to help please let me know. Happy to help any way I can! I'm sure we can get to the bottom of it. :slightly_smiling_face:

Hey <@U04DUL22NF5>, do you keep a strong reference to the HTMLVideoElement on the C# side?

Can you try to add a console.log()/breakpoint <https://github.com/livekit/client-sdk-unity-web/blob/19fbb8d964c0b47681c1ace55c085327ea07e09e/Runtime/Plugins/livekit-bridge.jslib#L320|here>?

Hey <@U022KB72NFJ>, I added a console log where you mentioned (and also double checked the app build to make sure the console log is actually there) but I don't see it in the browser console. :thinking_face:

I don't believe I'm keeping a strong reference to the HTMLVideoElement in my C# code? I am subscribing to `video.VideoReceived` via a lambda function and only using the texture. Here is my C# code for when a track is subscribed:

```private void OnTrackSubscribed(RemoteTrack track, RemoteTrackPublication publication, RemoteParticipant participant)
{
	if (track.Kind == TrackKind.Video)
	{
		var video = track.Attach() as HTMLVideoElement;

		// VideoReceived is called every time the video resolution changes
		video.VideoReceived += tex =&gt;
		{
			OnGotNewScreenShareVideoTexture(tex);
		};
	}
	else if (track.Kind == TrackKind.Audio)
	{
		track.Attach();
		Debug.Log("Got new audio track!");
	}
}

// This function creates a new material with the video texture, then applies that material onto a target renderer.
private void OnGotNewScreenShareVideoTexture(Texture2D screenShareVideoTexture)
{
	debugTargetRenderer.material = MaterialUtils.CreateNewMaterialUnlitMipMapBias(screenShareVideoTexture);
	Debug.Log("Got new screen share video texture! ("+ screenShareVideoTexture.width + "x" + screenShareVideoTexture.height + " Format: " + screenShareVideoTexture.format.ToString() + ")");
}```

Sounds good, let me know if there's anything else I can help with!

My full test conditions are Windows 10, Unity 2021.3.21f1 + Universal Rendering Pipeline (URP). Testing 3 instances of the same WebGL build at the same time (hosting the app on an Amazon AWS S3 storage bucket). Two of the instances are running Chrome, and one instance is running Firefox. I initiate the screen share from one of the Chrome instances. Then for the two other instances, I see a black screen on the receiving Chrome instance, and the receiving Firefox instance I do see the screen share.

Hi <@U022KB72NFJ>, were you able to reproduce the issue?

Screenshot 2023-04-19 at 11.14.40 PM.png

Hey I just tested using 2 chromes instances and I can’t reproduce the issue. I used the <https://github.com/livekit/client-sdk-unity-web/tree/main/Samples~/ExampleRoom|example app> inside the sdk repo.
I’m using Unity 2021.3.23f1.
Can you try to run this demo and see if you have the issue?
For the difference between the color space, can you try to  change the second boolean of this <https://github.com/livekit/client-sdk-unity-web/blob/main/Runtime/Scripts/Internal/HTMLVideoElement.cs|line> to `false` .

The colors seem normal for me but I maybe need a way to change the color space depending on some conditions.

I’m using my camera instead of screenshares but there is no difference on how I handle the texture creation on the receiving side.

On the Chrome console of the video receiver, do you see any WebGL errors or warnings?
Maybe something related to texImage2d

I tried on my Macbook, I will retry to reproduce the issue on my Windows machine tomorrow

Hey <@U022KB72NFJ>, thank you so much for the update! I will do as you mention and investigate further via the example app, then report back.

Could you tell me which line to change in `HTMLVideoElement.cs` , specifically? Looks like your link didn't point me to it.

I do not see any errors or warning in the browser console on Chrome for the receiver app - only a black screen.

My app differs from the example app in that I'm:
1. Creating a new unlit material (duplicating it from an existing "template" unlit material)
2. Applying the video texture to the material
3. Applying the material to the renderer of a 3D "screen" object in my scene (not using a `Raw Image` or Unity's canvas system at all).
So I would recommend trying this when attempting to reproduce the issue.

Please let me know what you find out on your Windows machine! If you still aren't able to reproduce the issue after testing on Windows via the method I described above, then I should create and send over a project with the reproducible issue for you to use in order to look into it further.

<@U04DUL22NF5> I think Theo meant <https://github.com/livekit/client-sdk-unity-web/blob/main/Runtime/Scripts/Internal/HTMLVideoElement.cs#L80|this line>, where it's creating a texture.. the parameter indicates
```linear	Is texture using linear color space?```

Hey <@U022KB72NFJ>, the example app also has the same "black screen" issue on Google Chrome. Here is a screenshot - Chrome is on the left side (black screen) and Firefox is on the right side (working):

I would recommend trying to reproduce on Windows 10, using Unity 2021.3.21f1

Please let me know if the same thing happens during your testing. If you are still unable to reproduce the issue on your end given those conditions, I could then create and send over a project with the reproducible issue for you. I'm hopeful we can get to the bottom of it together - this is currently the main issue barring us from using LiveKit since a vast majority of our users are on Chrome.

<@U022KB72NFJ> I have just confirmed with another colleague that this issue is also reproducible on their system. They are running Windows 11 and get a black screen for all video feeds on Chrome, and it works on Firefox.

Here is a build you can test yourself if you'd like: <https://rem5-unitywebgl-test.s3.us-east-2.amazonaws.com/LiveKit/Simulacra_LiveKitExampleTest_2023-04-20-001/index.html>

Ok this is nice, I’m going to debug, thank you :slightly_smiling_face:

Great! Please keep me updated about your findings. I am happy to help any way I can to get to the bottom of it!

Hey, when building the project locally, I can’t reproduce the issue. This is weird..
But I think I know what happens, I created a new branch on the repo called `theo/blackvideo` (<https://github.com/livekit/client-sdk-unity-web/commit/0d8f993eba8e5997571f3bae99587335998dba0e|link>)
It seems that the video does not start automatically.
Can you try it?

<@U022KB72NFJ> to test I simply just need to modify the `livekit-bridge.jslib` per your link and rebuild?

I confirm that when I use your link and retrieve the corresponding HTMLVideoElement, just calling `video.play()` fixes the issue

Hey <@U022KB72NFJ>, I'm happy to report that the code change to `livekit-bridge.jslib` that you provided appears to fix the issue :slightly_smiling_face: Left side is Chrome, right side is Firefox - both working now:

This also got screen share to work in my test scene with the screen being a 3D model in world space.

And the video preview now starts right away in Chrome, whereas it did not prior to this change.

Ok nice, I’m going to merge it to main, did you get a chance to test the difference between color spaces when editing this <https://github.com/livekit/client-sdk-unity-web/blob/main/Runtime/Scripts/Internal/HTMLVideoElement.cs#L80|line>?

I have not tested that yet - I will try this out next!

Thank you so much for being attentive to these issues - our team really appreciates it! :slightly_smiling_face: I will report back my findings once finished testing this out.

2023-04-20_LiveKit_Screenshare_Washed_Out_001.png

Hey <@U022KB72NFJ>, I've gone ahead and changed the second boolean to false in line 80 of `HTMLVideoElement.cs` . Unfortunately the result appears to be the exact same regardless of whether or not the boolean is set to true or false. Here's an image showing the "washed out" look without any changes:

2023-04-20_LiveKit_Screenshare_Washed_Out_002.png

And here is an image with the boolean changed to false:

I will direct message you a hosted app build that you can try out to observe this yourself...

SMPTE_Color_Bars_16x9.svg.png

Hey <@U022KB72NFJ> I have just sent you a DM for a build that you can try out to see the issue. Here's the test pattern image I'm using to verify correct colors also:

Hey <@U022KB72NFJ> just wanted to check in. Are you able to observe the same "washed out" look that we are seeing, via the build I sent you?

Hey <@U04DUL22NF5> I haven’t looked yet, I’ll check that in a bit

2023-04-21_LiveKit_Screenshare_Washed_Out_003.png

<@U022KB72NFJ> Here is one more example of the washed-out look, with an actual video instead of a test pattern:

Oh I just answered you in DM :slightly_smiling_face:

So yes, I just reproduced the issue, I’ll fix that :smile:

I have just discovered the cause of the issue, I will create a fix once I am on my Windows computer.
Thank you very much for reporting your issues :pray:

<@U022KB72NFJ> as long as you're updating things, looks like the Init() method in CopyPaste.cs might need to get wrapped into a compiler directive (or maybe even that whole script's code) so it doesn't run in the Unity Editor. I get this error when entering playmode without having any LiveKit logic in the scene:

Hey, I just fixed the color space on this <https://github.com/livekit/client-sdk-unity-web/pull/33|PR>. Can you give it a try?
I’ll create another PR for the CopyPaste.cs :slightly_smiling_face:

CopyPaste <https://github.com/livekit/client-sdk-unity-web/pull/34|PR>

Thank you <@U022KB72NFJ>! I will take a look as soon as I'm able today and let you know the result!

Hey <@U022KB72NFJ>, good news and bad news. The good news is that this does fix the problem with the "washed out" look - the colors look correct after applying your <https://github.com/livekit/client-sdk-unity-web/pull/33|PR>. The bad news is that for some reason this causes a _massive_ drop in performance. On a high-performance Windows PC the framerate drops from a smooth 60 fps down to around 12 fps when I begin screen sharing. I'll send you another build that I just made via DM so that you can compare the updated build against the previous build I sent you in terms of performance.

I would unfortunately not recommend pushing this update to main until we've discovered the cause of the drop in performance due to the color space update. :confused:

Actually, I have run into a very similar (perhaps identical) issue with a different plugin called <https://assetstore.unity.com/packages/tools/video/video-player-webgl-192420|Video Player WebGL>. This plugin also has a "washed out" look to the video color - same as the issue with LiveKit Unity WebGL. Awhile back, I emailed the developer about it and they said to change a line in their code dealing with the color space from:

`GLctx.texImage2D(GLctx.TEXTURE_2D, 0, GLctx.RGBA, GLctx.RGBA, GLctx.UNSIGNED_BYTE, video);`
to
`GLctx.texImage2D(GLctx.TEXTURE_2D, 0, GLctx.SRGB8_ALPHA8, GLctx.RGBA, GLctx.UNSIGNED_BYTE, video);`

which also resulted in a large drop in performance. This appears to be the exact same change that you made in `livekit-bridge.jslib` and resulting in the same exact performance issue. I don't believe a solve was ever figured out for the "Video Player WebGL" plugin and therefore the above change was never implemented in that plugin, but also the issue simply wasn't investigated further so I don't know the cause.

But it would appear that changing the color format from `RGBA`  to `SRGB8_ALPHA8` in the function `GLctx.SRGB8_ALPHA8` is somehow causing a very bad drop in frame rate of the app.

Here is an image where you can see the frames-per-second in the upper right corner. When I am screen sharing (top image), the color now looks correct but the FPS drops to 12 FPS. When I stop screen sharing (lower image) the FPS goes back up to 60 fps. This issue was not present prior to changing `RGBA`  to `SRGB8_ALPHA8` :

that's a great find Matt! I think the issue might be that `SRGB8_ALPHA8` isn't hardware accelerated.

I wonder if there's a way to avoid that and stick to RGBA space to begin with?

I just revert the PR, yes it seems like Chrome is sending the RGBA buffer to the CPU to do the SRGB to RGB conversion and then reuploading it to the GPU.
This is tricky, I’ll investigate further, thanks for the analysis

Thank you so much for investigating - please keep me updated and if there's anything you'd like me to test please let me know! :slightly_smiling_face:

Screenshot 2023-04-24 at 11.21.21 PM.png

<@U04DUL22NF5> Inside the BuildSettings, can you try these settings?
Make sure to remove the patch I made for the colorspaces (main branch on gh atm)

Uh, webgl 1.0 is already removed on the screenshot.
Also note that webgl 1.0 is deprecated, so I don’t think this is a big deal

These are my current settings that have been in builds thus far (they match your screenshot above):

mmh, can you try switching to gamma (just for me to understand how Unity works with linear textures)

Sure, you just need me to share a build with you that uses gamma color space?

Yes, it can be great :pray:
Or if ever, just check if the colors are now normal on the screenshares

Ok, working on it... The Unity project has to re-import all materials/shaders in order to change the color space and our project is rather large, so it's taking awhile. :sweat_smile: I cannot permanently change our project to gamma of course, because we've done our materials and lighting using the <https://docs.unity3d.com/Manual/LinearRendering-LinearOrGammaWorkflow.html|linear color space workflow> (which I believe is the default and recommended option in Unity) but I can certainly temporarily change it for you, for test purposes.

<@U022KB72NFJ> I've sent you another app build to try with the Gamma color space. With the color space set to Gamma in the Unity project settings, the color of the screen share video looks correct:

I've now reverted my own Unity project back to Linear color space because we need to use Linear (in general Gamma is now probably considered a "legacy" mode), but hopefully this helps!

If you need me to switch the color space mode again for test purposes, that should be more easy now. Unity appears to cache the imported shaders/materials, so there isn't another long import after the first time switching it.

Hey <@U022KB72NFJ>, I have discovered an in-Unity workaround to convert the texture to the correct colorspace via a shader:

This relies on Unity's Shadergraph system and the <https://docs.unity3d.com/Packages/com.unity.shadergraph@6.9/manual/Colorspace-Conversion-Node.html|Colorspace Conversion Node>.

Yes, this is a perfectly valid solution, but it is still a bug in our SDK because we should follow the project settings.

I’ll still fix that in the future, glad that you found a workaround :raised_hands:

But yes I agree, it should be adjusted correctly before it reaches Unity.