LiveKit Community

hey <@U04LRBN31L4>, thanks for helping with the debug, we should try to release a new version in a day or two (with some other fixes)

are you using the same token for both users?

no different tokens
actually it's working in one condition
 if the remote user already joined then it's not rendering the video but
if the remote user joined after me then it's working fine

also how to show the local participant ?
my video is not showing to the remote user
also i need to show local participant video in app.
(permissions are given) <@U01NZKTNE1H>

ah, after connecting add this code snippet:

```            // Connect to server.
            room.connect(
                url,
                token,
            )

            // Attach video of remote participant if already available.
            val remoteVideoTrack = room.remoteParticipants.values.firstOrNull()
                ?.getTrackPublication(Track.Source.CAMERA)
                ?.track as? VideoTrack

            if (remoteVideoTrack != null) {
                attachVideo(remoteVideoTrack)
            }```

for local video, you can use room.localParticipant and get the track publication the same way, and then attach that to a renderer view

hi <@U01NZKTNE1H> it's resolved
my video local video is visible for the other end user (who is using playground)

but in android app i'm not able to render both remote and local video at the same time

only i'm able to render any one of the video.

what maybe the issue or do i missing something?

Yes I've 2 renderer views but only one is rendering.

Did you setup the local renderer? <https://github.com/livekit/client-sdk-android/blob/main/sample-app-basic/src/main/java/io/livekit/android/sample/basic/MainActivity.kt#L34|https://github.com/livekit/client-sdk-android/blob/main/sample-app-basic/src/main/java/io/livekit/android/sample/basic/MainActivity.kt#L34>

```&lt;?xml version="1.0" encoding="utf-8"?&gt;
&lt;androidx.constraintlayout.widget.ConstraintLayout xmlns:android="<http://schemas.android.com/apk/res/android>"
    xmlns:tools="<http://schemas.android.com/tools>"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:background="@color/darkbg"
    tools:context=".ui.completeProfile.VideoCallFragment"&gt;

    &lt;FrameLayout
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:weightSum="2"
        android:orientation="vertical"&gt;
        &lt;io.livekit.android.renderer.SurfaceViewRenderer
            android:id="@+id/renderer"
            android:layout_width="match_parent"
            android:layout_height="@dimen/_300sdp" /&gt;

        &lt;io.livekit.android.renderer.SurfaceViewRenderer
            android:id="@+id/localRenderer"
            android:layout_width="match_parent"
            android:layout_marginTop="@dimen/_300sdp"
            android:layout_height="match_parent" /&gt;

        &lt;ProgressBar
            android:id="@+id/progress"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_gravity="center" /&gt;


    &lt;/FrameLayout&gt;

&lt;/androidx.constraintlayout.widget.ConstraintLayout&gt;```
this is my Xml file code

```package inspot.app.ui.completeProfile

import android.Manifest
import android.content.pm.PackageManager
import android.os.Bundle
import android.util.Log
import androidx.fragment.app.Fragment
import android.view.LayoutInflater
import android.view.View
import android.view.ViewGroup
import android.widget.Toast
import androidx.activity.result.contract.ActivityResultContracts
import androidx.core.content.ContextCompat
import androidx.lifecycle.lifecycleScope
import com.mapbox.maps.logD
import inspot.app.databinding.FragmentVideoCallBinding
import io.livekit.android.LiveKit
import io.livekit.android.RoomOptions
import io.livekit.android.events.RoomEvent
import io.livekit.android.events.collect
import io.livekit.android.room.Room
import io.livekit.android.room.participant.AudioTrackPublishDefaults
import io.livekit.android.room.participant.Participant
import io.livekit.android.room.participant.VideoTrackPublishDefaults
import io.livekit.android.room.track.*
import io.livekit.android.util.flow
import kotlinx.coroutines.flow.collectLatest
import kotlinx.coroutines.flow.map
import kotlinx.coroutines.launch

class VideoCallFragment : Fragment() {
    private lateinit var binding: FragmentVideoCallBinding
    lateinit var room: Room

    override fun onCreateView(
        inflater: LayoutInflater, container: ViewGroup?, savedInstanceState: Bundle?
    ): View? {
        // Inflate the layout for this fragment
        binding = FragmentVideoCallBinding.inflate(inflater, container, false)


        // option 1: set room defaults
        val options = RoomOptions(
            audioTrackCaptureDefaults = LocalAudioTrackOptions(
                noiseSuppression = true,
                echoCancellation = true,
                autoGainControl = true,
                highPassFilter = true,
                typingNoiseDetection = true,
            ),
            videoTrackCaptureDefaults = LocalVideoTrackOptions(
                deviceId = "",
                position = CameraPosition.FRONT,
                captureParams = VideoPreset169.HD.capture,
            ),
            audioTrackPublishDefaults = AudioTrackPublishDefaults(
                audioBitrate = 20_000,
                dtx = true,
            ),
            videoTrackPublishDefaults = VideoTrackPublishDefaults(
                videoEncoding = VideoPreset169.HD.encoding,
            )
        )



        room = activity?.applicationContext?.let { LiveKit.create(it, options) }!!
        // Setup the video renderer
        room.initVideoRenderer(binding.renderer)

        requestNeededPermissions { connectToRoom() }


        return binding.root
    }


    private fun connectToRoom() {

        val arg = arguments
        val url = arg?.getString("URL")
        val token = arg?.getString("TOKEN")
        lifecycleScope.launch {


            // Connect to server.
            room.connect(
                url!!,
                token!!,
            )

            // Turn on audio/video recording.
            val localParticipant = room.localParticipant
            localParticipant.setMicrophoneEnabled(true)
            localParticipant.setCameraEnabled(true)


            // Attach video of remote participant if already available.
            val remoteVideoTrack = room.remoteParticipants.values.firstOrNull()
                ?.getTrackPublication(Track.Source.CAMERA)
                ?.track as? VideoTrack

            if (remoteVideoTrack != null) {
                attachVideo(remoteVideoTrack)
            }


            val videoTrack = room.localParticipant.getTrackPublication(Track.Source.CAMERA)
                ?.track as? LocalVideoTrack
            if (videoTrack != null) {
                attachLocalVideo(videoTrack)
            }


            launch {
                room.events.collect { event -&gt;

                    when (event) {
                        is RoomEvent.TrackSubscribed -&gt; onTrackSubscribed(event)
                        else -&gt; {
                            Log.d("TAG", "connectToRoom: ${event} ")

                        }
                    }
                }
            }
        }

    }

    private fun onTrackSubscribed(event: RoomEvent.TrackSubscribed) {
        val track = event.track
        if (track is VideoTrack) {
            attachVideo(track)
        }
    }

    private fun attachVideo(videoTrack: VideoTrack) {
        videoTrack.addRenderer(binding.renderer)
        binding.progress.visibility = View.GONE
    }

    private fun attachLocalVideo(videoTrack: LocalVideoTrack) {
        Log.d("TAG", "attachLocalVideo: $videoTrack")
        Log.d("TAG", "attachLocalVideo: ${videoTrack.dimensions}")
        videoTrack.addRenderer(binding.localRenderer)
        binding.progress.visibility = View.GONE
    }

    private fun requestNeededPermissions(onHasPermissions: () -&gt; Unit) {
        val requestPermissionLauncher =
            registerForActivityResult(ActivityResultContracts.RequestMultiplePermissions()) { grants -&gt;
                var hasDenied = false
                // Check if any permissions weren't granted.
                for (grant in grants.entries) {
                    if (!grant.value) {
                        Toast.makeText(
                            requireContext(),
                            "Missing permission: ${grant.key}",
                            Toast.LENGTH_SHORT
                        ).show()

                        hasDenied = true
                    }
                }

                if (!hasDenied) {
                    onHasPermissions()
                }
            }

        // Assemble the needed permissions to request
        val neededPermissions = listOf(Manifest.permission.RECORD_AUDIO, Manifest.permission.CAMERA)
            .filter {
                ContextCompat.checkSelfPermission(
                    requireContext(),
                    it
                ) == PackageManager.PERMISSION_DENIED
            }
            .toTypedArray()

        if (neededPermissions.isNotEmpty()) {
            requestPermissionLauncher.launch(neededPermissions)
        } else {
            onHasPermissions()
        }
    }

    override fun onDestroy() {
        super.onDestroy()
        room.disconnect()
    }
}```
this is the kotlin code <@U01NZKTNE1H>

Missing the local renderer initialization

thanks for you support <@U01NZKTNE1H>
it's working.

this largely depends on what your app is doing. how many video tiles are each user subscribing to?

are you using adaptiveStream or dynacast on the client side?

<@U01KZ61GV5Z> Each user is subscribed to all accessor's videos.
In this test, less than 30 out of 120 users had video turned on
The client uses all of adaptiveStream, dynacast, and simulcast.

that's quite a lot of data for the server to forward. i.e.

```30 video tiles * 120 users * 1.7Mbps (for 720p video) = 6Gbps```
also adding the workload to forward audio tracks for 100 people, the server would end up performing a ton of work to support that.

do you require everyone's video to be shown? we are making some optimizations on our side that would make this more performant (mostly by reducing the amount of video that's transmitted, but in an automatic way).

<@U01KZ61GV5Z> Thank you for answer:slightly_smiling_face:

you can call `setLogLevel('debug')` which gives you the client-sdk-js debug logs

image.png

some logs of module cannot print at console. I am working on it, like the pic below

use console.log can see this log, bug log.debug not.

add this line, sdk debug dumped. Should it be a merge request? <@U01KZ61GV5Z>

image_2023_02_15T05_56_20_811Z-4.png

Hey <@U045CJPACG1> — just for clarity, is this Cloud or self-hosted?

self hosted - k8s OSS with livekit-server v1.3.3 and egress v1.5.3

that looks like a database error, but self-hosted uses redis and we never save anything called `recordingMarks`
<@U01KZ61GV5Z> any ideas?

what API did you call that returned this error? it doesn't look like the error is ours

<@U04CAHEBH9R> can you share further details on this ?

I believe both things are true. The room is automatically ended when the last participant has left, but this doesn’t happen right away. cc <@U01KZ61GV5Z>

EmptyTimeout is set to timeout if no one joins the room. Once a participant joins, empty timeout no longer applies. It's not possible to hold the room open once the last participant leaves

Thank for the answer, I get it. But how to keep room alive for a few minutes, to let streamer reconnect ? When room closed, all our egresses closed either, so we lost following HLS, MP4 and other recordings

are you saying the streamer has lost connectivity to the room? and this is a case that no other viewers are currently present?

Yes, there is no other viewers in the room, because we temporary use egress generated hls to show stream for our users. So technically no users in the room

you need to build `client-sdk-js` first with `yarn build`

I’m using the latest livekit and egress docker image

sure
```curl --location '127.0.0.1:7880/twirp/livekit.Egress/StartTrackEgress' \
--header 'Authorization: Bearer ********' \
--header 'Content-Type: application/json' \
--data '{
    "room_name": "my-first-room",
    "track_id": "TR_VCCPz9NASGgBKL",
    "file": {
        "filepath": "egress/track-test-{room_name}-{room_id}-{track_type}-{track_id}-{utc}.mp4",
        "s3": {
            "access_key": "********",
            "secret": "********",
            "region": "eu-central-1",
            "bucket": "egress-test"
        }
    }
}'```

Request looks ok, what egress version are you on? This was an issue a while back, but I haven't seen it recently

It should be able to handle resolution changes like that. I’ll have to look into it

I found the issue here. This is fixed in 1.5.6, which will be released next week once we have time to test

it’s fine to generate a new token, why do you ask?

@livekit/components-react don’t have onError callback for reconnection :disappointed:

<https://github.com/livekit/client-sdk-js>

We were using the @livekit/react-components, however, sometimes the connections fail and we would like to retry, however there’s no callback for that. I looked and you are developing a new react  components lib that offers that. There’s no way to trigger a connection if it fails.

<@U04DM2NALT0> This is the direction for components - <https://livekit-users.slack.com/archives/C01KZ61HZRR/p1674158758259689>

cc: <@U02EU2ZPTQU>

hi Raphael, the components-react are still in active development, do you want to open a GH issue about what exactly you would like to see? then we can take look at adding it in

Sure <@U02EU2ZPTQU>, it’s related with this issue:
<https://github.com/livekit/livekit/issues/1299#issuecomment-1433134895>
I am trying to implement a reconnection mechanism

However there’s no way to catch the connection error using the current react component

Acessing the room using the sdk is possible, but the component get lost. Shall I open an issue on rect-components?

that would be great, yes! That way it doesn’t get lost in slack :slightly_smiling_face:

<@U04DM2NALT0> I just double checked and there is an `onError` callback on the LiveKitRoom component already! Does that not work for you?

Screenshot 2023-02-17 at 16.06.00.png

Hey <@U02EU2ZPTQU>:
“@livekit/react-components”: “^1.1.0",

or better: <https://github.com/livekit/livekit-react/blob/master/packages/components/src/LiveKitRoom.tsx>

Ah, I thought you were using the new one’s already! we’ll have to deprecate the old ones soon to avoid this confusion

the name is quite confusing with the new one hehe..

yeah, agreed! it’s not ideal for having less confusion…

well.. expose onDisconnect should be quite easy  or even building a retry mechanism should not be an hard task I believe

are you interested to add this retry and leave disabled by default for backwards compatibility?

does
``` /** specifies how often an initial join connection is allowed to retry (only applicable if server is not reachable) */
  maxRetries: number;```
solve your issue maybe?
it’s a setting on the `RoomConnectOptions`

No.. this retry is in case the wss is not reachable

but if wss is reachable and the connection of the peers is not stablished, the connection state goes to disconnected with a error available

I would need to unmount and mount the entire LiveKitRoom component again to try to establish a new connection

How to make `livekit component` think the browser  is not a publisher from `RoomPage.tsx` ( do not use local participant)

there’s an audio only example in the nextjs example. It basically renders some volume bars

Oh ok! I will check on it as soon as i can 

here you go <@U03QLD2TNGK>:
<https://github.com/livekit/components-js/blob/main/packages/react/src/components/participant/AudioVisualizer.tsx>

to add to this, the component where the visualizer is being used is <https://github.com/livekit/components-js/blob/main/packages/react/src/prefabs/ParticipantAudioTile.tsx>

Thank you guys so much! :pray::skin-tone-2: 

Hmm, so usually if all participants disconnect the room is closed and recordings end.

What do you mean by “white screen of empty room recordings”? This sounds like maybe an egress bug.

I’ll defer to <@U01UB0LHTTQ> for options about suspending/recording to the same file, but I don’t _think_ we support that

There is no way to pause and resume egress currently, you would need to stop and then start a new one

it's mounted as a volume in the helm chart <https://github.com/livekit/livekit-helm/blob/master/livekit-server/templates/deployment.yaml#L109|here>.

Ohh I didn’t see that, I found out how it works now. Thanks for the hint :blush:

no problem! glad it's working for you :slightly_smiling_face:

I think this might be a question for <@U02EU2ZPTQU> :slightly_smiling_face:

Hey <@U032TUAL4KT>, thanks for your question. A big motivation for the new package is that we are trying to abstract away the need to manage callbacks yourself. So your question is quite interesting for us as it seems that we have not provided/documented all the APIs for you to achieve your goal. Would you mind sharing what you are trying to achieve and which RoomEvents or RoomState you are interested in?

Hi, using the LiveKitRoom component, i'm simply looking to act based on a specific message received. Usually you'd hook into the the connect and provide some callbacks for achieving this. But I don't think it is possible through that component.

<@U03RM7G8Q6A> With this said, maybe I've missed something and there's an easy way to work through this.

Currently, we only provide these <https://github.com/livekit/components-js/blob/959b8ceb98960fc549bcd73f427bc08849df1e72/packages/react/src/components/LiveKitRoom.tsx#L68-L70|three callbacks on the LiveKitRoom> component. Would this solve your use case or would you like to listen to a different RoomEvent?

<@U032TUAL4KT> with "specific message" do you mean a data channel message or a room event?

Hello <@U02EU2ZPTQU>, Yes that is correct.

Similar to what you guys have put together around that Chat Comp.

ah, perfect, we have a simple way to achieve that using the `useDataChannels()` hook! 

Ah so many events, glad we found the right one :D

Oh, darn, looks like I had missed it per say.

Is that being used by any of the examples?

<https://github.com/livekit/components-js/search?q=useDataChannels> I cannot seem to find anything around that though.

oh.. maybe you are talking about: <https://github.com/livekit/components-js/blob/main/packages/react/src/hooks/dataChannel-hooks.ts>

sorry, yes wrong name :face_palm: 
<https://github.com/livekit/components-js/search?q=useDatachannelMessages&amp;type=|https://github.com/livekit/components-js/search?q=useDatachannelMessages&amp;type=>

Currently there's no example using it. The chat component makes use of this hook internally, but is probably a more advanced usage example ... we'll have to add a minimal one!

Thanks <@U02EU2ZPTQU>! Appreciate the help and hopefully this might be helpful in the future for anyone else :slightly_smiling_face:

Hey <@U02EU2ZPTQU>, I was curious.. the ChannelId, this can be anything right?

that can be anything yes, it just helps you to differentiate different message types from the start. 
the hook is generic, you can call it with a type annotation in order to get full typescript support `useDataChannelMessages&lt;MyMessageType&gt;('mychannel')`

Let me implement this and Ill keep you posted if I'm running into any issues. Once again, thank you

I cannot seem to get this working. Shouldn't the `message`  prop, be `messages` so we can map through those or something?

<@U02EU2ZPTQU> Is there a new Data Message structure to pass to get this working with useDataChannel? Because it seems like the `isMessageType()` doesn't seem to match at any point because my GO Participant is not sending a specific type.

I've been going through the docs here: <https://docs.livekit.io/client/data-messages/> and I dont see anything about channel id anywhere.

Okay, I think I now understand, the React Component, works with a Hash being attached to the message payload, but none of the other SDKs do, hence why I cannot parse any of my message.

you're right, we'll have to change it to work with other sdks as well! 
thanks for the feedback, that was an oversight while designing that API :pray: 

<@U04QP4K56TT> If you are using iOS, you can refer to this readme document <https://github.com/livekit/client-sdk-flutter#ios>, try adding these lines to Info.plist
```xml
&lt;key&gt;UIBackgroundModes&lt;/key&gt;
&lt;array&gt;
     &lt;string&gt;audio&lt;/string&gt;
     &lt;string&gt;voip&lt;/string&gt;
&lt;/array&gt;
```
The Android sdk should be able sending video/audio after being minimized, but limited by the android  power management, when the phone goes black or sleeps, the livekit sdk will interrupt the sending

<@U02GAC58A79> any progress? how can i stop capture?

```D/EGL_emulation(14754): app_time_stats: avg=88.85ms min=80.63ms max=102.25ms count=12
D/EGL_emulation(14754): app_time_stats: avg=88.81ms min=87.86ms max=89.85ms count=12
D/EGL_emulation(14754): app_time_stats: avg=67.87ms min=15.92ms max=100.79ms count=15
I/org.webrtc.Logging(14754): CameraStatistics: Camera fps: 12.
W/FlutterWebRTCPlugin(14754): FlutterRTCVideoRenderer.setVideoTrack, set video track to null
I/flutter (14754): log dispose
I/org.webrtc.Logging(14754): EglRenderer: Releasing.
I/org.webrtc.Logging(14754): GlShader: Deleting shader.
I/org.webrtc.Logging(14754): EglRenderer: eglBase detach and release.
I/org.webrtc.Logging(14754): EglRenderer: Releasing done.
I/org.webrtc.Logging(14754): EglRenderer: Quitting render thread.
I/org.webrtc.Logging(14754): CameraStatistics: Camera fps: 11.
D/EGL_emulation(14754): app_time_stats: avg=139.53ms min=11.32ms max=2344.85ms count=19
I/org.webrtc.Logging(14754): CameraStatistics: Camera fps: 12.
I/org.webrtc.Logging(14754): CameraStatistics: Camera fps: 11.
D/EGL_emulation(14754): app_time_stats: avg=79.94ms min=12.93ms max=3183.67ms count=50
I/org.webrtc.Logging(14754): CameraStatistics: Camera fps: 12.```
I put a log in dispose function. As you can see CameraStatistics is doesn't stop.

screen-sharing-turns-into-camera-room-RM_nEPzGKTJgwkt.log

Hello <@U03QU9Q29C2>, to make it easier to help out, which SDK are you using, if I may ask?

Sorry I forgot to mention that.
We are using the JavaScript `livekit-client`  version `1.6.3`.

We use official docker image `livekit/livekit-server:v1.3.4` as server

<@U03QU9Q29C2> That is strange.

Just picking out the track specific logs, I can see that `190` publishes `CAMERA` and `SCREEN_SHARE` as you can see in `pending track added` logs.
• Camera is TR_VC8K7V9biHa9kv
• Screen share is TR_VSTkU8ftzZZAop
But, `mediaTrack published` (this is when the server detects a track sees media packets from a track) says that the camera track ID is published 5 times (3 simulcast layers for camera and 2 simulcast layers for screen share). Have not encountered that before.

Is this reproducible? If yes, can you please do a couple of things?
1. Set server log level at DEBUG and share the logs.
2. Share code snippet on how you are publishing.
cc: <@U02EU2ZPTQU> <@U03RM7G8Q6A> for any potential front end gotchas.

```2023-02-20T13:14:12.694Z	INFO	livekit	rtc/participant.go:1445	pending track added	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "186", "pID": "PA_3cXLDQt7jNzs", "remote": false, "trackID": "TR_AMDNU39PKGmVV5", "track": "sid:\"TR_AMDNU39PKGmVV5\" source:MICROPHONE", "request": "cid:\"b54ff50d-952c-47cb-8268-a08165ead735\" source:MICROPHONE"}
2023-02-20T13:14:12.694Z	INFO	livekit	rtc/participant.go:1445	pending track added	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "186", "pID": "PA_3cXLDQt7jNzs", "remote": false, "trackID": "TR_VCG2WW36HkSpJ7", "track": "sid:\"TR_VCG2WW36HkSpJ7\" type:VIDEO width:1280 height:720 source:CAMERA layers:{width:320 height:180 bitrate:120000} layers:{quality:MEDIUM width:640 height:360 bitrate:300000} layers:{quality:HIGH width:1280 height:720 bitrate:1700000}", "request": "cid:\"5fa764d0-b7ea-4bb3-b8e2-1c83e8b10453\" type:VIDEO width:1280 height:720 source:CAMERA layers:{width:320 height:180 bitrate:120000} layers:{quality:MEDIUM width:640 height:360 bitrate:300000} layers:{quality:HIGH width:1280 height:720 bitrate:1700000}"}
2023-02-20T13:14:13.340Z	INFO	livekit	rtc/participant.go:1107	mediaTrack published	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "186", "pID": "PA_3cXLDQt7jNzs", "remote": false, "kind": "audio", "trackID": "TR_AMDNU39PKGmVV5", "rid": "", "SSRC": 3215829651, "mime": "audio/red"}
2023-02-20T13:14:13.345Z	INFO	livekit	rtc/participant.go:1107	mediaTrack published	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "186", "pID": "PA_3cXLDQt7jNzs", "remote": false, "kind": "video", "trackID": "TR_VCG2WW36HkSpJ7", "rid": "q", "SSRC": 859028711, "mime": "video/VP8"}
2023-02-20T13:14:13.360Z	INFO	livekit	rtc/participant.go:1107	mediaTrack published	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "186", "pID": "PA_3cXLDQt7jNzs", "remote": false, "kind": "video", "trackID": "TR_VCG2WW36HkSpJ7", "rid": "h", "SSRC": 2429961750, "mime": "video/VP8"}
2023-02-20T13:14:13.762Z	INFO	livekit	rtc/participant.go:1107	mediaTrack published	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "186", "pID": "PA_3cXLDQt7jNzs", "remote": false, "kind": "video", "trackID": "TR_VCG2WW36HkSpJ7", "rid": "f", "SSRC": 2292411111, "mime": "video/VP8"}
2023-02-20T13:14:56.647Z	INFO	livekit	rtc/participant.go:1445	pending track added	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "192", "pID": "PA_4KFGJZixsmGw", "remote": false, "trackID": "TR_AMTtuyobuZe22u", "track": "sid:\"TR_AMTtuyobuZe22u\" source:MICROPHONE", "request": "cid:\"8cee53fc-e41b-4505-8a1f-094ddb3f7f54\" source:MICROPHONE"}
2023-02-20T13:14:56.655Z	INFO	livekit	rtc/participant.go:1445	pending track added	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "192", "pID": "PA_4KFGJZixsmGw", "remote": false, "trackID": "TR_VCJFKmx2GAM9bg", "track": "sid:\"TR_VCJFKmx2GAM9bg\" type:VIDEO width:1280 height:720 source:CAMERA layers:{width:320 height:180 bitrate:120000} layers:{quality:MEDIUM width:640 height:360 bitrate:300000} layers:{quality:HIGH width:1280 height:720 bitrate:1700000}", "request": "cid:\"1be255cb-f8f0-4349-8273-a27bb022c68b\" type:VIDEO width:1280 height:720 source:CAMERA layers:{width:320 height:180 bitrate:120000} layers:{quality:MEDIUM width:640 height:360 bitrate:300000} layers:{quality:HIGH width:1280 height:720 bitrate:1700000}"}
2023-02-20T13:14:56.712Z	INFO	livekit	rtc/participant.go:1445	pending track added	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "190", "pID": "PA_TPWTSimSwb5Z", "remote": false, "trackID": "TR_AMQ9Nmi4yxmLpB", "track": "sid:\"TR_AMQ9Nmi4yxmLpB\" source:MICROPHONE", "request": "cid:\"4c28a818-b295-4889-90b2-05de70e8221e\" source:MICROPHONE"}
2023-02-20T13:14:56.717Z	INFO	livekit	rtc/participant.go:1445	pending track added	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "190", "pID": "PA_TPWTSimSwb5Z", "remote": false, "trackID": "TR_VC8K7V9biHa9kv", "track": "sid:\"TR_VC8K7V9biHa9kv\" type:VIDEO width:1280 height:720 source:CAMERA layers:{width:320 height:180 bitrate:120000} layers:{quality:MEDIUM width:640 height:360 bitrate:300000} layers:{quality:HIGH width:1280 height:720 bitrate:1700000}", "request": "cid:\"ab42dd92-ae9e-4ec3-9cdd-db378f050657\" type:VIDEO width:1280 height:720 source:CAMERA layers:{width:320 height:180 bitrate:120000} layers:{quality:MEDIUM width:640 height:360 bitrate:300000} layers:{quality:HIGH width:1280 height:720 bitrate:1700000}"}
2023-02-20T13:14:57.299Z	INFO	livekit	rtc/participant.go:1107	mediaTrack published	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "190", "pID": "PA_TPWTSimSwb5Z", "remote": false, "kind": "audio", "trackID": "TR_AMQ9Nmi4yxmLpB", "rid": "", "SSRC": 1662664693, "mime": "audio/red"}
2023-02-20T13:14:57.476Z	INFO	livekit	rtc/participant.go:1107	mediaTrack published	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "192", "pID": "PA_4KFGJZixsmGw", "remote": false, "kind": "audio", "trackID": "TR_AMTtuyobuZe22u", "rid": "", "SSRC": 368627194, "mime": "audio/red"}
2023-02-20T13:14:57.532Z	INFO	livekit	rtc/participant.go:1107	mediaTrack published	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "192", "pID": "PA_4KFGJZixsmGw", "remote": false, "kind": "video", "trackID": "TR_VCJFKmx2GAM9bg", "rid": "q", "SSRC": 4107593707, "mime": "video/VP8"}
2023-02-20T13:14:57.560Z	INFO	livekit	rtc/participant.go:1107	mediaTrack published	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "190", "pID": "PA_TPWTSimSwb5Z", "remote": false, "kind": "video", "trackID": "TR_VC8K7V9biHa9kv", "rid": "q", "SSRC": 1278411893, "mime": "video/VP8"}
2023-02-20T13:14:57.603Z	INFO	livekit	rtc/participant.go:1107	mediaTrack published	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "192", "pID": "PA_4KFGJZixsmGw", "remote": false, "kind": "video", "trackID": "TR_VCJFKmx2GAM9bg", "rid": "h", "SSRC": 290170900, "mime": "video/VP8"}
2023-02-20T13:14:57.677Z	INFO	livekit	rtc/participant.go:1107	mediaTrack published	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "190", "pID": "PA_TPWTSimSwb5Z", "remote": false, "kind": "video", "trackID": "TR_VC8K7V9biHa9kv", "rid": "h", "SSRC": 336894810, "mime": "video/VP8"}
2023-02-20T13:14:58.210Z	INFO	livekit	rtc/participant.go:1107	mediaTrack published	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "190", "pID": "PA_TPWTSimSwb5Z", "remote": false, "kind": "video", "trackID": "TR_VC8K7V9biHa9kv", "rid": "f", "SSRC": 1215010090, "mime": "video/VP8"}
2023-02-20T13:14:58.328Z	INFO	livekit	rtc/participant.go:1107	mediaTrack published	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "192", "pID": "PA_4KFGJZixsmGw", "remote": false, "kind": "video", "trackID": "TR_VCJFKmx2GAM9bg", "rid": "f", "SSRC": 22206667, "mime": "video/VP8"}
2023-02-20T13:15:06.204Z	INFO	livekit	rtc/participant.go:1445	pending track added	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "190", "pID": "PA_TPWTSimSwb5Z", "remote": false, "trackID": "TR_VSTkU8ftzZZAop", "track": "sid:\"TR_VSTkU8ftzZZAop\" type:VIDEO width:1920 height:1080 source:SCREEN_SHARE layers:{width:960 height:540 bitrate:150000} layers:{quality:MEDIUM width:1920 height:1080 bitrate:300000}", "request": "cid:\"28e6cb38-cc3a-4988-a93c-24d6b9c6b3e1\" type:VIDEO width:1920 height:1080 source:SCREEN_SHARE layers:{width:960 height:540 bitrate:150000} layers:{quality:MEDIUM width:1920 height:1080 bitrate:300000}"}
2023-02-20T13:15:06.652Z	INFO	livekit	rtc/participant.go:1445	pending track added	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "190", "pID": "PA_TPWTSimSwb5Z", "remote": false, "trackID": "TR_AsAN6Ezj2HyrPz", "track": "sid:\"TR_AsAN6Ezj2HyrPz\" source:SCREEN_SHARE_AUDIO", "request": "cid:\"242f308a-3039-4651-b6a3-a2f50267f85a\" source:SCREEN_SHARE_AUDIO"}
2023-02-20T13:15:07.082Z	INFO	livekit	rtc/participant.go:1107	mediaTrack published	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "190", "pID": "PA_TPWTSimSwb5Z", "remote": false, "kind": "audio", "trackID": "TR_AsAN6Ezj2HyrPz", "rid": "", "SSRC": 712799976, "mime": "audio/red"}
2023-02-20T13:15:07.831Z	INFO	livekit	rtc/participant.go:1107	mediaTrack published	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "190", "pID": "PA_TPWTSimSwb5Z", "remote": false, "kind": "video", "trackID": "TR_VC8K7V9biHa9kv", "rid": "q", "SSRC": 4083911373, "mime": "video/VP8"}
2023-02-20T13:15:08.661Z	INFO	livekit	rtc/participant.go:1107	mediaTrack published	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "190", "pID": "PA_TPWTSimSwb5Z", "remote": false, "kind": "video", "trackID": "TR_VC8K7V9biHa9kv", "rid": "h", "SSRC": 3622693876, "mime": "video/VP8"}
2023-02-20T13:15:38.944Z	INFO	livekit	rtc/participant.go:1445	pending track added	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "192", "pID": "PA_4KFGJZixsmGw", "remote": false, "trackID": "TR_VS2ewvu6HuwqnF", "track": "sid:\"TR_VS2ewvu6HuwqnF\" type:VIDEO width:1920 height:1080 source:SCREEN_SHARE layers:{width:960 height:540 bitrate:150000} layers:{quality:MEDIUM width:1920 height:1080 bitrate:300000}", "request": "cid:\"1c2dc2d2-f480-49e3-a193-48cd90f3e122\" type:VIDEO width:1920 height:1080 source:SCREEN_SHARE layers:{width:960 height:540 bitrate:150000} layers:{quality:MEDIUM width:1920 height:1080 bitrate:300000}"}
2023-02-20T13:15:39.374Z	INFO	livekit	rtc/participant.go:1445	pending track added	{"room": "167049957", "roomID": "RM_nEPzGKTJgwkt", "participant": "192", "pID": "PA_4KFGJZixsmGw", "remote": false, "trackID": "TR_AshDnEgS26jTeb", "track": "sid:\"TR_AshDnEgS26jTeb\" source:SCREEN_SHARE_AUDIO", "request": "cid:\"3dba902d-5402-4450-9955-3f22c9b03315\" source:SCREEN_SHARE_AUDIO"}```

Unbenannt.ts

That's how we publish tracks. `msg.track` is of type `LocalTrack`  and `msg.options` is of type
```(property) options: {
    screenShareEncoding?: {
        maxBitrate: number;
        maxFramerate?: number | undefined;
    } | undefined;
} | null```
`canSimulcast()` will just return a boolean

room_458438232.log

here are the debug logs of the same situation

Why is having multiple layers surprising, <@U02CM28PB4Y>? I guess, I'm missing something here. But isn't that the purpose of simulcast to have multiple quality layers for the same camera/screen stream?

Having multiple layers is not surprising. I would expect 3 of those layers for camera and 2 for screenshare. But all 5 have camera track id in the logs. That is the unexplained/surprising part.

participant.png

This is participant "551" `room.localParticipant.videoTracks` showing he has 2 tracks published.

presenter.png

This is another participant ("presenter") seeing just one video track of participant "551" in `room.participants.get("...").videoTracks`

that is strange. on the other participant, it also shows three total tracks, but it's seeing an extra audio track instead of screenshare.

The screen share audio is the other audio track I believe.

yeah.. the strange thing is they are each seeing a different track..

Hi, are there any new ideas about this issue?

just a thought: did you explicitly set `localTrack.source` on the LocalTrack that you created?
adjacent: how are you creating the LocalTrack?

yes, we are setting track source explicitly. We will try to recreate this issue in livekit playground.

<@U02EU2ZPTQU> We wrap MediaStreamTracks with LocalTrack:

```
const camera = new LocalVideoTrack(cameraMediaStreamTrack)
camera.source = Track.Source.Camera

const microphone = new LocalAudioTrack(microphoneMediaStreamTrack)
microphone.source = Track.Source.Microphone

const tracks = await createLocalScreenTracks({ audio: true })
const screenShare_ = tracks.find(track =&gt; track.source === Track.Source.ScreenShare)
// providedByUser prevents video track from stopping when calling replaceTrack method
const providedByUser = true
const screenShare = new LocalVideoTrack(
    screenShare_.mediaStreamTrack,
    {},
    providedByUser,
)
screenShare.source = screenShare_.source

const screenShareAudio = tracks.find(
    track =&gt; track.source === Track.Source.ScreenShareAudio,
)

return { camera, microphone, screenShare, screenShareAudio }```

Also what I can see in the logs is `supervisor error on publication` for screen_share track. So the client receives info that publication was successful (local participant has screen share track in his tracks) but publication eventually fails on the server due to timeout and gets mixed with the camera track somehow.

<@U048B2EFGNN> which browser are you able to reproduce it with? (for the publisher)

<@U01KZ61GV5Z> Only in Chrome - it’s the same in  versions 109,110,111

<@U01KZ61GV5Z> <@U02CM28PB4Y> <@U02EU2ZPTQU> I see that there is similar `supervisor error on publication` output related to another issue. Shouldn’t the client js at least receive a failure instead of success when server fails to publish a track. Currently client gets info that everything wen’t fine despite the issue. <https://github.com/livekit/livekit/issues/1238#issuecomment-1483817069>

<@U048B2EFGNN> To answer to your GH comment, the change has not been approved in Pion and hence it it not deployed (I am talking about the Firefox issue when sharing screen followed by camera). It is not expected to work. Is it possible to for you to do the other way (publish camera followed by screen share).

I think the root of this message could be a different issue. Looks like it is as the issue at the root of this thread happens in Chrome, but the GH issue refers to Firefox.

For the Chrome issue, supervisor error happens 30 seconds after publish attempt. It is not part of the protocol. We have discussed it a little bit, but have not decided on any approach. One possibility is to do force a full reconnect from server side. Another option is like you mentioned, i. e. notify a client about a missing publish (it will come 30 or more seconds after publish attempt if the protocol is designed to tell clients about it).

Thanks <@U02CM28PB4Y>. It would be useful to be notified that publication failed. Right know to only way for a participant to know that his publication failed is to be told by other participants and that’s no optimal I think. But in case of the Firefox issue - it wouldn’t help, because stopping and starting camera again and again doesn’t work at all if screen share was published first.

<@U048B2EFGNN> our plan is to force a reconnect when this scenario happens.. clients should automatically republish all of their tracks upon reconnection.

<@U01KZ61GV5Z> Could you tell, when are you planning to implement and release it?

Hey it turns out this has been implemented in the code but just not documented

you'd have to set `rtc.reconnect_on_publication_error: true` to enable this behavior

Thanks <@U01KZ61GV5Z>, I’ll try this out.

<@U01KZ61GV5Z> So I tried it. The reconnection is enforced. But after reconnection tracks are published again in screen_share, camera order. This causes the same bug again. Actually both publications fail after reconnecting. Then the next reconnection is enforced and then only camera is published.

hmm could you file a bug for it? if publishing is consistently failing. we'd want to fix it.

if you could include any client code that could reproduce it, that'd be appreciate it!

<@U01KZ61GV5Z> It is reproducible with this issue: <https://github.com/livekit/livekit/issues/1238> You’d just have to set `rtc.reconnect_on_publication_error` on your side.

<@U01KZ61GV5Z> I have currently 3 issues:
• firefox - publishing camera after screen sharing doesn’t work
• publishing tracks while losing connection for a couple of seconds - client is notified that tracks were published but other participants do not see them
• in bad network conditions tracks are mixed - we publish camera and screen share - other participants subscribe to camera but they see screen share instead
All this issues are solved by hard reconnection (disconnect without stopping tracks, connect, publish tracks again in the correct order). For Firefox it is easy to detect when to perform reconnection. But I don’t know how to detect other issues. I see in the logs that peer reconnected but there is no event for that so that I could trigger full reconnection.
```[LiveKit] 14:11:32.649Z secondary PC state changed disconnected
[LiveKit] 14:11:33.297Z secondary PC state changed connected
[LiveKit] 14:11:33.728Z primary PC state changed disconnected
[LiveKit] 14:11:33.925Z primary PC state changed connected```
Also `connectionQualityChanged` is mostly good or excellent. I wouldn’t perform reconnection based on this event.

Could you share some suggestions? I’ve been working on it for 3 months already. I tried several server versions including self built from latest main branch of livekit-server. I keep client-js up-to-date but these issues still persist. I think hard reconnection could solve them but I have no idea how to wire it to the issues mentioned.

Got it, will check with the team on the above. Thank you for the detailed steps. We'll get this one resolved soon

<@U01KZ61GV5Z> One more clue that I have is that the issue with camera and screen tracks being mixed into one doesn’t occur when I disable simulcast.

Could you include it in the issue as well? Would be great to have a sequence of steps (ideally using <http://meet.livekit.io|meet.livekit.io>). if we can reproduce it, I'm certain we could fix it

I cannot reproduce it on <http://meet.livekit.io|meet.livekit.io>. Maybe I messed some configuration. What  versions of livekit-server and livekit-client are used in there?

<http://meet.livekit.io|meet.livekit.io> is running on LiveKit Cloud (which should match 1.4.0), and it's using client SDK 1.7.1

logs-with-simulcast.log

logs-no-simulcast.log

<@U01KZ61GV5Z> ~Could you also share livekit.yaml for livekit-server? Maybe I am missing something in there.~ 
Regarding the initial issue when a screen share track is mixed with a camera track. I connected the playground to my server, and despite trying to do the same in the playground I’m not able to reproduce it :confused:
I have no idea what could I do wrong on my client’s side. I can only see this error in the console: `Uncaught (in promise) DOMException: Failed to execute 'setRemoteDescription' on 'RTCPeerConnection': Failed to set remote answer sdp: The order of m-lines in answer doesn't match order in offer. Rejecting answer.` and then other participants see camera track which from time to time displays screen share content.

Here’s the playground. The issue isn’t reproducible here, but it generally does the same what my app does. Maybe you could suggest at least, where can I look for the cause of this issue on my side.
<https://livekit.io/playground#room=1&amp;code=1916736328b98cb76c205ae4>

Simulcast seems to cause `supervisor error on publication` in server-logs.
Disabling adaptiveStream = false and dynacast = false in room options, and simulcast = false in publication options seems to help, but we actually want these options to be enabled.  (server logs are different, see attachments)

I updated livekit-server to 1.4.0 and livekit-client to 1.7.1.

That's interesting to know.. could you include this information in the ticket? including the logs? If unable to reproduce with Meet/playground. Is there a way you can provide a minimally reproducible example (feel free to fork client-sdk-js's super simple sample app)

Without a way to repro it here, we can't tell what's really going on.

I’m still not able to reproduce it in any playground. But it seams `adaptiveStream: false` seems to fix it. I thought it is simulcast, but only disabling adaptiveStream options solves the issue in my app. I’ll continue to try to narrow down the problem.

what? hmm that makes no sense to me... are you able to reproduce with the <https://github.com/livekit/client-sdk-js#sdk-sample|JS sample app>?

<@U01KZ61GV5Z> I’m not able to reproduce it in the sample app. Only in my environment. New thing related to tracks being swapped that I discovered is that remote participant has only camera and screen share audio tracks in his tracks list but when I call `room.engine.primaryPC.getReceivers()` there are three items. One is screen share track, but it is marked as disabled and muted. When I attach camera track it switches between a picture of camera and screen share. Server logs still show publication timeout for screen share. I started to think that maybe there is some sort of a race condition between offers and answers that are being exchanged, or setting of publication layers.

Also when the issue occurs, when local participant unpublishes screen share, his camera is unsubscribed on the other participant’s side.

Things that seem to help:
• disabling simulcast,
• disabling adaptive stream,
• publishing screen share without screen share audio
Could you suggest what else could I look for so that I can find the issue’s root cause in my app?

hmm that is strange that you cannot repro with the sample apps. Can you think of any differences between what the sample app is doing and what your app is doing?

If you can possibly modify the sample app to perform a sequence that's similar to your own app, do you think it would repro then?

I'm also wondering.. when this issue is happening to a single subscriber.. does it also happen to all other subscribers in the room? (i.e. would all subscribers unsubscribe from the camera track when screenshare is unpublished?)

<@U01KZ61GV5Z> I found a way to reproduce it. What I found out is that the issue occurs when main JS thread is busy just after localParticipant.publishTrack is finished. I created a new issue on github <https://github.com/livekit/livekit/issues/1603>

oh wow.. thank you for the detailed steps. we'll take a look at this right away

<@U01KZ61GV5Z> <@U02GP9AENTH> Thanks for the fix. I tried the latest main branch of livekit-server and it looks like the issue is no more. I’m looking forward the next release.

```livekit-egress-1  | + export TMPDIR=/tmp/lkegress
livekit-egress-1  | + TMPDIR=/tmp/lkegress
livekit-egress-1  | + rm -rf /var/run/pulse /var/lib/pulse /root/.config/pulse
livekit-egress-1  | + pulseaudio -D --verbose --exit-idle-time=-1 --system --disallow-exit
livekit-egress-1  | W: [pulseaudio] main.c: Running in system mode, but --disallow-module-loading not set.
livekit-egress-1  | N: [pulseaudio] main.c: Running in system mode, forcibly disabling SHM mode.
livekit-egress-1  | I: [pulseaudio] main.c: Daemon startup successful.
livekit-egress-1  | + '[' -z /tmp/lkegress ']'
livekit-egress-1  | + mkdir -p /tmp/lkegress
livekit-egress-1  | + exec egress
livekit-egress-1  | 2023-02-20T15:26:44.959Z	INFO	egress	redis/redis.go:82	connecting to redis	{"nodeID": "NE_FakQwAYoBSub", "simple": true, "addr": "redis:6379"}
livekit-egress-1  | 2023-02-20T15:26:44.965Z	DEBUG	egress	service/http.go:24	debug handler disabled	{"nodeID": "NE_FakQwAYoBSub"}
livekit-egress-1  | 2023-02-20T15:26:44.965Z	DEBUG	egress	service/service.go:68	starting service	{"nodeID": "NE_FakQwAYoBSub", "version": "1.5.4"}
livekit-egress-1  | 2023-02-20T15:26:44.965Z	DEBUG	egress	service/http.go:33	starting debug handler on address :0	{"nodeID": "NE_FakQwAYoBSub"}
livekit-egress-1  | 2023-02-20T15:26:44.965Z	INFO	egress	stats/monitor.go:133	available CPU cores: 5 max cost: 4.500000	{"nodeID": "NE_FakQwAYoBSub"}
livekit-egress-1  | 2023-02-20T15:26:44.966Z	DEBUG	egress	service/service.go:93	service ready	{"nodeID": "NE_FakQwAYoBSub"}
livekit-egress-1  | 2023-02-20T15:27:30.101Z	INFO	egress	service/service.go:122	request received	{"nodeID": "NE_FakQwAYoBSub", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:30.143Z	DEBUG	egress	server/main.go:150	handler launched	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:30.144Z	INFO	egress	redis/redis.go:82	connecting to redis	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW", "simple": true, "addr": "redis:6379"}
livekit-egress-1  | 2023-02-20T15:27:30.150Z	DEBUG	egress	sdk/source.go:128	connecting to room	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:30.172Z	INFO	logger/logger.go:130	received offer for subscriber	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:30.179Z	INFO	logger/logger.go:130	ICE connected	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:30.335Z	INFO	logger/logger.go:130	received offer for subscriber	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:30.366Z	INFO	logger/logger.go:130	track subscribed	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW", "participant": "api-client", "track": {}, "kind": {}}
livekit-egress-1  | 2023-02-20T15:27:30.366Z	DEBUG	egress	sdk/source.go:35	track subscribed	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW", "trackID": "TR_AMRRoRMvF7nN2J", "mime": "audio/opus"}
livekit-egress-1  | 2023-02-20T15:27:31.040Z	DEBUG	egress	pipeline/pipeline.go:374	[src] STREAM-STATUS - Stream status from queue1: A new thread needs to be created	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:31.040Z	DEBUG	egress	pipeline/pipeline.go:374	[src] STREAM-STATUS - Stream status from audiomixer0: A new thread needs to be created	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:31.040Z	DEBUG	egress	pipeline/pipeline.go:374	[src] STREAM-STATUS - Stream status from audiotestsrc0: A new thread needs to be created	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:31.040Z	DEBUG	egress	pipeline/pipeline.go:374	[src] STREAM-STATUS - Stream status from queue1: A thread has entered its loop function	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:31.040Z	DEBUG	egress	pipeline/pipeline.go:374	[src] STREAM-STATUS - Stream status from queue0: A new thread needs to be created	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:31.041Z	DEBUG	egress	pipeline/pipeline.go:374	[opusdec0] LATENCY - Element's latency has changed	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:31.041Z	DEBUG	egress	pipeline/pipeline.go:374	[src] STREAM-STATUS - Stream status from audiomixer0: A thread has entered its loop function	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:31.041Z	DEBUG	egress	pipeline/pipeline.go:374	[src] STREAM-STATUS - Stream status from audioAppSrc: A new thread needs to be created	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:31.041Z	DEBUG	egress	pipeline/pipeline.go:374	[src] STREAM-STATUS - Stream status from audiotestsrc0: A thread has entered its loop function	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:31.041Z	DEBUG	egress	pipeline/pipeline.go:374	[src] STREAM-STATUS - Stream status from queue0: A thread has entered its loop function	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:31.041Z	DEBUG	egress	pipeline/pipeline.go:374	[pipeline] NEW-CLOCK - Got a new clock	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:31.041Z	DEBUG	egress	pipeline/pipeline.go:374	[src] STREAM-STATUS - Stream status from audioAppSrc: A thread has entered its loop function	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:31.072Z	DEBUG	egress	pipeline/pipeline.go:374	[pipeline] STREAM-START - Pipeline stream is starting	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:31.073Z	DEBUG	egress	pipeline/pipeline.go:374	[pipeline] ASYNC-DONE - Async task completed	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:31.073Z	DEBUG	egress	pipeline/pipeline.go:374	[appsink0] LATENCY - Element's latency has changed	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:27:31.073Z	INFO	egress	service/handler.go:224	egress updated	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW", "egressID": "EG_Vx2b5KiYsHoW", "request_type": "track", "output_type": "websocket", "status": "EGRESS_ACTIVE"}
livekit-egress-1  | 2023-02-20T15:28:21.048Z	ERROR	egress	output/websocket.go:55	cannot read AppSink samples	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW", "error": "write tcp 172.29.0.4:47706-&gt;192.168.65.2:8000: write: broken pipe"}
livekit-egress-1  | <http://github.com/livekit/egress/pkg/pipeline/output.buildWebsocketOutputBin.func2|github.com/livekit/egress/pkg/pipeline/output.buildWebsocketOutputBin.func2>
livekit-egress-1  | 	/workspace/pkg/pipeline/output/websocket.go:55
livekit-egress-1  | <http://github.com/tinyzimmer/go-gst/gst/app.goSinkNewSampleCb.func1|github.com/tinyzimmer/go-gst/gst/app.goSinkNewSampleCb.func1>
livekit-egress-1  | 	/root/go/pkg/mod/github.com/livekit/go-gst@v0.0.0-20220603230042-cef031256427/gst/app/cgo_exports.go:137
livekit-egress-1  | <http://github.com/tinyzimmer/go-glib/glib.(*Object).WithTransferOriginal|github.com/tinyzimmer/go-glib/glib.(*Object).WithTransferOriginal>
livekit-egress-1  | 	/root/go/pkg/mod/github.com/tinyzimmer/go-glib@v0.0.25/glib/gobject.go:343
livekit-egress-1  | <http://github.com/tinyzimmer/go-gst/gst/app.goSinkNewSampleCb|github.com/tinyzimmer/go-gst/gst/app.goSinkNewSampleCb>
livekit-egress-1  | 	/root/go/pkg/mod/github.com/livekit/go-gst@v0.0.0-20220603230042-cef031256427/gst/app/cgo_exports.go:137
livekit-egress-1  | _cgoexp_b784cf736173_goSinkNewSampleCb
livekit-egress-1  | 	_cgo_gotypes.go:1284
livekit-egress-1  | runtime.cgocallbackg1
livekit-egress-1  | 	/usr/lib/go-1.18/src/runtime/cgocall.go:314
livekit-egress-1  | runtime.cgocallbackg
livekit-egress-1  | 	/usr/lib/go-1.18/src/runtime/cgocall.go:233
livekit-egress-1  | runtime.cgocallback
livekit-egress-1  | 	/usr/lib/go-1.18/src/runtime/asm_arm64.s:1181
livekit-egress-1  | 2023-02-20T15:28:21.052Z	ERROR	egress	pipeline/pipeline.go:811	pipeline error	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW", "element": "GstQueue", "message": "streaming stopped, reason error (-5)", "error": "Internal data stream error."}
livekit-egress-1  | <http://github.com/livekit/egress/pkg/pipeline.(*Pipeline).handleError|github.com/livekit/egress/pkg/pipeline.(*Pipeline).handleError>
livekit-egress-1  | 	/workspace/pkg/pipeline/pipeline.go:811
livekit-egress-1  | <http://github.com/livekit/egress/pkg/pipeline.(*Pipeline).messageWatch|github.com/livekit/egress/pkg/pipeline.(*Pipeline).messageWatch>
livekit-egress-1  | 	/workspace/pkg/pipeline/pipeline.go:300
livekit-egress-1  | <http://github.com/tinyzimmer/go-gst/gst.goBusFunc|github.com/tinyzimmer/go-gst/gst.goBusFunc>
livekit-egress-1  | 	/root/go/pkg/mod/github.com/livekit/go-gst@v0.0.0-20220603230042-cef031256427/gst/cgo_exports.go:128
livekit-egress-1  | _cgoexp_752bbda710ba_goBusFunc
livekit-egress-1  | 	_cgo_gotypes.go:17363
livekit-egress-1  | runtime.cgocallbackg1
livekit-egress-1  | 	/usr/lib/go-1.18/src/runtime/cgocall.go:314
livekit-egress-1  | runtime.cgocallbackg
livekit-egress-1  | 	/usr/lib/go-1.18/src/runtime/cgocall.go:233
livekit-egress-1  | runtime.cgocallback
livekit-egress-1  | 	/usr/lib/go-1.18/src/runtime/asm_arm64.s:1181
livekit-egress-1  | runtime.systemstack_switch
livekit-egress-1  | 	/usr/lib/go-1.18/src/runtime/asm_arm64.s:194
livekit-egress-1  | runtime.cgocall
livekit-egress-1  | 	/usr/lib/go-1.18/src/runtime/cgocall.go:167
livekit-egress-1  | <http://github.com/tinyzimmer/go-glib/glib._Cfunc_g_main_loop_run|github.com/tinyzimmer/go-glib/glib._Cfunc_g_main_loop_run>
livekit-egress-1  | 	_cgo_gotypes.go:2822
livekit-egress-1  | <http://github.com/tinyzimmer/go-glib/glib.(*MainLoop).Run.func1|github.com/tinyzimmer/go-glib/glib.(*MainLoop).Run.func1>
livekit-egress-1  | 	/root/go/pkg/mod/github.com/tinyzimmer/go-glib@v0.0.25/glib/gmainloop.go:53
livekit-egress-1  | <http://github.com/tinyzimmer/go-glib/glib.(*MainLoop).Run|github.com/tinyzimmer/go-glib/glib.(*MainLoop).Run>
livekit-egress-1  | 	/root/go/pkg/mod/github.com/tinyzimmer/go-glib@v0.0.25/glib/gmainloop.go:53
livekit-egress-1  | <http://github.com/livekit/egress/pkg/pipeline.(*Pipeline).Run|github.com/livekit/egress/pkg/pipeline.(*Pipeline).Run>
livekit-egress-1  | 	/workspace/pkg/pipeline/pipeline.go:214
livekit-egress-1  | <http://github.com/livekit/egress/pkg/service.(*Handler).Run.func1|github.com/livekit/egress/pkg/service.(*Handler).Run.func1>
livekit-egress-1  | 	/workspace/pkg/service/handler.go:92
livekit-egress-1  | 2023-02-20T15:28:21.054Z	DEBUG	egress	pipeline/pipeline.go:643	pipeline stopped	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW"}
livekit-egress-1  | 2023-02-20T15:28:21.056Z	WARN	egress	service/handler.go:212	egress failed	{"nodeID": "NE_FakQwAYoBSub", "handlerID": "EGH_RxqrBKQEn7wt", "egressID": "EG_Vx2b5KiYsHoW", "egressID": "EG_Vx2b5KiYsHoW", "request_type": "track", "output_type": "websocket", "error": "Internal data stream error."}```

this seems related to <https://github.com/livekit/egress/issues/210>. I’ll be looking for a fix this week

Thanks. What’s strange to me that it always happens 30-40 seconds after I start the track egress. And it works without any issue for that 30-40 seconds.

For the record I tried it with 3 test websocket servers:
• Python with FastAPI
• Python with Websockets package
• Kotlin with Ktor Websocket
With the same results

hey <@U0411KFD5NX> :wave: livekit/components-js is built with and on top of client-sdk-js. It is not a replacement and the client-sdk-js will not go away.
livekit/components-js is an attempt to make building your apps easier by abstracting and handling most of the event and state logic related to LiveKit. It also comes with a set of React components that you can use and customize. We have some early docs on this here: <https://github.com/livekit/components-js/blob/main/docs/alpha-docs/README.md>
livekit/compoents-js is a replacement for our current <https://github.com/livekit/livekit-react|react package>. It is currently in dev preview and not officially released, but some people have already started using it because it is easier to use and makes customizing and styling your components easier. So if you don’t mind some sharp egdes and some API changes, feel free to use it. We also welcome any form of feedback and thoughts!

Why make publishonly deprecated <https://github.com/livekit/client-sdk-js/pull/310>

Is there any option to only subscribe all participant, do not publish?( do not let LocalParticipant join) OR, Display only the participant windows I specify

Hey <@U04BKGWEJ0J>, could you share some of your frontend code? This will make it easier for us to understand and eliminate potential frontend issues.

<@U04BKGWEJ0J> if you could also share your project id via DM, we can also make sure everything works on the backend.

<@U04BKGWEJ0J> in the Cloud Dashboard (under `Settings` &gt; `Project` &gt; `Connection limits` section, could you check if you have reached the limit for `concurrent ingress requests`?
If so, admins can request an increase of the limit.

We don’t have a specific Ionic SDK, but you ought to be able to use our JS SDK

I see there is React sdk and component js.  is there any difference?

Hello <@U04N2ULR17C>, As stated on the livekit-react package page on GitHub, the `livekit-react` is in a deprecated state currently in favor of using the `components-js` package. Which I would highly recommend using.

sorry my english is a bit bad.  I still don't understand your words.  so which package should i use for react and typescript

I would recommend to use: `components-js`

<https://github.com/livekit/components-js>

Correct, itll be using: <https://github.com/livekit/components-js/tree/main/packages/react>

There is 3 packages, under the `components-js`  which are: the core, the react &amp; if you want you can leverage the styles also.

I actually don’t remember how different Ionic is from vanilla React

so… give it a try and let us know if you run into any problems? :joy:

I think you'll be fine, it's rather a wrapper around react.

Flutter sdk and android ios both have voice input processing components.  I don't know if it's in Components-react?

<@U04N2ULR17C> perhaps you’re looking for something like this?
<https://github.com/livekit/components-js/blob/main/packages/react/src/components/participant/AudioVisualizer.tsx>

maybe not, actually. What kind of voice input processing are you looking to do?

The service URL is definitely wrong there. I'm not familiar with Unity but <@U022KB72NFJ> can comment on what we need for ServiceURL?

Hey,
1. All Unity version above 2021 should be fine
2. It seems like the JoinHandle.cs doesn’t have the Projection component properly setup inside the scene ( Can you try to add it inside the JoinScene ? )
3. The ServiceURL is used to generate join tokens. It is a simple Twirp API with a simple web server: <https://github.com/livekit/client-unity-demo/tree/main/demo-api|code>
    a. If you don’t want this service, you can try to generate the tokens from another server and edit the code a little bit. 
    b. This is where we request the tokens: <https://github.com/livekit/client-unity-demo/blob/552288270bcf8dfb5bd3298cb7809967a5e0d7f0/unity-project/Assets/Scripts/GameManager.cs#L72|line>  
EDIT: the errors related to the Unity packages ( ShaderGraph ) seems too be outdated inside the demo? Can you check if it is installed inside the package manager

<https://github.com/livekit/client-unity-demo/blob/main/unity-project/ProjectSettings/ProjectVersion.txt|Version used inside the project>

Great thanks!
1. the problem with the Projection component reference missing is that there's no GameObject with such component on the scene.. I can add some random object, and assign the Projection component so that I can fix the reference, but it'll probably break the logic..
2. Regarding Twirp, thanks for the explaination- I'll try to host the server on my own, and will report back in case of any issues.
3. I didn't see any place where I'd assign the access token from the LiveKit Cloud- is it not necessary in this case?

Hey, you should be able to find the Projection component somewhere in the scene. You can put your server tokens <https://github.com/livekit/client-unity-demo/blob/552288270bcf8dfb5bd3298cb7809967a5e0d7f0/demo-api/config-sample.yaml#L10|here>:

You can either put the config inside the env var `LK_UNITY_CONFIG` .
Or put the config path with --config when running the binary

&gt; Hey, you should be able to find the Projection component somewhere in the scene.
It's 100% not available in this version- I've found the Projection script in the project, and used "Search Reference in the Scene" option in Unity- and it showed nothing.

&gt; You can put your server tokens <https://github.com/livekit/client-unity-demo/blob/552288270bcf8dfb5bd3298cb7809967a5e0d7f0/demo-api/config-sample.yaml#L10|here>:
&gt; You can either put the config inside the env var `LK_UNITY_CONFIG` .
&gt; Or put the config path with --config when running the binary
But those are the server settings right?
I wanted to use LiveKit Cloud- and my understanding was that I don't need to host my own server for that?

Adam, you still need an api-key/secret pair to sign tokens with Cloud. You can get cloud tokens from project settings

you would still need a backend server to integrate with our token-signing SDK. (i.e. a server SDK). because you do not want to embed api key or secret on the front-end.

<@U0411KFD5NX> I’m not sure I fully understand what you are trying to accomplish here. Are you looking for the source maps to debug your application? Describing what you have done and what your goal is will make it much easier for us to help you.

If you are trying to add source maps to your build, you will need to enable this in your framework/bundler. Most of the time it is off by default for production builds and on for development builds. As an example, here is how to enable it for Next.js (<https://nextjs.org/docs/advanced-features/source-maps#configuration-flag>)

<@U03RM7G8Q6A> Thanks a lot. In livekit-react, if you click the console log, the networktrace will jump to the ts file. But components-js will jump to mjs file. I start the project with `yarn dev:next`

Thank you for the clarification. I’ll look into it and get back to you.

for example: `index.mjs?6f49:810 fetching token` changed to `tooken-hooks.ts:xxxx feting tokens`

I tried `  `productionBrowserSourceMaps: true,``    Does not work.

`DevTools failed to load source map: Could not load content for <chrome-extension://cofdbpoegempjloogbagkncekinflcnj/build/content.js.map>: System error: net::ERR_BLOCKED_BY_CLIENT` I found this error

Thanks for all the info! :pray: I will look into it, but it will take some time :)

<@U0411KFD5NX> We are investigating further, but I don’t have a quick fix for you. As a workaround, you could use the global search in our IDE to find the logged string, this way you can also find the correct source file. This is of course a bit cumbersome, but should work until we get to the root of the problem.

<@U03RM7G8Q6A> Thanks a a lot. I am using livekit-react for debugging the process. Waiting for your good news ( I have test whole day, but can not change mjs to ts :melting_face:)

<@U03RM7G8Q6A> Did you find a solution to this problem?

<@U0411KFD5NX>. Sorry, no solution yet. But we created a ticket for it. Just to let you know. It is not a high priority ticket because we are still in development preview and the problem does not prevent you from using the package. But we want to fix it before the public release. So thank you for bringing this to our attention. :)

here’s a fix for this: <https://github.com/livekit/components-js/pull/331>

apparently nextjs doesn’t load sourcemaps for ‘.mjs’ files by default

someone might've asked this before but do you know how to change the audio input/output for web?

You can grab a microphone MediaStreamTrack, transform it whatever you like, and publish a resulting MediaStreamTrack via
`room.localParticipant.publishTrack(yourTrack, {source: Source.Micrphone})`

Here is my devices list obtained from the sdk with the original sorting.
```audioinput, Microphone Array (Intel® Smart Sound Technology for Digital Microphones)
audioinput, Microphone (DroidCam Virtual Audio)
audioinput, Microphone (Audeze Mobius 2Ch)
audiooutput, Speakers (Realtek(R) Audio)
audiooutput, M28U (NVIDIA High Definition Audio)
audiooutput, Speakers (Audeze Mobius 2Ch)
audiooutput, M28U (NVIDIA High Definition Audio)
videoinput, Integrated Camera
videoinput, DroidCam Source 3```
SDK selects first device in group, but call goes via `Audeze Mobius 2Ch`