LiveKit Community

it’s this part
<https://github.com/livekit/client-example-swift/blob/main/Shared/LiveKitExample.swift#L35-L39>

clients do not have buffering when connection is lost. so in this case you will not have video for the 10 mins when publisher lost connection

does WHIP created for this kind of situation?

no, WHIP does not help here. WebRTC in general does not handle buffering to local storage in event of network failure

that's why youtube uses rtmp protocol, they want to save entire session

RTMP would not handle failures in network either.

all of these are streaming protocols, and none of them can handle network going down for 10 minutes

I'm assuming you are speaking of the JS SDK? If you've already acquired the track, then <https://docs.livekit.io/client-sdk-js/classes/LocalTrack.html#getDeviceId|LocalTrack.getDeviceId> would return the deviceId it's capturing from.

If you have not, then the device that is tagged as default by <https://docs.livekit.io/client-sdk-js/classes/Room.html#getLocalDevices|Room.getLocalDevices> will be acquired

CleanShot 2022-07-18 at 17.09.51@2x.png

`Room.getLocalDevices('audioinput')` returns this for me

is there some sort of `default: true` flag exposed to the sdk? or is this just internal?

I can get it after enabling the microphone for the first time, but I was hoping to show the default selection to the user before they connect (so they know what device they are connecting with).

hmm I guess not.. those are the devices returned by `getUserMedia`

ok, Is there any way to reproduce ? I will try to reproduce this issue on my android device.

```This problem occurred when disconnecting the wifi and connecting another wifi during video chatting. on Android phone (LG v50)```


docker run --rm \
    -p 7880:7880 \
    -p 7881:7881 \
    -p 7882:7882/udp \
    -v $PWD/livekit.yaml:/livekit.yaml \
    livekit/livekit-server \
    --config /livekit.yaml \
    --node-ip=127.0.0.1

Resolved guys, The issue was that i was in another directory . The correct directory had the livekit.yaml

I am getting the following error while connecting from TS/JS client SDK Error: could not establish signal connection

CleanShot 2022-07-18 at 17.16.07@2x.png

that's actually an issue with the docs. There's an option <https://docs.livekit.io/client-sdk-js/interfaces/TrackPublishDefaults.html#stopMicTrackOnMute|stopMicTrackOnMute> that could enable that behavior for microphone tracks.

however, it does come with tradeoffs when users are using BT headsets.

<@U01KZ61GV5Z> what tradeoffs do you mean? the notorious codec switching when the mic on BT headset is not used?

yeah, it's a mode switch for BT. pasting this from the linked docs
```when this is enabled, and BT devices are connected, they will transition between profiles (e.g. HFP to A2DP) and there will be an audible difference in playback.```

hmm, I haven't run into this before, is this reproable on the example itself?

It is reproducible on the example. Is their anyway to capture the logs from react-native/livekit-client side? Thanks.

that'd depend on your own dev environment. in vscode, the logs appear automatically. you can also try <https://stackoverflow.com/questions/30115372/how-to-do-logging-in-react-native>

what platform are you testing on?

I am testing on Qualcomm Snapdragon 778G. We will try to get some logs and analyze. Thanks.

Ah, if you're on Android, then it should show up in logcat. I'm unable to repro on my Android device, so logs would be helpful

Do you mean  that the logs in livekit-client could be output to logcat?

I have a capture, I can send it to you very soon

I also see some errors in camera handling in cpp code.

I think the direct call stack is above, there is a fatal exception generated, when trying to enable the media track, code found the media track is not there.

Do you have exact repro steps? please include whether you're enabling camera/microphone on each participant

It can be reproduced when two participants in a room, and both of them have audio and video enabled and opened, when one of the participant leaves, the one who has the android client exists. If the video is disabled, it can't be reproduced.

what's the platform of the other participant, or does it happen regardless of the other participant's platform? trying to figure out why it only repros on yours, so if you can write out every step you take, that'd be really helpful

It can be reproduce no matter what platform other participant is on, either web, or IOS. The steps: 1. Both of participants join in a room with video and audio disabled; 2. Both participants open the audio and video; 3. One of the participant clock the button leaving the room; 4. The android side will flash out.

1658319705378.png

I mean crash out.<https://files.slack.com/files-pri/T01KVSV78P7-F03QYDET3BJ/1658319705378.png>

and this is with no modifications to the livekit example?

It 100% reproducible in the livekit example when  Android uses livekt react-native example, and the other participant is from <http://example.livekit.io|example.livekit.io>. I think this is easiest way to reproduce.

what's the device and android version you're using?

It can be reproduced in every Android device I have. I have HONOR 50 Qualcomm, Android 12, Vivo X20A, Android 8.1.0 in hand. And some other devices

This issue is still there. What's the recommended way?

Looks it got fixed here？ <https://github.com/vector-im/element-android/pull/3587/commits/de64df3cdba8428290c226f2ec3a89b6102a91fc>

hmm, I'm still unable to repro this one, I can't identify what's causing this

The call stack is similar as the issue happens here

can you get a set of logs using the latest livekit-react-native example client? debug logging is turned on there

Log attached， please take a look, thanks.

<@U01NZKTNE1H>,can you help to check the log? Something happens every time here for video calls.

is this from the react-native example? seems like a bunch of logs that should be there are missing

20220919.log

are you touching track subscription permissions on either client at the moment?

I don't know how to change the track subscription permissions.

I uses <https://example.livekit.io/#/> one side, and example program at the other end, both of them are in the same room, then I exit room from <https://example.livekit.io/#/>,  the example program crashes.

can you replace the react-native-webrtc dependency in the example folder with this:

    `"react-native-webrtc": "<https://github.com/livekit/react-native-webrtc.git#dl/track_release_logging>"`

reupdate the dependencies with yarn and run the test again?

LiveKit does not provide application level functionality like that out-of-box.

You can build one on top of LiveKit. You can use data channels to accomplish that - <https://docs.livekit.io/guides/room/data/>

The Swift example has an implementation of chat

Hi <@U03EVR88RA5>
looks like `ObservableRoom` is missing some basic convenience properties / features. I will make improvements around SwiftUI :slightly_smiling_face:

for now you can extend `ObservableRoom` ,
override the delegates you would like to be notified with the code like
```DispatchQueue.main.async { self.objectWillChange.send() }```
I will let you know if there are updates :slightly_smiling_face:

Hi <@U03EVR88RA5>
I’ve made some changes in this commit
<https://github.com/livekit/client-sdk-swift/commit/0ea49268879f3c549d2e8be38fb4cefde77a3163>

You can use the `main` branch to try this out.

I was thinking more of actually making connectionState a @Published value within ObservableRoom. I'm currently doing it like this:
```// MARK: RoomDelegate
extension Call: RoomDelegate {
  /**
   Function that updates `Call`'s binding for `ConnectionState`. This must be done on the main thread
   as we are mutating a `Binding`.
    */
  fileprivate func updateConnectionStateBinding(connectionState: ConnectionState) {
    DispatchQueue.main.async {
      self.connectionState = connectionState
    }
  }
   
  /**
   `RoomDelegate` callback executed when the `ConnectionState` changes.
    */
  func room(_ room: Room, didUpdate connectionState: ConnectionState, oldValue: ConnectionState) {
    updateConnectionStateBinding(connectionState: connectionState)
  }
}```
then publishing it as...
```@Published var connectionState: ConnectionState = .disconnected()```
If there's a better way of doing this, I'd love to hear it :slightly_smiling_face:

Hi Dimitri, this is currently not possible. Are you having difficulty generating a token? Do you have a specific use case motivating being able to create a room without one?

no, I don't have trouble generating a token, but it adds complexity for my use case

I am developing an application to match 2 people randomly

It'd probably make the most sense for your business logic to generate a token for each user for one shared room

I succeeded, finally the token system is fantastic. Thank you very much for this wonderful work

excited to see what you’re building <@U03Q3LSH4LC>!

does the screen capture continue, or is it just the last frame showing?

cpu per track would also be a good measure, could I get more than 1 track per cpu in this scenario?

Hey Joao, we don't have similar load tests for egress server. But the good news is that it should scale well horizontally. There is capacity management functionality built into egress itself so when the server is heavily loaded, it will not take any additional requests.

Too bad, I was trying to get a rough cost estimate for my use case

Even without a proper benchmark do you have any value in mind as far as tracks per cpu with no composite involved? just so I can get a ballpark figure

Compositing requires transcoding, so if you are using a simple Track egress, it shouldn't be using a lot of resources. I don't have exact figures, but it wouldn't surprise me that it could do 50-100 concurrent per core

Thank you David, that is more that is good enough for some quick math :slightly_smiling_face:

image.png

Hi Rishi, just to confirm, are you using our React component to render the video?

not sure, let me double check with my team.

Hey Rishi, when you say `when you guys are capturing video` is this your app or a video output from Egress?

Hey <@U02KX7RH43Y>, you should explicitly disconnect from the room if possible. Our JS SDK automates it, but I don't believe we automatically handle it on mobile at the moment. It'd be a good enhancement though. I'll discuss with the team!

this is specifically for the case of the device powering off and not just the app being terminated by the system?

<@U01NZKTNE1H> exactly yes, although obviously if the case that the app is terminated could be handled too that would be great; I'm not sure if the OS allows that... Also, <@U02U3MSKYF7> with the current version of the JS SDK I'm not seeing that it handles browser tab close/refresh -- that participant "zombie" does stay hanging out on the call. For the time being we're manually leaving the call within the  "beforeunload" window event

<@U02KX7RH43Y> we do handle beforeunload <https://github.com/livekit/client-sdk-js/blob/main/src/room/Room.ts#L325|here>. curious why it's not working for you.

are you able to reproduce with the provided JS example app?

It’s odd — I do see that the event listener is being set, but when I enable the `beforeunload` breakpoint in Chrome dev tools, then close the tab, I break on the other three breakpoints, but not on the LiveKit one:

I think it’s iOS’s limitation, how about when you publish an audio track and go background ? (someone publishing at least 1 audio track in the room)

Are you testing with the latest Helm charts? can you share more about the load you are testing when performance issues started appearing?

Also, are you using recommended production configs with using a large port range? (i.e. rtc.udp_port_start/etc)

In Kube, how much resources are you giving it?

These variables would all impact performance in Kubernetes.

hey <@U01KZ61GV5Z>,

Yes, we are using latest Helm charts. We have all required ports open according to the docs.

We give 4 cores to Livekit and 3 cores to egress.

The experience is laggy with two persons connected in the room. It works fine for a few seconds but then there is delay with audio/video and packages lost, is almost impossible to have a conversation.
There is almost no cpu/ram usage so I guess it could be something with connectivity?

It’s really weird because using EC2 directly works really good, so it should be something with Kubernetes.

But the question is, what could be? :thinking_face:

Screen Shot 2022-07-29 at 9.44.20 AM.png

Can you capture a screenshot of all of the candidate table from <chrome://webrtc-internals>? it looks like this.

can you elaborate which platform? and by speaker selection, do you mean audio output selection?

I mean the react-native SDK, where it is used in both Android and IOS.

You are right, I mean the audio output selection, user can select where uses the speaker or earplug as the audio output. Also could be some advanced feature like when the earplug is plugged in, the audio is output to earplug automatically.

I see, React native SDK is currently in beta, and we do plan to improve the audio handling here. CC: <@U01NZKTNE1H>

hey everybody, sorry for reposting, but my question still remains: <https://livekit-users.slack.com/archives/C01KVTJH6BX/p1657868173200379>
it's about "could not handle new participant" errors in the log - should we be worried about them, should we ignore them, report them?

LocalTrack/localMediaStream cannot be played on AudioElement on flutter native, because there is no html5-like audio element tag on libwebrtc native (in fact, localtrack on native has only an empty AudioTrack interface, which does not contain AudioSink AudioSource). But there is another way , <https://github.com/flutter-webrtc/flutter-webrtc/blob/main/example/lib/src/loopback_sample_unified_tracks.dart>, in this code a local loop call is created using two pcs
audio track-&gt;pc1 --&gt;pc2--&gt;playout
video track-&gt;pc1 --&gt;pc2-&gt;video renderer

Which protocol is your ip camera using? Maybe Livekit will support access to these devices in the future.

you can do this with the CLI + FFMPEG today. take a look at <https://github.com/livekit/livekit-cli#publish-from-ffmpeg|this>.

Exemple of the logs:
```2022-07-25T08:56:15.062051612Z 2022-07-25T08:56:15.061Z	INFO	livekit	rtc/participant.go:873	connection quality not optimal	{"room": "nts-0-1", "roomID": "RM_zqh2j97Sgof2", "participant": "nts-0-1", "pID": "PA_Pzzjb9wb8tbU", "remote": false, "totalScore": 7.87, "numTracks": 2, "rating": "GOOD", "publisherScores": {"TR_VCeLkvz2E9rU2P":3.95,"TR_VCefAktSuEJJ4V":3.92}, "subscriberScores": {}}
2022-07-25T08:56:15.062081052Z 2022-07-25T08:56:15.061Z	INFO	livekit	rtc/participant.go:873	connection quality not optimal	{"room": "nts-0-1", "roomID": "RM_zqh2j97Sgof2", "participant": "690154072556175374", "pID": "PA_yVBCHjgdkUpX", "remote": false, "totalScore": 7.87, "numTracks": 2, "rating": "GOOD", "publisherScores": {}, "subscriberScores": {"TR_VCeLkvz2E9rU2P":3.99,"TR_VCefAktSuEJJ4V":3.88}}```

Video is on for all 12 clients?
Check for client machine's CPU usage.

Client CPU usage is pretty low. Roughly ~30% on 4 core for the client that stream 2 tracks and less than that for client who just consume. Server is a less than 1% CPU per core (so overall somewhere around 12% for the whole CPU)

this is normal, sometimes these logs are overly aggressive and inaccurate. this will be removed in the next version

Ok, thanks for the reply David :slightly_smiling_face:

```{"level":"error","ts":1658503379.7036145,"logger":"egress","caller":"pipeline/pipeline.go:309","msg":"could not upload file","nodeID":"NE_ttRbDhry3gwS","egressID":"EG_UUDbrjR2voz8","location":"S3","error":"RequestError: send request failed\ncaused by: Put \"<https://s3.eu-west-3.amazonaws.com/WIP/KAu7IlJMiSfs1vvnAQUAY/0K5BDH_ZbAmYz1KUk2MaQ/archive-2022-07-22-15h19m45s-UTC.raw.3.ePMwcArsI1QMvNLjTDco7.mp4>\": dial tcp: lookup <http://s3.eu-west-3.amazonaws.com|s3.eu-west-3.amazonaws.com> on 127.0.0.11:53: read udp 127.0.0.1:46505-&gt;127.0.0.11:53: i/o timeout","stacktrace":"<http://github.com/livekit/egress/pkg/pipeline.(*Pipeline).storeFile\n\t/workspace/pkg/pipeline/pipeline.go:309\ngithub.com/livekit/egress/pkg/pipeline.(*Pipeline).Run\n\t/workspace/pkg/pipeline/pipeline.go:230\ngithub.com/livekit/egress/pkg/service.(*Handler).HandleRequest.func2\n\t/workspace/pkg/service/handler.go:59|github.com/livekit/egress/pkg/pipeline.(*Pipeline).storeFile\n\t/workspace/pkg/pipeline/pipeline.go:309\ngithub.com/livekit/egress/pkg/pipeline.(*Pipeline).Run\n\t/workspace/pkg/pipeline/pipeline.go:230\ngithub.com/livekit/egress/pkg/service.(*Handler).HandleRequest.func2\n\t/workspace/pkg/service/handler.go:59>"}```

Nevermind, the issue is not related to Livekit or Egress at all

Hello <@U03R8BMAXC1>,

Maybe this would be helpful for you: <https://livekit-users.slack.com/archives/C01KVTJH6BX/p1655767538444439?thread_ts=1655766442.726039&amp;cid=C01KVTJH6BX>

Hi <@U032TUAL4KT> ! Thanks for the response -- saw those links/ documentation and was wondering specifically from the client side if it was possible

Hey <@U03R8BMAXC1>, I don't think there is currently an "easy way" to make work this out currently through the client SDK although I think the team has this in their backlog of feature &amp; improvements coming along.

This said, I'm not sure what you are after nor the goal, but if this is only for 1 user.. a virtual output card could always be used as a mic input. :man-shrugging:

The Unity SDK uses the Unity coroutines to handle tasks. The main reason for that is we don't use threading on WebGL. If you want to use async await, you can write a wrapper around IEnumerator 

You can use <https://github.com/zsaladin/Asyncoroutine|https://github.com/zsaladin/Asyncoroutine> for example 

How much resources have you given the egress instance? It sounds like it might be lagging behind due to insufficient CPU. CC <@U01UB0LHTTQ>

it’s only happening the first 4-5 seconds, maybe chrome puppeteer takes a while to load everything and that causes this?

that's my guess.. are you using our default template? or a custom template?

one way to solve it could be to start the real recording after puppeteer is stabilized?

we haven't been able to reproduce this, so it's just a guess. could you file a GH issue with detailed repro steps?

we'll get it over to the team to take a look

Hello <@U03PG3DG7QB>, have you had a chance to look at the LiveKit Doc? A whole lot of great information is provided through it with all the details around Server and Client SDKs. <https://docs.livekit.io/guides/getting-started/>

Also, if you are trying to spin up livekit, maybe the post from <@U01KZ61GV5Z> could be useful, which can be found here: <https://livekit-users.slack.com/archives/C01KZ61HZRR/p1658513005701529>

thank you for your reply , i check the doc but I don't know how I figure out the server and get some problems so I need tutorial or example

By figure out the server... can you explain which problems you are having specifically?

when I hit endpoint to create a room, room data is not stored. this the code
```import { AccessToken } from 'livekit-server-sdk';
import { RoomServiceClient, Room } from 'livekit-server-sdk';
import { WebhookReceiver } from 'livekit-server-sdk';
import express from 'express';

const app=express();
// if this room doesn't exist, it'll be automatically created when the first
// client joins
const roomName = 'name-of-room';
// identifier to be used for participant.
// it's available as LocalParticipant.identity with livekit-client SDK
const participantName = 'user-name';

const at = new AccessToken('api-key', 'secret-key', {
  identity: participantName,
});
at.addGrant({ roomJoin: true,
    room: roomName,
    canPublish: false,
    canSubscribe: true, });

const token = at.toJwt();
console.log('access token', token);


const livekitHost = '<https://my.livekit.host>';
const svc = new RoomServiceClient(livekitHost, 'api-key', 'secret-key');

// list rooms
svc.listRooms().then((rooms: Room[]) =&gt; {
  console.log('existing rooms', rooms);
});

// create a new room
const opts = {
  name: 'myroom11',
  // timeout in seconds
  emptyTimeout: 10 * 60,
  maxParticipants: 20,
};
// svc.createRoom(opts).then((room: Room) =&gt; {
//   console.log('room created', room);
// });

// delete a room
// svc.deleteRoom('myroom11').then(() =&gt; {
//   console.log('room deleted');
// });


const receiver = new WebhookReceiver('apikey', 'apisecret');

// In order to use the validator, WebhookReceiver must have access to the raw POSTed string (instead of a parsed JSON object)
// if you are using express middleware, ensure that express.raw is used for the webhook endpoint
// router.use('/webhook/path', express.raw());

<http://app.post|app.post>('/webhook-endpoint', (req, res) =&gt; {
  // event is a WebhookEvent object
  const event = receiver.receive(req.body, req.get('Authorization'));
});
app.get("/",(req,res)=&gt;{
    res.send("hello")
    console.log(opts);

    svc.createRoom(opts).then((room) =&gt; {
        console.log('room created', room);



        svc.listRooms().then((rooms: Room[]) =&gt; {
            console.log('existing rooms', rooms);
          });
      });


})
app.listen(5000,()=&gt;{
  console.log("server running");

});```

please if you have a example share it please.

Could you validate if you use the example page, <https://example.livekit.io/#/> .. and connect through it.. when doing a `listRooms` .. can you now view the room? If I recall correctly, sometimes rooms do not get created until a user joins it. I could be wrong but I would do the test.

Generate a token, use your host &amp; generated token on the example site, then see if you get the list of rooms.

Also, if you are looking for examples of how to connect and create an app, the example script is open source and can be found here:

React:  <https://github.com/livekit/livekit-react/tree/master/example/src>
JS: <https://github.com/livekit/client-sdk-js/tree/main/example>

I hope this helps.

thank you for your reply but all these is client I need js server example

Correct those examples are for the client.. it was simply as help. But have you tried my initial comment about what happens if you have a user session open if your server is returning any value from `listRooms`?