LiveKit Community

if you could share your use case, that'd help.

PCM would be awesome.    I have an ML model that runs on-device that measures features of voice such as personality and mood in real time.   Want to display it to the user

I see, I think connecting to the room with the client would be the most straight forward.. though there isn't a simple way to get the decoded audio data back today...

Egress can export PCM for each track, but it works by sending binary frames to websocket server.

Hmm.   Does it help if I only need the audio from the ~speaker~ person talking on the device?  Don't need it from anyone else in the room

Doesn't  have to go through the livekit code at all.  Audio could (in theory) be analyzed before it hits your stuff

I see, with the current APIs, I'm not sure if you are able to get the raw audio bytes directly before publishing to livekit.

we'd like to enable this though.. I'm just not sure if it's possible today.

hmm.   Ok so at least it's possible...  Would it be feasible for me to hire someone to do this?

got it. exporting raw audio bytes is something that's on our roadmap for flutter. CCing <@U02GAC58A79> and <@U01NZKTNE1H> to advise.

Sounds like insertable streams, I happened to be doing E2E Encryption for webrtc-sdk/flutter-webrtc this week, maybe I can see how to export/insert audio raw data.

That would be magical.   My startup has been stalled on this issue for months

If you have a donation endpoint, I'll give what I can

when you mean "get the audio from the speaker", do you mean get the audio bytes from the local device's microphone, or all the audio that's output to the device's speakerphone?

I need only the audio bytes from the local device's microphone (or headphone mic).

got it, the flutter-webrtc library does have access to that at the native level, but it isn't exposed at the moment to the flutter side. (for reference <@U02GAC58A79>, <https://github.com/flutter-webrtc/flutter-webrtc/blob/main/android/src/main/java/com/cloudwebrtc/webrtc/record/AudioSamplesInterceptor.java>, iOS should have a similar hook for the audio module to get the recorded audio bytes)

iOS raw data needs to be exposed from here.
<https://github.com/webrtc-sdk/webrtc/blob/a828846eb98e404bf3b00c1abf3787f93171a39c/sdk/objc/native/src/audio/audio_device_ios.mm#L426>

<@U01NZKTNE1H> is there any way to help make this happen?

issue filed: <https://github.com/livekit/client-sdk-flutter/issues/201>. <@U03TX02DEJD> you could follow the issue for updates.

Great!  Thanks!   I was just looking to hire someone from upwork to do this...    If I were to hire someone, do you have an estimate for how long it would take?   Do you have someone that you would recommend?

It'd be hard to give an estimate on it. It really depends on the expertise of the person working on it.

Screenshot 2022-11-02 at 9.49.11 AM.png

thanks for the feedback!

For free quota, are you referring to the information on the pricing page?

CC: <@U01KZ0SCDKL> <@U0469AQ3RKM>

<@U0492FYHJBV> good to see you here! you never told me what your project actually is but I’m sure it will be cool :slightly_smiling_face:

thanks for the feedback, those are good points

hey Julius! Good to see you here.

have you tried setting `--node-ip` when starting the service?

Thank you <@U01KZ61GV5Z>
I'll give it a try :eyes:

Hey <@U01KZ61GV5Z> how can I set external IP in configuration file? Smth like that?
```rtc:
  node_ip: <http://xxx.xxx.xxx.xxx|xxx.xxx.xxx.xxx>```

Can’t find it in sample configuration file <https://github.com/livekit/livekit/blob/master/config-sample.yaml>

yup, that's correct, we need to update the config example to include this field.

bandwidth consumption depends on the specific resolution &amp; bitrates that you are publishing. load tester uses a specific bitrate (it's actually a pre-recorded video played on loop).

also, screen share uses dynamic bitrate, and typically it's not high due to the nature of the screen being mostly static

a participant sends the video &amp; audio data to the livekit-server and livekit-server sends other participants video &amp; audio data to the participant. So if the total receive bandwidth is higher on the server side, shouldn’t be the transmit bandwidth also higher?

what parameters did you use with the load tester?

```livekit-cli join-room --url wss://${url} --api-key ${key} --api-secret ${secret} --room room:123 --identity 1 --publish ~/file1.ivf --publish ~/file1.ogg --fps 23.98```
```livekit-cli join-room --url wss://${url} --api-key ${key} --api-secret ${secret} --room room:123 --identity 2 --publish ~/file2.ivf --publish ~/file2.ogg --fps 23.98```
```livekit-cli join-room --url wss://${url} --api-key ${key} --api-secret ${secret} --room room:123 --identity 3 --publish ~/file3.ivf --publish ~/file3.ogg --fps 23.98```
```livekit-cli load-test --url https://${url} --api-key  ${key} --api-secret ${secret} --room room:123 --publishers 13```

what are you trying to do? the commands you've ran simulated 16 publishers and 3 subscribers.

to simulate 13 participants that are all subscribing, you would want to do:
```livekit-cli load-test --url https://${url} --api-key  ${key} --api-secret ${secret} --room room:123 --video-publishers 13 --subscribers 13```

Thanks for the feedback david, I missed subscriber part. I’ll try with this command

Hi, I would ask... (I have tested all before manually.. server side, client side ... Sharing screen ..  even wss, and all it's ok) but I would ask,  Trying this part <https://github.com/livekit/livekit-cli#publish-media-files|https://github.com/livekit/livekit-cli#publish-media-files> in my side as a real user I only see a gray screen, not the video that I'm publishing as file... What I'm missing? Thank a lot

your media files are probably not encoded correctly. did you re-encode them into the correct formats?

Umm in theory it's ok... I can see it with VLC... Bu I'll try later with a few other example... I'll comment later, thx

VLC plays just about anything.. WebRTC is very particular about supported formats

Hi, I codec as IVF (vp8) and no luck... The command livekit-cli join-room --room yourroom --identity publisher  --publish-demo should show a sample video too? The gray screen in client it's because no video html tag... Maybe I miss something in the default react client?

yeah, --publish-demo would send a valid sample video to the room. It sounds like your client isn't rendering it correctly? can you try <https://example.livekit.io>

Screenshot_20221111-131632.png

I think this url is the react client (the same/ersion that mine)

Using my server... That with my front the results are the same (using --publish-demo attribute

If works for you... Maybe something wrong in my server side? Some port or something else?

hmm that seems odd. does it behave this way on a desktop browser too?

I just tested it with the latest server, and it's working well here.

Well ... i did some progress, upgrade to 1.3.1 from 1.2.3 and with a few research more... i think my big problem with my configuration was *rtc.use_external_ip: true* for some reason if change to *false* it works (to see configuration check this <https://equilibristofgo.github.io/blog/posts/livekit_telegram/>) and... if use standard if use *--publish-demo* with *firefox 102.4.0esr* in debian ... "errorVerbose": "unable to start track, *codec is not supported by remote*\nsetting remote description failed) but now with my ivf video ... it's work... i'll keep diggin in it :sweat_smile: thanks for now

that's not the meaning of emptyTimeout. see <https://docs.livekit.io/server/room-management/#createroom|docs>.

however, it shouldn't be delayed that long.. can you add that note to <https://github.com/livekit/livekit/issues/241|this issue>?

when did framerate go down? while it's backgrounded?

I don't know if we have any controls over that.

It is only on local pc. Other participans receive normal framerate. I fond it when try to record)))

local video rendering is not controlled by adaptive stream, you are seeing a browser limitation due to background tabs

what is the issue? could you share it here?

<https://github.com/livekit/egress/issues/125>

<@U01KZ61GV5Z> Hi, are you missing some piece of information ? Can I help for something to go forward with this issue ?

Do you generate a new token on every request somehow? What is your setup?

This is my script for creating users for a room `$1`
is the username
```livekit-cli create-token \
    --api-key server-api-key --api-secret server-api-key-secret \
    --join --room my-first-room --identity $1 \
    --valid-for 24h```

I can’t find it out right now <@U049DUFBE0Y> But I discovered that the participant is connected to the room but he get PeerConnection timeout. So it looks like he cannot estabilish WebRTC connection.

<@U048B2EFGNN> Yes are you using it locally or is it deployed? Have you tried using the React  Javascript client to connect?

<@U049DUFBE0Y> It’s deployed. I only use js sdk.

You probably cannot use livekit over http on public IP
<https://stackoverflow.com/questions/41922515/webrtc-video-streaming-doesnt-work-through-http>

``` Er Error
Please use a keyframe frequency of four seconds or less. Currently, keyframes are not being sent often enough, which can cause buffering. The current keyframe frequency is 9.2 seconds. Note that ingestion errors can cause incorrect GOP (group of pictures) sizes.
8:19 PM
Warning
The stream's current bitrate (45.14 Kbps) is lower than the recommended bitrate. We recommend that you use a stream bitrate of 1000 Kbpsor
Please use a keyframe frequency of four seconds or less. Currently, keyframes are not being sent often enough, which can cause buffering. The current keyframe frequency is 9.2 seconds. Note that ingestion errors can cause incorrect GOP (group of pictures) sizes.
8:19 PM
Warning
The stream's current bitrate (45.14 Kbps) is lower than the recommended bitrate. We recommend that you use a stream bitrate of 1000 Kbps```


```const encodingOptions = {
  width: 1920,
  height: 1080,
  depth: 24,
  framerate: 30,
  audioCodec: 2,
  audioBitrate: 128,
  audioFrequency: 44100,
  videoCodec: 2,
  videoBitrate: 4500

}```

I've perhaps gotten closer to the issue. It seems the background color of  `grid` and `grid-light`  are showing but not any of the video or audio

solved... please make sure your egress and server api keys are the same

I have domain , ssl and server use window server but i can not config it. Can I have an example config on window server and ports?

Hi there!
Is there any particular reason why executing room create API call does not trigger room started event?

Hello guys, does anyone know how to check the permission for share share … ??

<@U01KZ61GV5Z> Maybe you have an idea on how can I do this or the way of doing this is actually rendering each VideoTrack and manually removing for each Participant the camera … if possible of course :sweat_smile:

you should be using `localParticipant.setCameraEnabled(false)`

will check and let you know :slightly_smiling_face:

<@U01KZ61GV5Z> my case is for turning off a `RemoteTrackPublication` camera off

For the video streaming solution we are implementing we wanna let the participants choose if they wanna receive the video stream from others

and for this I used on the RemoteTrackPublication the function called `setEnabled` to stop receiving the video stream from other participants but the thing is it’s freezes there camera on the last frame received witch makes sense but also would make sense removing it since is not receiving any data so why would it be present, right?
PS: With removing I mean just not showing the last frame and just put a placeholder like the grey we have when camera is off

For the implementation I’ve created seems like is not possible.

Google Meet for example does this when your choosing to not receive video stream from others it’s also turning other participants camera off.

<https://docs.livekit.io/client-sdk-js/classes/RemoteTrackPublication.html#setEnabled>

what I need is also to remote the video from appearing

we don't control "showing grey". you should monitor the track's status yourself and render an alternative image when disabled.

Ok, I see but this would mean i have to render myself the participants and not relay on the build in solution

since I cannot impact this behavior w/o doing that so

yes, you'd want to add some handling here.. or feel free to submit a PR to the components framework to handle it

I’ve asked since I thought would be a nice “feature” having the component do this since stopping the server from sending data and having as a consequence the camera turned off a good/wanted behavior :)

Thank you <@U01KZ61GV5Z> for your help and response :slightly_smiling_face:

Android should be doing simulcast as well. Can you share the relevant code? CC: <@U01NZKTNE1H>

there's a PR out addressing this issue, but I haven't had a chance to verify it yet:

<https://github.com/webrtc-sdk/webrtc/pull/48>

`When I reload the Chrome tab, participant gains a new identity`

identity is a string given by you. that should not change between reloads

you are likely using the same identity between diff users

Hmm, could that lead to this error <@U01KZ61GV5Z>? I updated livekit-server to 1.2.5 so I get more logs now.
```2022-11-04T13:41:48.722Z	INFO	livekit	rtc/participant_signal.go:180	could not send message to participant	{"room": "248879965", "roomID": "RM_QvCjEFWW3iCN", "participant": "513", "pID": "PA_n7ozAjE5PJHD", "remote": false, "message": "*livekit.SignalResponse_Update", "error": "no response sink"}
2022-11-04T13:41:48.722Z	ERROR	livekit	rtc/room.go:865	could not send update to participant	{"room": "248879965", "roomID": "RM_QvCjEFWW3iCN", "participant": "513", "pID": "PA_n7ozAjE5PJHD", "error": "no response sink"}
<http://github.com/livekit/livekit-server/pkg/rtc.(*Room).sendParticipantUpdates|github.com/livekit/livekit-server/pkg/rtc.(*Room).sendParticipantUpdates>
	/workspace/pkg/rtc/room.go:865
<http://github.com/livekit/livekit-server/pkg/rtc.(*Room).broadcastParticipantState|github.com/livekit/livekit-server/pkg/rtc.(*Room).broadcastParticipantState>
	/workspace/pkg/rtc/room.go:853
<http://github.com/livekit/livekit-server/pkg/rtc.(*Room).onTrackUpdated|github.com/livekit/livekit-server/pkg/rtc.(*Room).onTrackUpdated>
	/workspace/pkg/rtc/room.go:759
<http://github.com/livekit/livekit-server/pkg/rtc.(*ParticipantImpl).onMediaTrack|github.com/livekit/livekit-server/pkg/rtc.(*ParticipantImpl).onMediaTrack>
	/workspace/pkg/rtc/participant.go:1152```

#Android #LiveKit #version_1.1.4
Hi there, please take a look here
<https://github.com/livekit/client-sdk-android/issues/159>

sure, you can use any framework you want but you have to build it from scratch using the raw JS sdk

<https://github.com/livekit/client-sdk-js>

Build at: 2022-11-04T12:43:10.255Z - Hash: 0f8747c8ee0b6389 - Time: 339ms

Error: node_modules/livekit-client/dist/src/proto/livekit_models.d.ts:1:8 - error TS1192: Module '"/Users/nuzzgunners/Works/Angular/livekit_app_01/node_modules/protobufjs/minimal"' has no default export.

1 import _m0 from "protobufjs/minimal";
         ~~~


Error: node_modules/livekit-client/dist/src/proto/livekit_rtc.d.ts:1:8 - error TS1192: Module '"/Users/nuzzgunners/Works/Angular/livekit_app_01/node_modules/protobufjs/minimal"' has no default export.

1 import _m0 from "protobufjs/minimal";
         ~~~




:heavy_multiplication_x: Failed to compile.

I'm using Angular 14. Import by typescript with error.

hey! you can try with this setting in your tsconfig <https://www.typescriptlang.org/tsconfig/allowSyntheticDefaultImports.html|https://www.typescriptlang.org/tsconfig/allowSyntheticDefaultImports.html>

```Thank you. 
Add "allowSyntheticDefaultImports": true in tsconfig.```


Error: src/app/app.component.ts:83:13 - error TS2339: Property 'detach' does not exist on type 'LocalTrackPublication'.

83       track.detach();

What do you mean by client process? The client applications talking to eachother (WEBRTC) or the server crashing? I don't see and crash logs on your output.

Are you testing locally? What is your testing setup?

In theory it should be possible. give it a try and let us know what you find there.

ok will do, I cant test at the moment as when I try to install the Livekit i get an error, the same error reported by someone else <https://github.com/livekit/client-sdk-react-native/issues/29>

update: same issue affects mp4 recording.

Hey Tyler, what streams are you feeding it for testing? This sounds related to how we are synchronizing the tracks together.

If you can create an issue in livekit/egress repo, we can move the discussion there. :pray:

can you be specific here? which client SDK?

how would you reproduce `screenshare stream disconnect`?

I am using the react-native sdk. When I put the network of myphone in fight mode about 1-2 seconds,  the screenshare will become black, I consider it as screenshare stream disconnected.

Can you file an issue in livekit/client-sdk-react-native? Please include details such as the above. Also, which app are you in when airplane mode is activated? (your own app, or another app that you are trying to screen share?)

I think the issue is related to broadcast extension and IPC.

We had some issues with 1.5.0 on older linux kernels so we removed the 1.5.0 release. We'll release a fixed version shortly.

What do you mean by `I'm getting this error very often when refreshing the app` ?

It works the first time but errors out after a refresh?

How are you running your server? and which version

<@U01KZ61GV5Z> It works the first time but errors out after a refresh - this is the most common scenario. But then after some time it ‘resets’. It works once and after refresh stops working again.

 The server is spun up with docker compose. The image in use is livekit/livekit-server:v1.2.5

did you use our generator script to create the install? or did you make it custom?

<@U01KZ61GV5Z> I am not aware of any generator script. I think it is custom.
There is also similar issue reported on github <https://github.com/livekit/livekit/issues/1060>

config generator is here: <https://docs.livekit.io/oss/deployment/vm/>

I would give that a try first and ensure the ports/etc are mapped correctly.

Well, after a week of investigation we restarted my colleague’s router and it started to work fine :exploding_head:
Thank you <@U01KZ61GV5Z> and <@U049DUFBE0Y> for helping me with this one :pray:

glad to hear it! I wonder if the router is blocking UDP packets? it's unclear why connection test would work but not example.

<@U01KZ61GV5Z> There was no blocking on the router. But it could cache UDP packets.

it definitely is. you can pass in your own base url

here are docs on developing your own template: <https://docs.livekit.io/server/egress/custom-template/|https://docs.livekit.io/server/egress/custom-template/>

when starting room composite, it'll be one of the options to pass in

livekit-server exposes a websocket endpoint. This is what you pass to LK client SDKs, which establish a signaling connection to your server. You can interact with the server in some fashion from three different places: 1) a client SDK 2) cli 3) server SDK. Each allows for a different set of capabilities.

Tokens are not generated by calling a livekit-server http endpoint. Instead, you should integrate a <https://docs.livekit.io/getting-started/quickstarts/#backend|server SDK> into your own backend. You should expose an http endpoint which authenticates your user however you see fit and then uses our SDK to generate a room token.

The reason why you would have an own http endpoint which integrates the server SKD is: so that not ANYone can just join rooms and subscribe to tracks?

If I don't mind THIS is it ok to not use the server sdk but let the user from clients directly authenticate and join rooms?

This is correct, it’s this way primarily for security. If you don’t mind this, you could sign tokens yourself from the client using the APP ID and secret, but you’d have to port in the signing code from our server SDK. Is there a particular reason that setting up a server endpoint is difficult for you?

It is just another layer, which I thought is not needed

and the "security reason" is for me not 100% clear yet.. since when you have an open website (online chat) where everyone can join and subscribe - it should be ok (when it is labeld as such: hey everyone who knows the room name can see your stream)

"you’d have to port in the signing code from our server SDK." and now with this info, it looks like to omit the server sdk is even more work

In your case, the problem is someone could take your API key + secret and use it to generate tokens to be used in any other application. All of that usage would consume your resources and either potentially DDoS your server (self-hosted) or have you paying the bill (cloud).

The authentication mechanism we enforce here with server-side token generation is a pretty common security model to protect against situations like the above.

Does it make sense why authentication is needed even for your particular application?

yes I think so.

that it is possible to make join requests even from other software was the "aha" moment. but maybe just this could have been integrated into the core server; "allow requests only from defined domains"

no cross origin requests to the core server

How do you enforce no cross-origin requests from non-browser clients?

is cross-origin just a browser thing? hmm I mean the backend (core server) could check the request header and it will see if it was done from a browser and from a specific domain right?

I need to understand this hehe sorry hmm

a backend is browser/client agnostic.. so the backend can refuse requests from the client (in this case a browser with a website with specific domain)

so the backend knows: IsItFromMyWebsit:boolean

Yes, cross-origin requests can be spoofed. The browser enforces this through its security model, but there’s no foolproof way to determine the origin of a request without key signing.

That’s what we effectively do via JWT tokens. Determine the origin of a connection request, in a way that works across all client types.

so if this can be spoofed.. why should my custom server (that one which I need to have between the client and the livekit core server) not be spoofed as well?

Because you are using a key/secret that we agree upon.

aha because I need to have additional logic.. authentications and so on hmm

ok ok sloooowly I am getting there :smile: hehe

This is why you don’t want to share/leak your key and secret

I saw you are the cofounder - what a nice software you guys created! wow! :heart:

^ if yes, this could be explained in a nice diagram

You need to create a token off the Server SDK.

or if you would like you can use the livekit-cli to generate one

is this token the same token which I generate for a new user to join the room?

why is everything called just "token" hehe

`curl -X POST <http://localhost:7880/twirp/livekit.RoomService/ListRooms> \`
	`-H "Authorization: Bearer eyJhbGciOiJIUzI1.eyJleHAiOjp0cnVlfX0.B1jFc_D7ubGpwXHZ9KaRc" \`
	`-H 'Content-Type: application/json' \`
	`-d '{ "names": ["my-first-room"] }'`

{"code":"unauthenticated","msg":"permissions denied"}