LiveKit Community

hi,
currently setting video simulcastlayers is only supported within `RoomOptions`
```const roomOpts: RoomOptions = {
      publishDefaults: {
        simulcast,
        videoSimulcastLayers: [
          VideoPresets.h90,
          VideoPresets.h216,
        ],
      },```
All local (camera) tracks will then be published with those simulcast layers.
If this isn’t sufficient for your usecase, could you elaborate a bit on what your usecase looks like?

O thanks. I did see the example, but I am using the livekit-react to connect to the room. Eg:

```const {connect} = useRoom();

...

await connect(url, token, {...roomOpts})```
But I don't think we can change pass the videoSimulcastLayers in the ConnectOptions so incase of using the connect from livekit-react how can we use the simulcast?

ah, got it. in this case `const {connect} = useRoom(roomOpts)``  should work

With this, I can see 2 video streams in the <chrome://web-internals> but the stats are incorrect, the streams are not respecting the set constraints of width, height, fps, and bandwidth...

I am publishing with
`const track = createLocalVideoTrack({deviceId: id})`
`room.localParticipant.publishTrack(track)`
When creating the track I don't need to specify any resolution right, for the simulcast?(I did use the useRoom(roomOpts) and called const room = await connect())

Screenshot 2022-03-28 at 19.01.56.png

Note: I am using livekit-react v0.7.1 and I see these are the RoomOptions where the publishDefaults and many other params are missing.

my bad, these are currently two different kinds of RoomOptions. Looks like this warrants a fix within livekit-react.
I opened an issue for it <https://github.com/livekit/livekit-react/issues/37>

thanks for tracking this down!

Hi Adrian! Cloudfront only supports HLS (and HLS-derivative) video streaming, and currently LiveKit only streams video using WebRTC.  (WebRTC video is newer and doesn’t have the lag inherent in HLS video.)  Eventually we want LiveKit to support multiple egress protocols, but for now I don’t think you’ll be able to use LiveKit with Cloudfront.

Sanjay, can you be more specific about what you are seeing? What does the client code look like and what are you trying to accomplish?

thanks William! There'll for sure be updates to flutter SDK. It's fairly stable now though so we don't have anything urgent to address.

we are getting closer to a 1.0 release across the board though

<@U01KZ61GV5Z> There are some bug fixes that were implemented that fix some issues I'm having, such as device switching after joining a room.

`autoManageVideo` has been renamed to `adaptiveStream` some versions ago (try using v0.17.x)

And yes, it doesn't matter if it's within wrappers.. as long as intersection observer cannot detect it within the document's viewport, it would pause streams from the server.

Hey, not currently. Migration won't do too much for single homed Rooms (like livekit server supports today). We are putting that in place for multi-homed support in our cloud offering.

Ahh, gotcha. I'm also curious about the cloud offering. I currently run a multi-regional cluster for Patreon supporters of my Foundry module. Depending on the offering, I might end up steering people toward that instead.

Hi Team, good day.
I meet the project this week and i was trying to setup a multi-region cluster with 7 livekit-nodes + 1 redis node.

I am using digital ocean as VM provider, AWS Route53 as DNS provider (geolocation/latency) and docker to deploy the livekit service.
My problem is: i am not being able to generate SSL certificates using Let's Encrypt using docker livekit-caddy. I read the docs but dont find much about it.

Please, can someone help giving some information about how to accomplish this?

you should be able to keep track of this in two ways:
1. each participant as a joinedAt field, that keeps a timestamp of when they joined the room
2. listen to webhook events when participants join and keep track of it in your own DB

Hi Kay, LiveKit is designed for multi-user conferencing, for cases where traditional WebRTC doesn't scale. We also don't currently support a p2p mode (for say, conferences less than 4 participants) and then dynamically switch to SFU mode if a room grows larger than that.

Thank You.
I'm planning to implement p2p how do you suggest I should begin

There’s no client-side recording but I believe it’s possible at the server-side.

You can do it using our recording service
<https://docs.livekit.io/guides/recording/>

Recording is server-side, so yes it can be used with any client

Recording happens on the server, your client can be mac app

hi Wainny! welcome.

I know of at least a few people building telehealth using LiveKit. As a piece of self-hosted software, HIPAA compliance is something that applies to the hosting provider. We know folks that are using AWS or other platform's HIPAA compliance to ensure their solution is compliant.

From LiveKit's perspective, it's pure infrastructure that does not collect any user data. There isn't anything within the software that makes it non-compliant.

thanks a lot <@U01KZ61GV5Z>! Sorry for my late message reply to this!

Another question I have: is it possible to “get” the audio from a specific app (on mac) to then assign to a virtual audio input? 

kotlin is usable with java, but coroutines may be difficult to use in a java only project

06.PNG

we don't have any java only examples at the moment, have you tried using kotlin?

kotlin and java are interoperable, both can be used in the same project

that's because you'll need to use coroutines to call those methods

your calling code is java, which is why you're getting the errors

like I mentioned, can you try using kotlin in your project?

you can use kotlin for just the parts that need to use livekit, you don't need it for the entire project

alternatively, if you want to use java, you can call suspend functions from java like so:
<https://stackoverflow.com/questions/54022921/how-to-call-kotlin-suspending-coroutine-function-from-java-7>

connect(android.content.Context, java.lang.String, java.lang.String, io.livekit.android.ConnectOptions, io.livekit.android.RoomOptions, io.livekit.android.room.RoomListener, io.livekit.android.dagger.LiveKitOverrides, ...)' in 'io.livekit.android.LiveKit.Companion' cannot be applied to '(sq.rogue.rosettadrone.ConnectionActivity, java.lang.String, java.lang.String, io.livekit.android.ConnectOptions, io.livekit.android.RoomOptions, null, io.livekit.android.dagger.LiveKitOverrides)'

You're missing the Continuation argument needed for coroutine methods

```new Continuation&lt;Room&gt;() {
    @NonNull
    @Override
    public CoroutineContext getContext() {
        return null;
    }

    @Override
    public void resumeWith(@NonNull Object o) {

    }
}```
<@U01NZKTNE1H> I added argument but what to do in this argument

resumeWith is called with the result of the suspend function

Means return of function is getting in resumeWith Am i right?

<@U01NZKTNE1H> I got thi error when i integrate(0.9.1) and run project :- C:\Users\visha\.gradle\caches\transforms-2\files-2.1\bd40ee63d6b91443fe035430991406e6\core-1.7.0\res\values\values.xml:105:5-114:25: AAPT: error: resource android:attr/lStar not found.

That doesn't seem like it's related to livekit

Sounds like you're referencing something that doesn't exist

But when i remove live kit library then this error don't comes

<https://stackoverflow.com/questions/69021225/resource-linking-fails-on-lstar>

looks like you'll need to update your compileSdkVersion

Hey all, for those of you that are looking to build with react-native, we have some good news! Our React Native SDK is ready for you to checkout: <https://github.com/livekit/client-sdk-react-native/>

This is currently in beta, and builds on top of the JS SDK. Let us know how this is working out for you!

Hi <@U03B52AJCUA> :wave:, I’m assuming this is related to image processing on the video. There’s no Python SDK for LiveKit yet, but the LiveKit team is working on an egress solution. One of the features is to stream video bytes of a LiveKit room over WebSocket to your own server, which can be in any language. 

<@U02SGSET1RT> Thanks you for your response, let me describe my flow:
1. Python get RTSP stream from Camera 
2. Decode and do AI Prediction Tasks (Like Object Detection) 
3. Do Visualization on raw frame 
4. Stream the frame (In numpy array)  to [Janus](<https://github.com/meetecho/janus-gateway>) using [aiortc](<https://github.com/aiortc/aiortc/blob/main/examples/janus/janus.py>) library in python
5. Frontend join room and show the stream 
I would want to replace Janus to LiveKit but I haven’t known how to replace yet, could you help me?

I’m not actually with the LiveKit team, so I don’t think I can add much to this. Maybe <@U01KZ61GV5Z> can chime in :)

<@U02SGSET1RT> Thanks a lot for your support :+1:

Viet, we don't have a client SDK for python at the moment.. but if you are trying to publish streams with aiortc, take a look at <https://docs.livekit.io/internals/client-protocol|protocol> docs (it's covering version 3, but should still apply to the current version). It should be straight forward to implement the protobuf over websocket signaling for it.

<@U01KZ61GV5Z> Thanks for your update, I checked other language SDK but there are quite a lot of code to understand and convert. Do you have plan to have a Python SDK, I would help but I have not much experience with your client SDK’s interface, so it should be better having pre-defined interface to follow up

<@U01KZ61GV5Z> Ping you again, I would want to request Livekit team having a plan for Livekit Python SDK, I’m a big fan of Livekit Product and want to use Livekit in my development plan

Hey Viet, it'd be great to see a Python SDK, though this isn't something we currently have cycles for at the moment. I would highly encourage community contributions to one though!

<@U01KZ61GV5Z> After make some research, I think there would two better ways to make it work:
1. Using gstreamer, I am using way like this, stream to mediasoup server: <https://github.com/vpalmisano/mediasoupbin>, do you have a plan to support this
2. Create H265 stream, and use Go SDK the way like your example: <https://github.com/livekit/server-sdk-go> (Publishing tracks to Room), but is there any way to use direct H265 link instead of dump it to a file
Sorry that I don’t have enough Webrtc expertise to implement the first method, and I think it would be better if you have a sample for the second method, thanks!

ok, so we plan to support a wide range of formats for ingest, and I think that's the best way. With the Go SDK, your app would need to feed it pre-encoded h.264 buffers (we don't support H.265, but will support AV1 soon).

Here's what I think you'd want to do:

1. build a Go app that connects to livekit with the Go SDK, and reads from STDIN and publishes
2. run the CLI from your python app, and feed it h.264 buffers

<@U01KZ61GV5Z> Could you provide an example to feed H264 stream to Go SDK through CLI?

<@U03B52AJCUA> do you know if the data you want to publish to LiveKit is already encoded in H264 ? If it’s already encoded, I think I can create an example CLI for you. But if it’s just raw video, you’ll need to encode it either by using FFMPEG or GStreamer before passing it to the CLI.

<@U02SGSET1RT> Thanks a lot, let me describe more detail: I’m using python/C++ to process Video stream from RTSP camera, so I want to use Livekit to show the result to frontend, It’s hard to publish directly to Go SDK, so I think it’s better that I:
1. Encode the processed stream (Using gstreamer) to H264 encoded and make it public to local IP like this: <https://stackoverflow.com/questions/44131009/gstreamer-launch-rtsp-server-for-restreaming-ip-camera-h264>
       <https://stackoverflow.com/questions/26999595/what-steps-are-needed-to-stream-rtsp-from-ffmpeg>
2.    Using Go SDK, Input the  <http://localhost:1234/h264_encoded_stream/> and the Go SDK will handle all the rest, like <https://github.com/deepch/RTSPtoWebRTC>
Can you help me

I just want you to help at Step 2, because I don’t have expertise in Go

Instead of ./livekit-cli join-room --room stark-tower --identity Viet --publish output.ivf --publish output.ogg --fps 23.98, it should be ./livekit-cli join-room --room stark-tower --identity Viet --publish <http://localhost:1234/h264_encoded_stream/> --fps 23.98

The simplest workflow for me is as follows, which sounds like Approach 1:
1. Your Python/C++ code reads from RTSP camera
2. Same code draws boundary boxes on the video feed (this is the hardest part IMO), and encode it to your format (VP8 / H264)
3. Send the video feed to a Go app either via stdin (CLI) or UDP port (HTTP server)
4. The Go app publishes the modified video feed to LiveKit room

With Approach 2, I don’t think it’s possible today as I’m not sure if LiveKit CLI supports live video track. Maybe <@U01KZ61GV5Z> can chime in on this

thinking about this, it's probably easiest to modify `livekit-cli` to accept a `stdin` in put with the `--publish` flag.

then the python script could just use livekit-cli (since it handles everything else)

I think it’s a good idea <@U01KZ61GV5Z>. But specifically for <@U03B52AJCUA>’s use-case, I’m still unsure about the workflow. If there needs to be some processing on the video (drawing box for object detection) the Python script needs to encode it before passing it to CLI, so eventually GStreamer or a similar tool is required

I think that'll have to happen on the Python/caller side. video processing is out of scope for livekit today.

<@U03B52AJCUA> what I can recommend is to ensure the video feed leaving the Python code is indeed encoded. To test this, instead of sending it to a port, save it to a file and use FFMPEG to play it. If it’s an H264 stream, the file extension should be `.h264` .

Once you verify that the file can be played, then it’s just sending it to a local port instead of a file, and LiveKit CLI or something similar can push the processed stream to the LiveKit room

<@U02SGSET1RT> <@U01KZ61GV5Z>
I’m sure that Python script can encode the video stream to H264 already, but:
• If I save stream to a file like:
```ffmpeg -i &lt;input.mp4&gt; \
  -c:v libvpx -keyint_min 120 -qmax 50 -maxrate 2M -b:v 1M &lt;output.ivf&gt; \
  -c:a libopus -page_duration 20000 -vn &lt;output.ogg&gt;```
It’s ok to stream using Go SDK
• If I save stream to stream file (Because the program will loop live video forever): 
```ffmpeg -stream_loop -1 -i &lt;input.mp4&gt; \
  -c:v libvpx -keyint_min 120 -qmax 50 -maxrate 2M -b:v 1M &lt;output.ivf&gt; \
  -c:a libopus -page_duration 20000 -vn &lt;output.ogg&gt;```
The go SDK will not work
All I want is the Go SDK works with stream video/RTSP … to adapt with this scenario

<@U02SGSET1RT> <@U01KZ61GV5Z>
Is this hard to support Livekit Cli like this:
```./livekit-cli join-room --room stark-tower --identity Viet --publish <http://localhost:1234/h264_encoded_stream/> --fps 23.98```
The way I think is changing webrtc.NewTrackLocalStaticRTP <https://github.com/livekit/server-sdk-go/blob/ae1f6617c2249613f5533e8ea46913c12a186c9f/localsampletrack.go#L51|Link> to somehow like <https://github.com/pion/rtsp-bench/blob/master/server/main.go#L128|Link>

it's fairly simple to fetch from an URL, though would your source be live? or is it a file?

<@U01KZ61GV5Z> I think it’s better to be rtsp live, so it can stream even unprocessed Stream (H264 Encoded Camera) + Processed Stream (Decode Camera -&gt; Process AI -&gt; Visualization -&gt; Encode H264 to RTSP IP) -&gt; Use livekit cli to do the rest

I found the example above is outdate, I added the new one here about getting RTSP and RTC it: <https://github.com/deepch/RTSPtoWebRTC/blob/master/stream.go#L44|Link>
<@U01KZ61GV5Z>

Hi <@U01KZ61GV5Z> <@U02SGSET1RT> Can you take a look at this?

hey <@U03B52AJCUA> , I think what you are looking for is something we'd want to support eventually. However, it's not something we are currently working on. I'm happy to provide pointers if you are looking to try to build this now, but I'm afraid you'd have to wait for a few months before we'll have time to take a look at it.

<@U01KZ61GV5Z> Thanks for your response, I think it could be better if you can give me some advises that I can try to work on it :+1:

<@U01KZ61GV5Z> I’m happy to take this up after `universal-io` .

Anw, I really appreciate your great work, I already researched for a month until I found Livekit, it’s all my expected

<@U02UMFKPC6Q> this could be done fairly easy through the ClientSDK. Looking up if there's less than 2 participant in the room and achieve this solution through a timer and have an event being called out once it reaches your threashold.

I don’t know much about iPhone, but I know that some platforms like chrome and safari require a user gesture before autoplaying anything with sound

This is JS SDK ? any ideas <@U02EU2ZPTQU> :slightly_smiling_face:

Btw if you reject the microphone permission the screen will stay black, it start working only if you grant permission.

we have two black-screen fixes in the pipeline for safari that aren't released yet, but there's a preview available on npm. <@U038X6AQJKD> could you try changing your livekit-client dependency to `"livekit-client": "next"` to see if this fixes the issues you are seeing?

Are you using the react components of `livekit-react` to render the video or is it a custom solution?

I use &lt;video&gt; tag to render stream. From livekit-react I onlu use useParticipant, useRoom and ControlButton (patched little bit)

good to know, make sure that you have the `muted` and `playsInLine` attributes set on the video element, otherwise safari is a lot more likely to prevent auto play

ohh, great will test this too, will keep you posted

Seem that switching to next version and mute + playsInline helped! Thank you

<@U038X6AQJKD> that’s great to hear. we just released v0.18.0 with the fixes, it makes sense to switch to that version instead of the pre-release in order to stay up to date!

You can access the raw peerconnection stats by using `room.engine.subscriber.pc` and `room.engine.publisher.pc`

We are also building an analytics product for the upcoming hosted offering.  That should give you insights to the end-user experience. We'd be happy to chat if that's more what you are looking for.

Yeah. That's exactly what we are looking at. I also looked at <http://callstats.io|callstats.io> but it seems to be focussed only on confrencing. We have a slightly different use case.

When I add multiple tracks to a participant using server-sdk-go, do I need to be concerned about synchronization between tracks or is that handled by livekit? <https://github.com/pion/webrtc/discussions/1825>
This pion issue mentions sender reports but I also saw a commit in Pion itself implementing sender reports.
<https://github.com/pion/interceptor/commit/bc30165ae456af3e8a9ae25e1c3d25dc3a7ccc41>

Ah, I see there is a recording beta right now. When this recording happens, does it only record the output, or is it possible to record all tracks separately for optimal post-processing?

recording individual tracks isn’t available yet, but it’s something I’ll be working on very soon

Thanks for the response <@U01UB0LHTTQ>. Is it possible to route the tracks to audio inputs/outputs in the OS on the server or client side?

It should be possible, but I wouldn’t know how. You’d need to join the room and subscribe to the tracks using one of the server sdks, then find a way to route them

Gotcha. thanks! Do you have a time frame, ooc, for working multi-track recording? No worries if not!

<@U03BEG6STE2> very broadly, this will ship in Q2, but don’t have specific dates yet. If it helps, it is high on our list of priorities!

<@U02T8UUS3J8> we need to improve the docs here, but by default the load tester simulates a realistic layout where subscribers are not able to render all published tracks. By default it simulates the <https://github.com/livekit/livekit-cli/blob/main/pkg/loadtester/loadtester.go#L38|speaker> layout, where it disable the tracks that are not displayed. You can simulate another layout when running the load test yourself.

When using adaptive stream, LiveKit handles these stream pauses/resolution adaptation automatically.

screen-capture.webm

Do you have any plans to support SVC (scalable video coding) instead of simulcast ?

Simulcast generally doesn't work that way. A participant publishes a fixed number of streams at varying resolutions and the SFU manages which resolution stream a subscriber should receive based on their network conditions. The number of simulcast layers certainly does not (and could not) scale with the number of participants.

And I can say from digging into the code, livekit does 1, 2, or 3 streams when using simulcast depending on the highest resolution selected. If you want 3 streams, your main resolution needs to have a height of at least 720px.

Also, with dynacast enabled, the client will only send streams that are actually in use. So, if all clients have thumbnail images up, the sender may only send the lowest resolution. This even scales down to zero streams if no other participant is viewing that user's video.

So, if we have just 5 participants, 1 participant generate 4*x video streams ?