LiveKit Community

my guess is your yaml is not formatted incorrectly. check your indentations and use a <http://www.yamllint.com/|YAML parser> to validate

Ekran Resmi 2021-12-23 12.36.57.png

I have turn servers but not see in config file
Thanks guys

Ouch. I hadn't upgraded yet. That's going to break my use as well.

similar thing happened to me when I tried to make my own video renderer, but the TrackStreamUpdatedEvent is also marked as internal. I was forced to use the video_track_renderer provided by client-sdk-flutter

we moved the creation APIs away from being a global to provide better isolation for apps that create multiple rooms.
I see how that could impact use cases of creating local tracks prior to connect though.
Here are a few ways to proceed, let me know ur thoughts:
• you could create the Room and utilize the scoped create track options *before* connecting to it.
• introduce back global functions for backwards compatibility, but using them wouldn't respect any default capture settings
• create your own tracks with getUserMedia, but still be able to publish it into livekit (publishTrack can take in a mediastreamtrack)

With any of these options, are tracks transferable to another room? One of the benefits of the recent change of being able to not stop tracks when unpublished was that I could implement breakout rooms without having to tear down and recreate the tracks.

I think the 3rd option would be the best, allow publishVideoTrack to take a mediaStreamTrack

yes. they are transferrable across rooms. track publications are tied to a room, but tracks don't carry room state

<@U02C10ZEP8R> fwiw, for breakout rooms, I think you could just create two separate rooms, mute the previous one; and put the user into a new room. 

moving tracks across rooms should work. but might be tricky to coordinate everything.. vs having both rooms capture independently.

The way it works today is that I just leave the existing room and join the new room. That actually works better for my setup and means I don't need to track the state of two rooms at once. I just don't want to have to re-create tracks when doing so since I would sometimes hit an edge case where releasing a track and starting a new one would request the camera before the OS had completely released the hardware, and it would fail to start the camera back up on the new track. It's been much faster and cleaner since I've been able to keep tracks running and just attach them to the new room.

I haven't tried the new release at all yet, though. I can play with it in the next few days and provide better feedback :slightly_smiling_face:

got it, you are not moving back &amp; forth, but a one time transfer. that makes total sense :slightly_smiling_face:

I think there are some pretty handy things included in the `LocalTrack` type (like source). It would be a pity to lose those if the way to go forward is to provide mediastreamtracks directly to the publish function.
My vote would be towards option two. Especially in applications with multiple rooms, I think the ability to manage tracks separatly from rooms is pretty nice. And if livekit has its own type for tracks, I'd be happiest if I can consistently deal with this type throughout the whole app.

we'll look into it. could you try disabling optimizeVideo in client options? <https://github.com/livekit/client-sdk-flutter/blob/2b375033e9cb2cd1b7eba2e547248b3b35d3c9d7/lib/src/options.dart#L53|https://github.com/livekit/client-sdk-flutter/blob/2b375033e9cb2cd1b7eba2e547248b3b35d3c9d7/lib/src/options.dart#L53>

yes, that fixes the issue! thanks david, I'll update the report.

thanks for confirming! we will fix the root cause shortly.

hey alex, yes you can. TrackSubscribed gives you access to `webrtc.TrackRemote` , you can call the method: `trackRemote.Codec().MimeType`

for unsubscribed tracks, we currently don't store the codec in the metadata layer. though it might be interesting to have in there

thanks <@U01KZ61GV5Z>, it'd be nice to have the codec info in <https://pkg.go.dev/github.com/livekit/protocol/livekit#TrackInfo|TrackInfo>

image.png

the api changed in 0.15.0
<https://github.com/livekit/client-sdk-js/releases/tag/v0.15.0>

Top level `connect` will still be supported for backwards compatibility. Several functions have been moved as well:
• `createLocalTracks` -&gt; LocalParticipant.createTracks
• `createLocalVideoTrack` -&gt; LocalParticipant.createTracks({video: true})
• `createLocalAudioTrack` -&gt; LocalParticipant.createTracks({audio: true})
• `createLocalScreenTracks` -&gt; LocalParticipant.createScreenTracks


Any `vaapi` errors should be safe to ignore, we don’t use it, and docker shouldn’t need any mappings or special privileges

are there any other logs in your recorder or livekit-server about the failure?

`livekit-server` should print a `recording ended` message that contains the error

<@U01UB0LHTTQ>: Here is the last bit of my log

```live-kit-server_1  | {"level":"error","ts":1640382886.2863224,"logger":"livekit-recorder","caller":"logger/logger.go:34","msg":"error running pipeline","error":"Failed to change state to PLAYING","stacktrace":"<http://github.com/livekit/protocol/logger.Errorw\n\t/root/go/pkg/mod/github.com/livekit/protocol@v0.11.1-0.20211215011801-f77609470e70/logger/logger.go:34\ngithub.com/livekit/livekit-recorder/pkg/recorder.(*Recorder).Run\n\t/workspace/pkg/recorder/recorder.go:92\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).handleRequest.func1\n\t/workspace/pkg/service/handler.go:88|github.com/livekit/protocol/logger.Errorw\n\t/root/go/pkg/mod/github.com/livekit/protocol@v0.11.1-0.20211215011801-f77609470e70/logger/logger.go:34\ngithub.com/livekit/livekit-recorder/pkg/recorder.(*Recorder).Run\n\t/workspace/pkg/recorder/recorder.go:92\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).handleRequest.func1\n\t/workspace/pkg/service/handler.go:88>"}
live-kit-server_1  | {"level":"error","ts":1640382886.286414,"logger":"livekit-recorder","caller":"logger/logger.go:34","msg":"recording failed","recordingID":"RR_Pjr9amXhjS2U","error":"Failed to change state to PLAYING","stacktrace":"<http://github.com/livekit/protocol/logger.Errorw\n\t/root/go/pkg/mod/github.com/livekit/protocol@v0.11.1-0.20211215011801-f77609470e70/logger/logger.go:34\ngithub.com/livekit/livekit-recorder/pkg/service.LogResult\n\t/workspace/pkg/service/handler.go:132\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).handleRecording\n\t/workspace/pkg/service/handler.go:42\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).Run\n\t/workspace/pkg/service/service.go:86\nmain.runService\n\t/workspace/cmd/server/service.go:50\nmain.run\n\t/workspace/cmd/server/main.go:58\ngithub.com/urfave/cli/v2.(*App).RunContext\n\t/root/go/pkg/mod/github.com/urfave/cli/v2@v2.3.0/app.go:322\ngithub.com/urfave/cli/v2.(*App).Run\n\t/root/go/pkg/mod/github.com/urfave/cli/v2@v2.3.0/app.go:224\nmain.main\n\t/workspace/cmd/server/main.go:51\nruntime.main\n\t/usr/lib/go-1.16/src/runtime/proc.go:225|github.com/livekit/protocol/logger.Errorw\n\t/root/go/pkg/mod/github.com/livekit/protocol@v0.11.1-0.20211215011801-f77609470e70/logger/logger.go:34\ngithub.com/livekit/livekit-recorder/pkg/service.LogResult\n\t/workspace/pkg/service/handler.go:132\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).handleRecording\n\t/workspace/pkg/service/handler.go:42\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).Run\n\t/workspace/pkg/service/service.go:86\nmain.runService\n\t/workspace/cmd/server/service.go:50\nmain.run\n\t/workspace/cmd/server/main.go:58\ngithub.com/urfave/cli/v2.(*App).RunContext\n\t/root/go/pkg/mod/github.com/urfave/cli/v2@v2.3.0/app.go:322\ngithub.com/urfave/cli/v2.(*App).Run\n\t/root/go/pkg/mod/github.com/urfave/cli/v2@v2.3.0/app.go:224\nmain.main\n\t/workspace/cmd/server/main.go:51\nruntime.main\n\t/usr/lib/go-1.16/src/runtime/proc.go:225>"}
live-kit-server_1  | {"level":"debug","ts":1640382886.3740387,"logger":"livekit-recorder","caller":"logger/logger.go:22","msg":"recorder waiting"}```


what machine is this running on? I’ve had problems running it on the M1 macbook pro

you might need to set `GST_DEBUG` to `3` in your env to get more detailed logs, unfortunately we don’t get much info from that error (it comes from the following):
```	// start playing
	err := p.pipeline.SetState(gst.StatePlaying)
	if err != nil {
		return err
	}```

I’m running on an m5 in aws. Not macbook. I’ll try adding that debug flag to my deployment for the recorder and report back

Ok I tried adding GST_DEBUG=3. I don’t see anything else in the log

```live-kit-server_1  | 0:00:01.142329362 [32m    1[00m 0x7f4de8367240 [33;01mWARN   [00m [00m               pulse pulsesrc.c:1013:gst_pulsesrc_open:&lt;pulsesrc0&gt;[00m error: Failed to connect: Connection refused
live-kit-server_1  | {"level":"error","ts":1640751844.3875778,"logger":"livekit-recorder","caller":"logger/logger.go:34","msg":"error running pipeline","error":"Failed to change state to PLAYING","stacktrace":"<http://github.com/livekit/protocol/logger.Errorw\n\t/root/go/pkg/mod/github.com/livekit/protocol@v0.11.1-0.20211215011801-f77609470e70/logger/logger.go:34\ngithub.com/livekit/livekit-recorder/pkg/recorder.(*Recorder).Run\n\t/workspace/pkg/recorder/recorder.go:92\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).handleRequest.func1\n\t/workspace/pkg/service/handler.go:88|github.com/livekit/protocol/logger.Errorw\n\t/root/go/pkg/mod/github.com/livekit/protocol@v0.11.1-0.20211215011801-f77609470e70/logger/logger.go:34\ngithub.com/livekit/livekit-recorder/pkg/recorder.(*Recorder).Run\n\t/workspace/pkg/recorder/recorder.go:92\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).handleRequest.func1\n\t/workspace/pkg/service/handler.go:88>"}
live-kit-server_1  | {"level":"error","ts":1640751844.3876972,"logger":"livekit-recorder","caller":"logger/logger.go:34","msg":"recording failed","recordingID":"RR_bfUGd254sbF6","error":"Failed to change state to PLAYING","stacktrace":"<http://github.com/livekit/protocol/logger.Errorw\n\t/root/go/pkg/mod/github.com/livekit/protocol@v0.11.1-0.20211215011801-f77609470e70/logger/logger.go:34\ngithub.com/livekit/livekit-recorder/pkg/service.LogResult\n\t/workspace/pkg/service/handler.go:132\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).handleRecording\n\t/workspace/pkg/service/handler.go:42\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).Run\n\t/workspace/pkg/service/service.go:86\nmain.runService\n\t/workspace/cmd/server/service.go:50\nmain.run\n\t/workspace/cmd/server/main.go:58\ngithub.com/urfave/cli/v2.(*App).RunContext\n\t/root/go/pkg/mod/github.com/urfave/cli/v2@v2.3.0/app.go:322\ngithub.com/urfave/cli/v2.(*App).Run\n\t/root/go/pkg/mod/github.com/urfave/cli/v2@v2.3.0/app.go:224\nmain.main\n\t/workspace/cmd/server/main.go:51\nruntime.main\n\t/usr/lib/go-1.16/src/runtime/proc.go:225|github.com/livekit/protocol/logger.Errorw\n\t/root/go/pkg/mod/github.com/livekit/protocol@v0.11.1-0.20211215011801-f77609470e70/logger/logger.go:34\ngithub.com/livekit/livekit-recorder/pkg/service.LogResult\n\t/workspace/pkg/service/handler.go:132\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).handleRecording\n\t/workspace/pkg/service/handler.go:42\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).Run\n\t/workspace/pkg/service/service.go:86\nmain.runService\n\t/workspace/cmd/server/service.go:50\nmain.run\n\t/workspace/cmd/server/main.go:58\ngithub.com/urfave/cli/v2.(*App).RunContext\n\t/root/go/pkg/mod/github.com/urfave/cli/v2@v2.3.0/app.go:322\ngithub.com/urfave/cli/v2.(*App).Run\n\t/root/go/pkg/mod/github.com/urfave/cli/v2@v2.3.0/app.go:224\nmain.main\n\t/workspace/cmd/server/main.go:51\nruntime.main\n\t/usr/lib/go-1.16/src/runtime/proc.go:225>"}
live-kit-server_1  | {"level":"debug","ts":1640751844.4618165,"logger":"livekit-recorder","caller":"logger/logger.go:22","msg":"recorder waiting"}```


<@U01UB0LHTTQ>: I just noticed I have the warning gst_pulsesrc_open Failed to connect. Is that also normal?

that must be the issue. `pulsesrc` is the audio source for the recorder

I’m not seeing anything about m5 that jumps out to me as an issue, can you try running it on a c5 and see if it still fails for you?

<@U01UB0LHTTQ>: Seems the result is the same. I assume this maybe because there is no actual sound device installed on the instance. Just searching around I did find this thread. <https://askubuntu.com/questions/60837/record-a-programs-output-with-pulseaudio/60856#60856>. Looks like you can configure a null device for pulse audio to capture the input. Perhaps something that forwards the chrome audio to a null device that can be captured? Don’t know I’m not the expert here

Came from this thread - <https://www.reddit.com/r/aws/comments/ip15mq/is_it_possible_to_record_the_speaker_output_of_an/>

I’ll have to look into this. It does load a null sink (<https://github.com/livekit/livekit-recorder/blob/main/entrypoint.sh#L11>), and works on our production server on AWS

What are you using for your production server?

Ok just thinking about this if I create a docker-compose to pass config and other parameters does this shell script get called?
Say if I override command or entrypoint in the docker-compose file

<@U01UB0LHTTQ>: I think the problem is I’m trying to run it via docker-compose but if I use the livekit-recorder exe directly it doesn’t setup the necessary gst-sink.
It looks like you can only set the config via an environment variable as the entrypoint.sh doesn’t take any command line parameters. And when run that way it expects a request.json. Is there a way to just spin up the container supplying a config without starting a request?

AWS C5 for our prod server. I haven't used docker compose so I'm not sure about the shell script question

If you set the `SERVICE_MODE` env var to `1` it will start in service mode and listen for requests from redis

Ok I think I figured it out. I noticed the SERVICE_MODE environment variable
It’s running and takes my recording request. But using the go sdk when I issue the StartRecordingRequest it starts the recording. But returns an error and doesn’t give me the recordingId. Although the recording is running :sweat_smile:

This is the error - twirp error internal: Internal service panic

Can you upgrade your server sdk to `v0.8.2` (I just pushed the new tag)

I think v0.8.1 and older are missing some protocol updates for latest server/recorder

Looks like I’m getting a panic in the livekit-server

```	/usr/local/go/src/net/http/server.go:2994 +0x39b
live-kit-server_1  | 2021/12/30 01:39:18 http: superfluous response.WriteHeader call from <http://github.com/urfave/negroni.(*responseWriter).WriteHeader|github.com/urfave/negroni.(*responseWriter).WriteHeader> (response_writer.go:53)
live-kit-server_1  | [negroni] PANIC: interface conversion: livekit.isStartRecordingRequest_Input is *livekit.StartRecordingRequest_Url, not *livekit.StartRecordingRequest_Template
live-kit-server_1  | goroutine 588960 [running]:
live-kit-server_1  | <http://github.com/urfave/negroni.(*Recovery).ServeHTTP.func1(0x7f7c2cac6ab8|github.com/urfave/negroni.(*Recovery).ServeHTTP.func1(0x7f7c2cac6ab8>, 0xc00000ec10, 0xc0000bc780, 0xc0001dc800)
live-kit-server_1  | 	/go/pkg/mod/github.com/urfave/negroni@v1.0.0/recovery.go:159 +0xd2
live-kit-server_1  | panic(0xea9d60, 0xc0001fe2d0)
live-kit-server_1  | 	/usr/local/go/src/runtime/panic.go:965 +0x1b9
live-kit-server_1  | <http://github.com/livekit/protocol/livekit.ensurePanicResponses(0x111f0b8|github.com/livekit/protocol/livekit.ensurePanicResponses(0x111f0b8>, 0xc0001ede30, 0x7f7c2cac6ab8, 0xc00000ec10, 0x0)
live-kit-server_1  | 	/go/pkg/mod/github.com/livekit/protocol@v0.11.1/livekit/livekit_recording.twirp.go:1701 +0x1fd
live-kit-server_1  | panic(0xea9d60, 0xc0001fe2d0)```


I thought we could use either Url or Template in the StartRecordingRequest

oh wow. Url requests are just broken in lk-server

I’ll have an update in a couple minutes

Just pushed the fix, we just need to run tests and verify everything and then we’ll push server v0.15.2

easier question: is it possible to get the orientation of a track, or the actual width and height?

I don’t think flutter_webrtc provides this but I know with the native ios sdk it is possible.
I would need to make a PR to expose this to flutter.

<@U02MAJLGN65> it would be nice in order to calculate the position of UI elements inside the video area. Imagine you have a container (grey area) where the VideoTrackRenderer is placed, if "contain" fit is used, we can never know the actual size that the video (blue area) is going to be rendered at. We need that size/orientation info in order to place a label or any other widget inside of it (username in this example).

I'm thinking through dynamic layouts that change during the lifetime of a room. E.g. stages with participants added/removed. Which would require custom events and then also some add'l storage of a current layout of a room (redis would be ideal for that)

there is already room metadata that might be useful in this kind of scenario. you can set it via the server sdks and it triggers an event on each client within that room

<https://docs.livekit.io/guides/server-api#updateroommetadata>

0.15.1 includes a stream allocator, which adapts to a subscriber's receiving conditions. that could explain the slower startup time. this allocator is used for all camera tracks (but turned off for screenshare and audio)

how long is the delay that you typically observe?

since 0.15 ships with a few significant changes, it def could be related. it's pretty complex to deal with the wide range of bandwidth situations and it's quite possible there's a regression here for some use cases

Yeah definitely smells like the stream allocator not able to sustain tracks

<@U01KZ61GV5Z> It depends. Some were popping up within a few seconds, some weren't ever showing after a couple of minutes of testing.

Is it possible to disable the adaptive stream allocator? For my use-case, I'd much rather just tell it to always try to send the published streams all of the time. I get the need for adaptive streaming for large conference rooms, but in my case users will likely just complain that it's broken if some videos don't appear.

There will be some cases where it legitimately should be cutting off video to avoid client bandwidth congestion. However, I'm already giving them an indicator of connection quality using the data coming back from LiveKit - so if it is too much for a client, I'd rather it get shown in that way instead of just cutting off the track.

<@U02C10ZEP8R> that makes sense. I think as with other advanced features, it would make sense for us to let folks opt-out.

question: for the cases that you saw, are the network actually congested? or was there plenty of headroom with bandwidth?

<@U01KZ61GV5Z> There should be plenty of bandwidth considering the small resolution of the tracks. However, I was opening up 8 different sessions all from the same desktop just for testing. I've seen tests like that mess with bandwidth estimates. I haven't looked into the logic of how livekit is handling that part yet.

Yeah, bandwidth estimation will suffer when the client machine is overloaded. We are using REMB and allowing browsers to do the bandwidth estimation. Even if we switch to send side bandwidth estimation, it will still not be perfect. Bandwidth estimation (at least the Google Congestion Control we are working with) looks at difference in one-way-delay (OWD). Basically, send side time stamps the packets and receive side also does the same. Under ideal conditions, the difference of difference will be 0 (i. e. send side sending a packet every X ms should have receive side also seeing the same X ms between packets). Any increase in the receive side in inter packet gap means there is some bottleneck link in between queuing up packets and the bandwidth estimate is reduced. With an overloaded client, it is entirely possible that packet gap is unpredictable leading to congestion detection.

Screen Shot 2021-12-29 at 3.51.49 PM.png

it sounds like you are publishing data, but not audio/video? what server/client versions are you using?

Ean and I are working on this together. We're using v0.13 (of the server). And yes trying to send a string payload to the room without having published audio or video stream yet.

Versions:
```    "livekit-client": "^0.15.0",
    "livekit-server-sdk": "^0.5.7",```


I'm unable to reproduce this with the sample app. though this is testing against server v0.15.1.

that might be the primary difference here, can you give the latest server a try? Can you reproduce with the <https://github.com/livekit/client-sdk-js#sdk-sample|sample app> ? that'd be the easiest for us to get a repro here.

Is there documentation on "the right" way to update the server? We deployed with this strategy <https://docs.livekit.io/deploy/vm>

I'm guessing run this:
```  - export LIVEKIT_VERSION=v0.15.1 # set LiveKit version
  - curl -o /etc/systemd/system/docker.livekit-server@.service -O <https://raw.githubusercontent.com/livekit/livekit-server/master/deploy/docker.livekit-server%40.service>
  - systemctl enable docker.livekit-server@${LIVEKIT_VERSION}
  - systemctl start docker.livekit-server@${LIVEKIT_VERSION}
  - curl -o /etc/nginx/conf.d/livekit.conf -O <https://raw.githubusercontent.com/livekit/livekit-server/master/deploy/livekit.nginx.conf>
  - systemctl restart nginx```

I'm able to receive data from the example app on my local project, but not the other way around.

When I try to send data like this:
```      const encoder = new TextEncoder()
      const msg = encoder.encode('testing')
      room.localParticipant.publishData(msg, DataPacket_Kind.RELIABLE)```
I still get the error:
```Uncaught (in promise) Error: could not establish publisher connection, state new
    at RTCEngine.&lt;anonymous&gt; (RTCEngine.js:372)
    at Generator.next (&lt;anonymous&gt;)
    at fulfilled (RTCEngine.js:5)```

<@U01KZ61GV5Z> it appears as though the error is resolved by reducing the rate at which I send data. I reduced the frequency from 60fps to 10fps and was able to get data to send successfully

is there a way to send data faster than 10fps?

what kind of data are you sending? what is the bitrate?

I was sending a test string message:
```const msg = encoder.encode('testing')```
Not sure about the bitrate

I see, so a simple message 30 times a second had failed for you? interesting..

what if you tried introducing a delay between the first message and subsequent messages? say wait 5s after the initial message. The error seems to indicate that the connection isn't fully established.

I can successfully send the message at 10fps

So as long as there is 100ms delay between the messages everything works fine

I understand that.. I think that might be only applicable to the *first* message and *second* message.. because it takes time to establish the connection when you first call sendData

Ah, very interesting… thanks for the suggestion, I'll try that and see what happens

~I'm running into the same issue trying to sync state with other peers. can you think of a way to work around this with a promise that resolves once the datachannel is good to go rather than an arbitrary timeout? my error message is slightly different however: `Uncaught (in promise) Error: could not establish publisher connection, state undefined`~
-&gt; buggy code on my side. was trying to send with an outdated room object at times

<@U01KZ61GV5Z> your suggestion worked! Thanks so much for the assist on this, we really appreciate you!

<@U02S26MTY86> glad to hear that worked! :raised_hands:

let me see if there's a way we could avoid that

<@U01KZ61GV5Z> for what it's worth, here is how I'm currently handling it. It seems to work alright, but I'm open to a better way haha

I have a Room.js file with this function:
```async function sendData(room, data) {
    if (!establishedConnection) {
        setTimeout(() =&gt; {
            //try again later
            sendData(room, data)
        }, 100)
        return
    }
    const dataString = JSON.stringify(data)
    const msg = encoder.encode(dataString)
    await room.localParticipant.publishData(msg, DataPacket_Kind.RELIABLE)
}```
Let me know if there are any glaring issues with this, or if you have come up with a better solution, thanks!

hey, give 0.15.2 a try, I made a couple of fixes that should improve initial data reliability

Hey Vik! :wave:

We don’t support P2P at this time, but may consider adding it in the future. Can I ask what your needs are for P2P?

Hi <@U01KZ0SCDKL>,  Considering approach of one to one call at concurrency of above million users in SFU Unit,
will be consuming a CPU/MeM/Data right ?

Fall back P2P or P2P only, will be helpful

Vik, I’m not sure I fully understand, yet. From what I’m hearing, it sounds like you’re looking to build an application for one-to-one calls, but would like a P2P fallback for cost concerns?

Application has both Group and P2P, in case of
- A group call reduced to 2 people - (P2P Fallback)
- Most of the call will be in P2P  - (only P2P with TURN)

hello, I think it should be, the `localParticipant` object has a `publishVideoTrack` method

Yes. But it requires to create LocalVideoTrack where I cannot simply put my rtc video track

there’s a constructor in LocalVideoTrack that takes the webrtc track along with the source and capturer parameters, the name I’d think it can just be set to “video” and since you’re not creating the track through LiveKit’s SDK I assume the `LocalVideoTrackOptions`  can be created with the default constructor, that should wrap your webrtc video track as a localVideoTrack object to publish :thinking_face:

for `LocalVideoTrack`, at the moment it takes in a bunch of parameters needed for `restartTrack()` method, which you might not need. You should be able to pass in simple fakes for those

```
    // these can be faked
    private val peerConnectionFactory: PeerConnectionFactory,
    private val context: Context,
    private val eglBase: EglBase,
    private val defaultsManager: DefaultsManager,
    private val trackFactory: Factory,```

this can be cleaned up to better support user-created tracks, but until then, this workaround should do it

Hope you don't mind me jumping in here, but I'm doing something similar and wondering how to fake a LocalVideoTrack.Factory?

This is my attempt but I think I'm missing something. In trying to create a fake LocalVideoTrack.Factory, I find myself needing to return a LocalVideoTrack object (which is the thing I was originally trying to create via constructor and that required the LocalVideoTrack.Factory parameter in the constructor)

```import io.livekit.android.room.track.LocalVideoTrack
import io.livekit.android.room.track.LocalVideoTrackOptions
import org.webrtc.*

class FakeLocalVideoTrackFactory : LocalVideoTrack.Factory {

    override fun create(
        capturer: VideoCapturer,
        source: VideoSource,
        name: String,
        options: LocalVideoTrackOptions,
        rtcTrack: VideoTrack
    ): LocalVideoTrack {
        return ?
    }
}```

<@U02QH52M4A3> mmm, if you have the other objects/fakes available, you should be able create it with the same arguments?

<@U01NZKTNE1H> I don't think so, because I would need a LocalVideoTrack.Factory to pass in the constructor. It's kind of like a cyclic dependency. In order to create the LocalVideoTrack, I need to have a LocalVideoTrack.Factory and in order to have a LocalVideoTrack.Factory fake, I need to create and return a LocalVideoTrack (Requiring a <http://LocalVideoTrack.in|LocalVideoTrack.Factory >object) in the create() override.

Actually, it seems I can simply create a lateinit var for the LocalVideoTrack and just not initialise it since I'll never call the create() function for my FakeLocalVideoTrack.Factory class.

```class FakeLocalVideoTrackFactory() : LocalVideoTrack.Factory {

    private lateinit var localVideoTrack: LocalVideoTrack


    override fun create(
        capturer: VideoCapturer,
        source: VideoSource,
        name: String,
        options: LocalVideoTrackOptions,
        rtcTrack: VideoTrack
    ): LocalVideoTrack {
        return localVideoTrack
    }
}```


that would be a great option to have. feel free to make a PR for it. It would be something similar to <https://github.com/livekit/livekit-recorder/pull/23/files|this>, when a community member added support for Azure.

FYI just added this as it was something I was needing as well: <https://github.com/livekit/livekit-recorder/pull/43>

I did notice that there is a <https://gstreamer.freedesktop.org/documentation/gs/gssink.html?gi-language=c|gssink> which i might look into in the futre to do streaming writes to gcp

awesome stuff <@U02Q4J8KS85>, we'll take a look and hope to get it m erged in soon

streaming writes would be fantastic.. though ideally the writes take place within our app instead of gstreamer, to ensure consistency in behavior across storage backends

gotcha,  i'll have to learn a little more about gstreamer pipelines and how to do incremental writes from a sink, but definitely something i'll look at/work on in the future if it's not there yet

there are some annoyances too with mp4, since it needs to rewrite the headers after recording is complete to ensure it's easily streamable

Hi William! welcome. We are actually planning to do this. I'm guessing you are trying to build a live streaming experience? Is latency the reason why you are looking at WebRTC?

In part. My main reason is for adoption. I'm using flutter to create a streaming platform and would like users to be able to switch easily on OBS or similar.

Pion allows for that already right? RTMP-&gt;WebRTC

Yeah that makes sense. Pion itself doesn't deal with the RTMP part. the <https://github.com/livekit/livekit-recorder/pull/23/files|example here> publishes to Twitch via FFMPEG. There needs to be a chunk of code that acts as an RTMP server (that part isn't trivial and there isn't a Go OSS package that does that well AFAIK). We are looking to build this natively within the livekit ecosystem.

Great thanks! The OBS virtual camera can work in the meantime.

I don't have a deep insight yet, but maybe <https://github.com/yutopp/go-rtmp> could be a good starting point

I don’t have an answer for you but note that there are sdks in node and go that you can use instead of calling the api directly <https://docs.livekit.io/references/server-sdks>

You should be able to post to them. The Server SDKs automates signing of Authorization token, but it's effectively the same. <https://github.com/livekit/server-sdk-js/blob/main/src/TwirpRPC.ts#L46|Here's> how we are doing it within the JS server SDK.

I think I see the issue, you'd need to send a valid content type and request body: `application/json` and `{}`

<@U01KZ61GV5Z> Thank you! I get it to working. Thanks for updating <https://github.com/livekit/livekit-docs/commit/b07503d269f0899b2bf5fb67d59d178729e7667f|docs> too!

Thanks for bringing this one to our attention, Alex. I'll make an update to the guide/documentation.

hey Mike, this seems like an issue related to docker networking. are `10.128.0.47` and `10.0.1.47` the same hosts? See if you can connect to the redis address without livekit:

```docker run -it --rm redis redis-cli -h 10.0.1.47```

:man-facepalming: Thanks for the second set of eyes.  A pure typo!  Seems to be working now.

Although now when I try to run the CLI I still get an issue:

~/go/bin/livekit-cli start-recording --url <https://my.server> --api-key xyz --api-secret 123 --request room.json
twirp error internal: recording not configured (redis required)

FYI, 10.0.1.47:6379 is mapped to my redis master.

hey Mike, which version of livekit are you running?

```REPOSITORY               TAG       IMAGE ID       CREATED       SIZE
livekit/livekit-server   latest    54a0cd892e65   11 days ago   33.2MB```

it should tell you when you attempt to start the server.. i.e.
```starting LiveKit server	{"addr": ":7880", "nodeID": "ND_Qt0g2vWD", "nodeIP": "192.168.103.156", "version": "0.15.1", "rtc.portTCP": 7881, "rtc.portUDP": 7882, "portPrometheus": 6789, "region": "myregion"}```

```2022-01-03T20:20:45.873Z	INFO	livekit	logger/logger.go:26	starting LiveKit server	{"addr": ":7880", "nodeID": "ND_5lmgZVf9", "nodeIP": "172.17.0.2", "version": "0.15.1", "rtc.portTCP": 7881, "rtc.portUDP": 7882}```

great, thanks. will take a look to see if we can reproduce.

on the basis of nothing at all, I wonder if it might have something to do with my using authentication, rather than running wide open

do you see this line printed? `using multi-node routing via redis	{"addr": "localhost:6379"}`

```2022-01-03T20:22:31.888Z	INFO	livekit	logger/logger.go:26	using multi-node routing via redis	{"addr": "10.128.0.47:6379"}```

this is really odd.. I'm unable to reproduce this one here. it doesn't appear to be a redis-auth issue either

Are you running redis locally on the same server as livekit-server?  Because I am not.  I wonder if that could make a difference.

I think I'm going to try simplifying everything by installing livekit-server directly (i.e. not dockerized) on a server with redis and livekit-cli all running together and see what happens

it was not.. the code path shouldn't care about where redis is running.

Just an update, the new experimental configuration is as follows:

1. livekit-server is installed directly on a VM
2. redis is installed on the same VM w/o authentication
3. livekit-cli installed on same VM
4. recorder service installed in k8s, installed with helm, configured as follows to call out to redis running on the VM
```replicaCount: 1

recorder:
  log_level: info
  health_port: 8080
  redis:
    address: VM_IP:6379
  apiKey: "abc"
  apiSecret: "123"
  # default recording options
  output:
    # preset: 720p30, 720p60, 1080p30, or 1080p60
    input_width: 1920
    input_height: 1080
    # 0 means no scaling
    output_width: 0
    output_height: 0
    depth: 24
    framerate: 30
    audioBitrate: 128
    audioFrequency: 44100
    videoBitrate: 4500

# autoscaling requires resources to be defined
autoscaling:
  # set to true to enable autoscaling. when set, ignores replicaCount
  enabled: false
  minReplicas: 1
  maxReplicas: 5
  targetCPUUtilizationPercentage: 60

# if recorder should run only on specific nodes
# this can be used to isolate designated nodes
nodeSelector: {}
# <http://node.kubernetes.io/instance-type|node.kubernetes.io/instance-type>: c5.2xlarge

resources: {}```
When I try to start a recording from the VM, I get the following error:
```~/go/bin/livekit-cli start-recording --url <http://localhost:7880> --api-key abc --api-secret 123 --request room.json
twirp error internal: missing API key or secret key```
I see the following in the livekit-recorder service log:
```{"level":"info","ts":1641253798.1562278,"logger":"livekit-recorder","caller":"logger/logger.go:26","msg":"connecting to redis work queue","addr":"34.133.32.183:6379"}
{"level":"error","ts":1641253879.3856685,"logger":"livekit-recorder","caller":"logger/logger.go:34","msg":"error handling request","recordingId":"RR_deypvUeHdvJi","requestId":"RPC223tiHPYcgwa","error":"missing API key or secret key","stacktrace":"<http://github.com/livekit/protocol/logger.Errorw\n\t/root/go/pkg/mod/github.com/livekit/protocol@v0.11.1-0.20211215011801-f77609470e70/logger/logger.go:34\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).handleResponse\n\t/workspace/pkg/service/handler.go:117\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).handleRequest\n\t/workspace/pkg/service/handler.go:111\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).handleRecording\n\t/workspace/pkg/service/handler.go:59\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).Run\n\t/workspace/pkg/service/service.go:86\nmain.runService\n\t/workspace/cmd/server/service.go:50\nmain.run\n\t/workspace/cmd/server/main.go:58\ngithub.com/urfave/cli/v2.(*App).RunContext\n\t/root/go/pkg/mod/github.com/urfave/cli/v2@v2.3.0/app.go:322\ngithub.com/urfave/cli/v2.(*App).Run\n\t/root/go/pkg/mod/github.com/urfave/cli/v2@v2.3.0/app.go:224\nmain.main\n\t/workspace/cmd/server/main.go:51\nruntime.main\n\t/usr/lib/go-1.16/src/runtime/proc.go:225|github.com/livekit/protocol/logger.Errorw\n\t/root/go/pkg/mod/github.com/livekit/protocol@v0.11.1-0.20211215011801-f77609470e70/logger/logger.go:34\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).handleResponse\n\t/workspace/pkg/service/handler.go:117\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).handleRequest\n\t/workspace/pkg/service/handler.go:111\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).handleRecording\n\t/workspace/pkg/service/handler.go:59\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).Run\n\t/workspace/pkg/service/service.go:86\nmain.runService\n\t/workspace/cmd/server/service.go:50\nmain.run\n\t/workspace/cmd/server/main.go:58\ngithub.com/urfave/cli/v2.(*App).RunContext\n\t/root/go/pkg/mod/github.com/urfave/cli/v2@v2.3.0/app.go:322\ngithub.com/urfave/cli/v2.(*App).Run\n\t/root/go/pkg/mod/github.com/urfave/cli/v2@v2.3.0/app.go:224\nmain.main\n\t/workspace/cmd/server/main.go:51\nruntime.main\n\t/usr/lib/go-1.16/src/runtime/proc.go:225>"}
{"level":"error","ts":1641253879.386552,"logger":"livekit-recorder","caller":"logger/logger.go:34","msg":"recording failed","recordingID":"RR_deypvUeHdvJi","error":"missing API key or secret key","stacktrace":"<http://github.com/livekit/protocol/logger.Errorw\n\t/root/go/pkg/mod/github.com/livekit/protocol@v0.11.1-0.20211215011801-f77609470e70/logger/logger.go:34\ngithub.com/livekit/livekit-recorder/pkg/service.LogResult\n\t/workspace/pkg/service/handler.go:132\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).handleRecording\n\t/workspace/pkg/service/handler.go:42\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).Run\n\t/workspace/pkg/service/service.go:86\nmain.runService\n\t/workspace/cmd/server/service.go:50\nmain.run\n\t/workspace/cmd/server/main.go:58\ngithub.com/urfave/cli/v2.(*App).RunContext\n\t/root/go/pkg/mod/github.com/urfave/cli/v2@v2.3.0/app.go:322\ngithub.com/urfave/cli/v2.(*App).Run\n\t/root/go/pkg/mod/github.com/urfave/cli/v2@v2.3.0/app.go:224\nmain.main\n\t/workspace/cmd/server/main.go:51\nruntime.main\n\t/usr/lib/go-1.16/src/runtime/proc.go:225|github.com/livekit/protocol/logger.Errorw\n\t/root/go/pkg/mod/github.com/livekit/protocol@v0.11.1-0.20211215011801-f77609470e70/logger/logger.go:34\ngithub.com/livekit/livekit-recorder/pkg/service.LogResult\n\t/workspace/pkg/service/handler.go:132\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).handleRecording\n\t/workspace/pkg/service/handler.go:42\ngithub.com/livekit/livekit-recorder/pkg/service.(*Service).Run\n\t/workspace/pkg/service/service.go:86\nmain.runService\n\t/workspace/cmd/server/service.go:50\nmain.run\n\t/workspace/cmd/server/main.go:58\ngithub.com/urfave/cli/v2.(*App).RunContext\n\t/root/go/pkg/mod/github.com/urfave/cli/v2@v2.3.0/app.go:322\ngithub.com/urfave/cli/v2.(*App).Run\n\t/root/go/pkg/mod/github.com/urfave/cli/v2@v2.3.0/app.go:224\nmain.main\n\t/workspace/cmd/server/main.go:51\nruntime.main\n\t/usr/lib/go-1.16/src/runtime/proc.go:225>"}```
Additionally, I started livekit server as follows:
```export LIVEKIT_KEYS="abc: 123"
./bin/livekit-server --dev --redis-host=127.0.0.1:6379```
I will continue trying to figure it out but the secrets/keys match between the recorder config file, the livekit-cli request, and the env vars when livekit-server was started

that error is odd.. I would suggest trying something that doesn't get parsed as an int as your api secret, just to eliminate that as a possibility

I just put those as placeholders.  I am using keys generated by livekit

oh I see it.. the helm values config had keys like `apiKey` and `apiSecret`, when they should be `api_key` and `api_secret`

this is an issue with our helm chart... unfortunately the example file had become outdated..

Updated <https://github.com/livekit/livekit-helm/blob/master/recorder-sample.yaml|config example>

huzzah, thanks <@U01KZ61GV5Z>.  I am now getting `Recording started. Recording ID: RR_Cfoaw5drE5a4` so that gives me something to work with.  I'll try changing things around a piece at a time until I discover where the original issue was.

<@U01KZ61GV5Z> wanted to follow up on this as I am still encountering some issues.  I can start the recording OK, as mentioned above, but I can't seem to stop it.  Do you see anything obviously wrong here?
```mike@livekit-server-2:~/livekit-server$  ~/go/bin/livekit-cli start-recording --url <http://localhost:7880> --api-key API... --api-secret B5X... --request room.json
Recording started. Recording ID: RR_cQNycyP26DSF
mike@livekit-server-2:~/livekit-server$  ~/go/bin/livekit-cli end-recording --url <http://localhost:7880> --api-key API... --api-secret B5X... --id RR_cQNycyP26DSF
twirp error internal: request timeout```
Doesn't matter how many times I try to end the recording, the request always times out.  I know the server is still responsive because I can attempt to start another recording and get the message `twirp error internal: no recorders available` as expected.

my recorder config looks like this, BTW:
```replicaCount: 1

recorder:
  log_level: info
  health_port: 8080
  redis:
    address: my.redis:6379
  api_key: "API..."
  api_secret: "B5X..."
  ws_url: "<wss://my.server>"

autoscaling:
  enabled: false
  minReplicas: 1
  maxReplicas: 5
  targetCPUUtilizationPercentage: 60

nodeSelector: {}

resources: {}```

hey Mike, we'll look into this and will get back to you. I'm not sure why it would be frozen here.

<@U01KZ61GV5Z> if it would be helpful, I can also examine the contents of my redis instance.  I am trying to make my way through the code, although I am not primarily a go coder.  Are the expected contents of redis documented anywhere?

we are using redis primarily as a message bus to process RPC. I think the issue is related to something unrelated to redis FWIW.

This is even after I have triggered a recording.

there shouldn’t be anything stored in redis related to recordings, it’s only used as a pubsub

OK.  How is the state of the recording managed?

```127.0.0.1:6379&gt; pubsub channels
1) "RESERVE_RECORDER"
2) "signal_channel:ND_QRZVTfNa"
3) "rtc_channel:ND_QRZVTfNa"
4) "RECORDING_RESULT"```
These are the channels I see

If there's anything else I can check on my side, happy to do it

That is expected. While a recording is active, for example with recording ID `RR_Cfoaw5drE5a4`, the recorder should be listening to `RECORDING_REQUEST_RR_Cfoaw5drE5a4`.
The livekit server sends a request on that channel and listens for responses on the `RECORDING_RESPONSE_RR_Cfoaw5drE5a4` channel

does the recording end despite the timeout? Or does the recording continue

<@U01UB0LHTTQ> To be honest I am not 100% sure the recording ever starts.  I have it configured to save locally but I see no trace of the file on the recorder pod and I don't see anything in its logs either.

However, until I kill the pod I can't start another recording session.  I get the message about no recorders available.

easiest way to know is by checking the CPU usage, it’ll be close to 0 when not recording and around 2.5 while recording

```top - 00:34:20 up 20:07,  0 users,  load average: 0.33, 0.38, 0.42
Tasks:  19 total,   1 running,  17 sleeping,   0 stopped,   1 zombie
%Cpu(s): 13.5 us,  9.8 sy,  0.7 ni, 75.7 id,  0.0 wa,  0.0 hi,  0.3 si,  0.0 st
MiB Mem :   3937.1 total,    135.2 free,   2917.5 used,    884.5 buff/cache
MiB Swap:      0.0 total,      0.0 free,      0.0 used.   1074.9 avail Mem 

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                                                                                                                                            
    104 root      20   0   24.3g  47508  29756 S   2.0   1.2   3:12.69 chrome                                                                                                                                                                                                             
     14 root      20   0 1429344  18960   7564 S   0.3   0.5   0:05.57 livekit-recorde                                                                                                                                                                                                    
      1 root      20   0    4204   1260    948 S   0.0   0.0   0:00.02 bash                                                                                                                                                                                                               
     10 pulse     20   0  232724   3612   2328 S   0.0   0.1   0:00.58 pulseaudio                                                                                                                                                                                                         
     22 root      20   0  763320  44096  24020 S   0.0   1.1   0:00.09 Xvfb                                                                                                                                                                                                               
     23 root      20   0   16.6g  79632  43648 S   0.0   2.0   0:03.27 chrome                                                                                                                                                                                                             
     28 root      20   0    2672     92      0 S   0.0   0.0   0:00.00 cat                                                                                                                                                                                                                
     29 root      20   0       0      0      0 Z   0.0   0.0   0:00.00 cat                                                                                                                                                                                                                
     41 root      20   0   16.0g   1244    996 S   0.0   0.0   0:00.00 chrome_crashpad                                                                                                                                                                                                    
     43 root      20   0   16.0g    168      0 S   0.0   0.0   0:00.00 chrome_crashpad                                                                                                                                                                                                    
     48 root      20   0   16.2g  24228  13744 S   0.0   0.6   0:00.03 chrome                                                                                                                                                                                                             
     49 root      20   0   16.2g  24928  14448 S   0.0   0.6   0:00.05 chrome                                                                                                                                                                                                             
     50 root      20   0   16.0g   2848   1764 S   0.0   0.1   0:00.00 nacl_helper                                                                                                                                                                                                        
     73 root      20   0   16.4g  63364  38488 S   0.0   1.6   0:00.52 chrome                                                                                                                                                                                                             
     78 root      20   0   16.3g  29484  18668 S   0.0   0.7   0:00.31 chrome                                                                                                                                                                                                             
    103 root      20   0   24.3g  32516  19248 S   0.0   0.8   0:00.10 chrome                                                                                                                                                                                                             
    175 root      20   0   16.4g  30216  19316 S   0.0   0.7   0:00.82 chrome                                                                                                                                                                                                             
    842 root      20   0    2624    704    608 S   0.0   0.0   0:00.01 sh                                                                                                                                                                                                                 
    848 root      20   0    7084   3192   2660 R   0.0   0.1   0:00.01 top ```

bunch of chrome processes but hardly any CPU in use

are you starting the recording before or after joining the room?

ok, I’m guessing there’s an issue with this
```	// wait for START_RECORDING console log if using template
	switch r.req.Input.(type) {
	case *livekit.StartRecordingRequest_Template:
		r.display.WaitForRoom()
	}```

where the recorder is reserved, but doesn’t think it should start for some reason

can you try joining the room first and then starting the recording?

if that works, then I’m sure the issue is with `WaitForRoom`

I still can't stop it.  Get the same timeout message.

actually, the CPU went down on the recorder after I tried to stop.  I still don't see the file though.

I also get the no recorders available message

Nothing in the recorder logs except the usual:
```{"level":"info","ts":1641429471.7434206,"logger":"livekit-recorder","caller":"logger/logger.go:26","msg":"connecting to redis work queue","addr":"my.server:6379"}
0:00:06.723836064   184 0x55a327f25790 ERROR            msdkcontext gstmsdkcontext.c:140:gst_msdk_context_use_vaapi: Couldn't find a drm device node to open

(gst-plugin-scanner:184): GStreamer-WARNING **: 00:39:20.534: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstvaapi.so': libva-wayland.so.2: cannot open shared object file: No such file or directory```

But I read elsewhere that those errors are kind of expected

can you change your `log_level` to debug?

```{"level":"info","ts":1641429903.9560893,"logger":"livekit-recorder","caller":"logger/logger.go:26","msg":"connecting to redis work queue","addr":"34.133.32.183:6379"}
{"level":"debug","ts":1641429903.9596014,"logger":"livekit-recorder","caller":"logger/logger.go:22","msg":"starting service"}
{"level":"debug","ts":1641429903.9642107,"logger":"livekit-recorder","caller":"logger/logger.go:22","msg":"recorder waiting"}
{"level":"debug","ts":1641429924.429079,"logger":"livekit-recorder","caller":"logger/logger.go:22","msg":"request received"}
{"level":"debug","ts":1641429924.4555187,"logger":"livekit-recorder","caller":"logger/logger.go:22","msg":"request claimed","ID":"RR_6saVohXHfnyr"}
{"level":"debug","ts":1641429924.456883,"logger":"livekit-recorder","caller":"logger/logger.go:22","msg":"request handled","recordingId":"RR_6saVohXHfnyr","requestId":""}
{"level":"debug","ts":1641429924.4581244,"logger":"livekit-recorder","caller":"logger/logger.go:22","msg":"waiting for requests","recordingId":"RR_6saVohXHfnyr"}
{"level":"debug","ts":1641429924.4694812,"logger":"livekit-recorder","caller":"logger/logger.go:22","msg":"handling request","recordingId":"RR_6saVohXHfnyr","requestId":"RPCsRnfxoQoKgAg"}
{"level":"debug","ts":1641429924.471241,"logger":"livekit-recorder","caller":"logger/logger.go:22","msg":"request validated","url":"<https://recorder.livekit.io/#/speaker-dark?url=wss%3A%2F%2Fmy.server&amp;token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NDE1MTYzMjQsImlzcyI6IkFQSXNFQTdrVUFqaDhNYyIsImp0aSI6IlJSX3o3VmFvcU5KcUtpQSIsIm5iZiI6MTY0MTQyOTkyNCwic3ViIjoiUlJfejdWYW9xTkpxS2lBIiwidmlkZW8iOnsiY2FuUHVibGlzaCI6ZmFsc2UsImNhblB1Ymxpc2hEYXRhIjpmYWxzZSwiY2FuU3Vic2NyaWJlIjp0cnVlLCJoaWRkZW4iOnRydWUsInJlY29yZGVyIjp0cnVlLCJyb29tIjoibWlrZS1yb29tIiwicm9vbUpvaW4iOnRydWV9fQ.xfgSUFcaEdbnt9Qa6M6sGJC5CSfsuBy6bVZyWf-kIi0>"}
{"level":"debug","ts":1641429924.472027,"logger":"livekit-recorder","caller":"logger/logger.go:22","msg":"request handled","recordingId":"RR_6saVohXHfnyr","requestId":"RPCsRnfxoQoKgAg"}
{"level":"debug","ts":1641429924.473811,"logger":"livekit-recorder","caller":"logger/logger.go:22","msg":"launching xvfb","dims":"1920x1080x24"}
{"level":"debug","ts":1641429924.5145967,"logger":"livekit-recorder","caller":"logger/logger.go:22","msg":"launching chrome","url":"<https://recorder.livekit.io/#/speaker-dark?url=wss%3A%2F%2Fmy.server&amp;token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NDE1MTYzMjQsImlzcyI6IkFQSXNFQTdrVUFqaDhNYyIsImp0aSI6IlJSX3o3VmFvcU5KcUtpQSIsIm5iZiI6MTY0MTQyOTkyNCwic3ViIjoiUlJfejdWYW9xTkpxS2lBIiwidmlkZW8iOnsiY2FuUHVibGlzaCI6ZmFsc2UsImNhblB1Ymxpc2hEYXRhIjpmYWxzZSwiY2FuU3Vic2NyaWJlIjp0cnVlLCJoaWRkZW4iOnRydWUsInJlY29yZGVyIjp0cnVlLCJyb29tIjoibWlrZS1yb29tIiwicm9vbUpvaW4iOnRydWV9fQ.xfgSUFcaEdbnt9Qa6M6sGJC5CSfsuBy6bVZyWf-kIi0>"}
0:00:01.029830444   179 0x5593752d2b50 WARN                 default gstsf.c:97:gst_sf_create_audio_template_caps: format 0x120000: 'AVR (Audio Visual Research)' is not mapped
0:00:01.029883695   179 0x5593752d2b50 WARN                 default gstsf.c:97:gst_sf_create_audio_template_caps: format 0x180000: 'CAF (Apple Core Audio File)' is not mapped
0:00:01.029891585   179 0x5593752d2b50 WARN                 default gstsf.c:97:gst_sf_create_audio_template_caps: format 0x100000: 'HTK (HMM Tool Kit)' is not mapped
0:00:01.029905354   179 0x5593752d2b50 WARN                 default gstsf.c:97:gst_sf_create_audio_template_caps: format 0xc0000: 'MAT4 (GNU Octave 2.0 / Matlab 4.2)' is not mapped
0:00:01.029912915   179 0x5593752d2b50 WARN                 default gstsf.c:97:gst_sf_create_audio_template_caps: format 0xd0000: 'MAT5 (GNU Octave 2.1 / Matlab 5.0)' is not mapped
0:00:01.029919985   179 0x5593752d2b50 WARN                 default gstsf.c:97:gst_sf_create_audio_template_caps: format 0x210000: 'MPC (Akai MPC 2k)' is not mapped
0:00:01.029928655   179 0x5593752d2b50 WARN                 default gstsf.c:97:gst_sf_create_audio_template_caps: format 0xe0000: 'PVF (Portable Voice Format)' is not mapped
0:00:01.029936952   179 0x5593752d2b50 WARN                 default gstsf.c:97:gst_sf_create_audio_template_caps: format 0x160000: 'SD2 (Sound Designer II)' is not mapped
0:00:01.029947044   179 0x5593752d2b50 WARN                 default gstsf.c:97:gst_sf_create_audio_template_caps: format 0x190000: 'WVE (Psion Series 3)' is not mapped
0:00:06.016742904   179 0x5593752d2b50 ERROR            msdkcontext gstmsdkcontext.c:140:gst_msdk_context_use_vaapi: Couldn't find a drm device node to open
0:00:06.033441451   179 0x5593752d2b50 WARN      GST_PLUGIN_LOADING gstplugin.c:534:gst_plugin_register_func: plugin "/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstmsdk.so" failed to initialise
0:00:06.057028873   179 0x5593752d2b50 WARN      GST_PLUGIN_LOADING gstplugin.c:805:_priv_gst_plugin_load_file_for_registry: module_open failed: libva-wayland.so.2: cannot open shared object file: No such file or directory

(gst-plugin-scanner:179): GStreamer-WARNING **: 00:45:35.840: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstvaapi.so': libva-wayland.so.2: cannot open shared object file: No such file or directory
0:00:06.220617486   179 0x5593752d2b50 WARN                 nvcodec gstcudaloader.c:114:gst_cuda_load_library: Could not open library libcuda.so.1, libcuda.so.1: cannot open shared object file: No such file or directory
0:00:06.220656277   179 0x5593752d2b50 WARN                 nvcodec plugin.c:66:plugin_init: Failed to load cuda library
{"level":"debug","ts":1641429945.2097945,"logger":"livekit-recorder","caller":"logger/logger.go:22","msg":"handling request","recordingId":"RR_6saVohXHfnyr","requestId":"RPCBmbm2dWnb89T"}```

This is from starting and attempting to stop against on ongoing session

if you go to `<http://recorder-address:8080>` what do you see?

```# curl <http://localhost:8080>
stopping```

ok, I’m quite sure this is an issue with template and `WaitForRoom`. It’s in a `stopping` state but is waiting to start so everything gets cleaned up properly

I’ll need to dig into this further, but if you need a quick fix in the meantime it should work if you comment out <https://github.com/livekit/livekit-recorder/blob/main/pkg/recorder/recorder.go#L71|this line> and rebuild

Progress!  This time I was able to stop the recording with no message.  I can see the file on disk now, but it is zero-length.  Also, the recorder is not available for re-use.  It is still in "stopping" status.

is there a `sending EOS to pipeline` log message?

also, `EOS received`, and `pipeline stopped`? (I’m guessing you have the first one but not these two)

I’m noticing `url=wss%3A%2F%2Fmy.server` - is that the actual address?

(that is url encoded, <wss://my.server> would be the unencoded form)

What appears is exactly like that, except 'my.server' is my actual server address

Screen Shot 2022-01-05 at 6.30.29 PM.png

you obscured it in the logs you pasted as well?

It seems to me like the recorder is never connecting to the room

Let me get into the pod again and make sure I can connect from there

Yes, the pod can connect to the server on port 443

hi <@U02T4NWN664>, we don't support recording tracks separately, but it should be possible with our Go SDK. It is a bit involved, but should be do-able

thanks.
Are “livekit/livekit-recorder” and “livekit/server-sdk-go recording feature” different features?

livekit-recorder would output a single video with everyone composited together.

hey <@U02Q46USG4D> you could use this to create a local track for preview: <https://docs.livekit.io/client-sdk-flutter/livekit_client/LocalVideoTrack/createCameraTrack.html>

<@U02Q46USG4D> are you talking about previewing your own video or the video of the other folks in the room?

also the video preview seems a lot more performant than the agora one fwiw

The preview? That’s odd/surprising, I wonder what’s going on there.

yeah though i just realized this was iPhone XS vs iPhone X testing

not a rigorous test so i wouldn’t take it to the bank