https://livekit.io logo
Join Slack
Powered by
This message was deleted.
# helpdesk
s
This message was deleted.
l
For this I would use go sdk. It allows to send pre encoded data or video from files. For example I was able to easily read rtsp stream from ip camera using h264 and publish to livekit server
a
Interesting. But I would need that in Python, so I suppose there is no way?
@loud-jewelry-13301 Could you please point me to some sample code for the GO SDK to achieve this?
l
@adventurous-twilight-28575 Please refer to they github repository https://github.com/livekit/server-sdk-go . There are a good examples. For example how to read it from file and send to a room as participant. Also it can accept io.ReadCloser interface and it's up to developer how here will get raw data and will send to livekit (using this I implemented RTSP to Livekit). Also please check this code https://github.com/livekit/livekit-cli/blob/main/cmd/livekit-cli/join.go . It has a lot of good examples
a
hey @adventurous-twilight-28575, before I get into the details of your question, what are you trying to do (at a high level)? ie, why do you want to publish pre-encoded frames?
(my understanding is that the Python SDK isn’t going to support sending pre-encoded frames because it uses libwebrtc under the hood, which does not expose that api)
d
@adventurous-twilight-28575 If you are using Gstreamer to encode, take a look at livekitwebrtcsink created by the GStreamer team. This should let treat a LiveKit room as a standard gstreamer sink
a
Thank you all for your interesting answers. @dry-elephant-14928 Thanks for the pointer. Looks like it comes out of the Rust components corner. Not impossible, but hard to compile them on a Raspberry. @acoustic-engineer-41666 Matches my observations. Because you asked, I'm required to do Full HD @ 30fps or even 720p. It is pretty much impossible to capture this raw on a Raspberry PI via USB (CSI possible), not to talk about encoding it w/o hardware support for acceptable results. I'm using the caps of some cheap cams, like the CSI cam on the PI or some Chinese H.264 USB cams. This can be directly fed into GST webrtcbin or (as said) into AIORTC. Then, in another project, I'm getting I420 frames on a software interface from a drone SDK and putting this into a GST pipeline using v4l2h264enc (a hardware H.264 encoder) in order to be able to encode Full HD @ 30fps. I'm trying to prevent transcoding or even encoding w/o hardware support as much as possible. And the results are pretty stunning. Latency at around 100 ms at subscriber end even in SFU scenarios, decent bitrates and full 30fps, nearly no load to the PI. @loud-jewelry-13301 Yesterday was my first day with GO and at least I managed to enter a room and subscribe to a stream sent by a Python publisher. My preliminary idea is to wrap all the signaling and the encoded input interface into a GO module and use that from Python on the PI, since the drone SDK requires me to use Python as primary controlling language. Thanks for the pointers
...and before I forget: I have all this running in a Mediasoup/AIORTC environment. Just simulcast producing is not working. My first attempts with LiveKit did show a well working simulcast producing. Then, the entire concept, the documentation and even this forum makes me think about giving LiveKit a try.
🙌 1
d
Thanks for the thorough explanation! Definitely helps to understand your use case. Really cool use case; let us know if it ends up working out. or if you have any additional questions we could help with.
a
Thanks @dry-elephant-14928 👍
Wondering if somebody could give some explanations on this: https://github.com/livekit/livekit-cli/blob/main/README.md (Sorry, left too early): What can be said regarding the expected formats? For the gstreamer reference I would say, it could be RTP (would that work for VP8 too?) For the UnixSock I have no clue.
(OK, I was tricked several times by typeing ENTER while writing, so now I wrote this post in a separate editor) To answer my own question: For H.264 a H.264 bitstream (NAL units) is required. This is great, since it meets my requirements. For those of you who want to follow: I'm on a Mac, you will most likely have to change the H.264 encoder and the video source to something different (e.g. 
v4l2src device=/dev/video0
for 
avfvideosrc
and 
avenc_h264
for 
avenc_h264_videotoolbox
) Then there is a typo in the a.m. demo: The H.264 URL has one slash too much, livekit-cli complains. Then I had to change 
127.0.0.1
to 
localhost
. Unknown reason, but the connection was refused with 
127.0.0.1
. Maybe it was attempted via IPv6, that's why. I didn't investigate. Also somehow important the selection of capture resolution and encoding bitrate. Otherwise it was capturing in 1521 x 1521 or so over here and encoding with about 300 kBps -> terrible quality Here is the gist: - Fire up a live-kit dev server in one terminal window: 
livekit-server --dev
- Fire up the GStreamer pipeline in another terminal window: 
gst-launch-1.0 avfvideosrc device-index=0 ! video/x-raw, width=1920, height=1080, framerate=30/1 ! avenc_h264_videotoolbox bitrate=5000000 ! tcpserversink port=16400
- Fire up the publisher in a third window: 
livekit-cli join-room --url <ws://localhost:7880> --api-key devkey --api-secret secret --room abcde --identity bot --publish <h264://localhost:16400>
- Now you would need a subscriber. I was using an 
index.html
with the content below and "organized" a token by 
livekit-cli create-token --api-key devkey --api-secret secret --join --room abcde --identity Neil --valid-for 160h
. - So a fourth terminal window is now hosting the 
index.html
by using 
live-server
, installed globally with Node and NPM. 
index.html
is available at 
localhost:8080
in your browser then. Results were pretty nice, full HD in decent qualtity and latency. Chrome's 
WebRTC-Internals
confirmed to have H.264 at the configured bitrate of 5 MBit/s in full HD at 30 fps. Not bad, not bad :)
Copy code
<!DOCTYPE html>
<html lang="en">

<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Livekit Example</title>
</head>

<body>

  <div id="video-container">


  </div>

  <script src="<https://cdn.jsdelivr.net/npm/livekit-client/dist/livekit-client.umd.js>"></script>

  <script>
    const url= "<ws://localhost:7880>"
    const token = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE3MDEzMDQxMDUsImlzcyI6ImRldmtleSIsIm5hbWUiOiJOZWlsIiwibmJmIjoxNzAwNzI4MTA1LCJzdWIiOiJOZWlsIiwidmlkZW8iOnsicm9vbSI6ImFiY2RlIiwicm9vbUpvaW4iOnRydWV9fQ.gGnm0ifnrLRSzX69YFV-3acqjDuwpRYmRIUISEA3LjM"

    debugger
    // creates a new room with options
    const room = new LivekitClient.Room()

    // pre-warm connection, this can be called as early as your page is loaded
    room.prepareConnection(url, token)

    // set up event listeners
    room
      .on(LivekitClient.RoomEvent.TrackSubscribed, handleTrackSubscribed)
      .on(LivekitClient.RoomEvent.TrackUnsubscribed, handleTrackUnsubscribed)
      .on(LivekitClient.RoomEvent.ActiveSpeakersChanged, handleActiveSpeakerChange)
      .on(LivekitClient.RoomEvent.Disconnected, handleDisconnect)
      .on(LivekitClient.RoomEvent.LocalTrackUnpublished, handleLocalTrackUnpublished);

    function handleTrackSubscribed(track, publication, participant) {

      if (track.kind === "video") {
        // attach it to a new HTMLVideoElement or HTMLAudioElement
        const videoElement = document.getElementById('video-container');
        const element = track.attach();
        videoElement.appendChild(element);
      }
    }

    function handleTrackUnsubscribed(track, publication, participant) {
      // remove tracks from all attached elements
      track.detach();
    }

    function handleLocalTrackUnpublished(publication, participant) {
      // when local tracks are ended, update UI to remove them from rendering
      publication.track.detach();
    }

    function handleActiveSpeakerChange(speakers) {
      // show UI indicators when participant is speaking
    }

    function handleDisconnect() {
      console.log('disconnected from room');
    }


    async function subscribeToRoom(token, url) {

      await room.connect(url, token)
      console.log('connected to room', room.name);

    }

    subscribeToRoom(token, url);
  </script>

</body>

</html>
Trying the same with a 
vp8
source fails: `gst-launch-1.0 avfvideosrc device-index=0 ! video/x-raw, width=1280, height=720, framerate=30/1 ! videoconvert ! vp8enc deadline=1 ! tcpserversink port=16400`` Publishing like so: 
livekit-cli join-room --url <ws://localhost:7880> --api-key devkey --api-secret secret --room abcde --identity bot --publish <vp8://localhost:16400>
It starts but fires an error quickly: 
2023/11/23 10:00:50 "msg"="Could not start writing" "error"="IVF signature mismatch"
OK, I could solve the VP8 problem. I just had to be fast enough to launch 
livekit-cli
after having started the pipeline (see https://github.com/livekit/livekit-cli/issues/251)
In between I also managed to feed NAL units from a Python app via UnixSockets into livekit-cli (using the example 
livekit-cli join-room --url <ws://localhost:7880> --api-key devkey --api-secret secret --room abcde --identity bot --publish <h264://tmp/myvideo.sock>
The python app provides a UnixSocks server and starts the GStreamer pipeline then. The results are perfect, but - here we go again: No simulcast. Thinking twice I believe there must be a mistake somewhere. I can't really imagine where - with the provision of only one H.264 stream - simulcast is supposed to come from... But how can I make several encoded streams available with the resources I have? I think I need to take another look at the demo example. What exactly happens there? Basically, I can only imagine that 3 streams with different spatial layers are generated simultaneously from some endless raw loop and these are then - depending on the bandwidth - made available to the consumer...
3 Views
Open in Slack
PreviousNext
Powered by