LiveKit Community

Good morning <@U04UWFL1PBN> :sunny: 

1. Yes, and maybe there’s a better url for reference (cc <@U03RM7G8Q6A>), but here’s a couple resources:
    a. <https://github.com/livekit/components-js/blob/main/packages/react/src/prefabs/Chat.tsx|https://github.com/livekit/components-js/blob/main/packages/react/src/prefabs/Chat.tsx>
    b. <https://github.com/livekit-examples/livestream|https://github.com/livekit-examples/livestream>
2. Yes, <https://github.com/livekit/components-js#-style-them-the-way-you-want|https://github.com/livekit/components-js#-style-them-the-way-you-want>

Hey <@U04UWFL1PBN>, <@U01KZ0SCDKL> posted the best references for the chat component, I just add one more: <https://github.com/livekit/components-js/blob/main/packages/react/src/prefabs/VideoConference.tsx>
In VideoConference you can see how the chat component is used in a simple app. We use the VideoConference component without any changes for the <https://meet.livekit.io/> demo.

Thank you so much <@U01KZ0SCDKL> and <@U03RM7G8Q6A> for the answer, it helped a lot :smile:
I have a question regarding importing the styles into the react components of the livekit.
I followed your documentation which indicates to do the following import in the code:
```import '@livekit/components-styles';```
I installed the package, imported the styles but still got warnings and 1 error in my console like this:
```WARNING in ./node_modules/@livekit/components-styles/dist/general/index.css (./node_modules/css-loader/dist/cjs.js??ruleSet[1].rules[1].oneOf[5].use[1]!./node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[1].oneOf[5].use[2]!./node_modules/source-map-loader/dist/cjs.js!./node_modules/@livekit/components-styles/dist/general/index.css)
Module Warning (from ./node_modules/source-map-loader/dist/cjs.js):
Failed to parse source map from '/Users/dgois/Desktop/ic-projects/augmetec/leiaa-fe/node_modules/@livekit/components-styles/scss/_preflight.scss' file: Error: ENOENT: no such file or directory, open '/Users/dgois/Desktop/ic-projects/augmetec/leiaa-fe/node_modules/@livekit/components-styles/scss/_preflight.scss'```
Do you know what these warnings could be and how to solve them?

<@U04UWFL1PBN> glad to hear that you got unblocked :slightly_smiling_face:
I’m cc <@U02EU2ZPTQU> here. He is the import/packaging wizard.
Just to get a little more context. What kind of setup are we dealing with here - Next.js app? What bundler are you using? Do you get the same error also when running the example apps?

hi diogo!

This looks like the source map loader plugin cannot find the source .scss files and throws an error because of that. If you have control over the plugins that are being loaded you could confirm by disabling the source map loader.

I guess the problem here is that we currently do not include the source files in the npm package but “only” the dist and asset folders.

<https://github.com/livekit/components-js/pull/490|PR>

<@U03RM7G8Q6A>
Our setup is a simple react app with typescript, using the (cra command) and uses webpack as the underlying bundler. I didnt setup any new/different bundler(s).
I never had this error running the examples

<@U02EU2ZPTQU>
Does this pull request fix the problem? Or do i need to do some extra configuration on my side?
Right now i have 25 warnings like the one i provided in the message.

you could confirm if this fixes your issues by setting `GENERATE_SOURCEMAP` to `false` in your environment for testing. If this gets rid of the warnings, then the PR should fix it

hi <@U02EU2ZPTQU>, good morning!
yes, I configured that setting on my env file, started the application again and now i dont any any warnings or error.
Thank you

ok, cool, then I’m pretty sure the PR will fix the warnings, and then you can re-enable the sourcemap generation once that is released :slightly_smiling_face:

Hi <@U059A5UEMMW>, our intention with KITT was to build a fully backend agent. Meaning it publishes audio and video tracks like any other client-side human participant. 

As you might notice in the code, the part we haven’t gotten around to yet is rendering (and publishing) a video-based avatar/face on the server side. Currently, we have a React component which renders KITT’s face (the red lights). That could be ported to mobile, of course — perhaps the easiest way being (full disclosure: I haven’t written any RN code) in React Native. 

Another option is to implement this functionality this with Go, but you’ll have to encode the video frames you generate. I believe an example of this (for VP8) can be found <https://github.com/livekit/livekit-cli/blob/main/pkg/provider/vp8looper.go|here>.

A third option is to use <https://github.com/livekit/client-sdk-rust/tree/main/livekit-api/src|Rust>, which will handle the encoding/decoding for you (note: we don’t actually decode the audio for STT in Go) automatically. The link shared above is directly to the server-side crate. An example of publishing a video into a room as a server-side participant can be found <https://github.com/livekit/client-sdk-rust/tree/main/examples/wgpu_room/src|here>. While I think this might be the best long-term approach, you’ll have to port/re-implement the KITT functionality from the current repo (written in Go) to Rust.

Russ,
I wanted to personally reach out and express my sincere gratitude for your incredibly helpful response regarding the integration of KITT and the insights you provided on the KITT implementation in your *blog post*. It is evident that your blog post was written with pure passion and attention to detail, providing valuable insights that have been tremendously beneficial for my understanding of the integration process.
I must say that I am truly impressed by how well the virtual AI assistant, KITT, has been integrated as part of the backend infrastructure. The server-side implementation, particularly in the *lkgpt-service* folder, showcases the seamless integration of speech processing, ChatGPT interactions, and text-to-speech capabilities. The ability to handle these functionalities on the server side not only simplifies the client-side implementation but also allows for a more streamlined and centralized approach.
At this stage, I will be focusing on integrating the functionality without incorporating the video avatar. However, I do have a few questions regarding setting up the LiveKit cloud hosted server. In order to make all this work as a backend solution, could you please provide some guidance on the steps I should follow? Specifically, I would like to know if the *lkgpt-service* folder is the only component that needs to be uploaded to the server. It appears that there are two servers mentioned in the repository, and I want to ensure I have a clear understanding of the deployment process.
Once again, I want to express my appreciation for the exceptional work you and your team have done in making the virtual AI assistant an integral part of the backend integration. The level of detail and thought put into this implementation is truly impressive, and it has inspired me to pursue this integration with great enthusiasm.
Thank you once again for your time, expertise, and the invaluable insights you have shared. I look forward to your guidance regarding the LiveKit cloud hosted server setup.

Hey <@U051SF14H0C>, you are absolutely right. This was an unfortunate bug on our end and our tests wasn't configured correctly to catch it. We'll be cutting a release soon with the fix.

The PR that fixed it is <https://github.com/livekit/livekit/pull/1721|here>.

Hey <@U01KZ61GV5Z> thanks for acknowledging.
After getting this issue I decided to switch back to the original Redis based pub sub mode(by disabling the signal_relay). Earlier to using 1.4.1 I was using an older version of the product(1.3.5).
Where I was easily able to perform the multi Node scenario, runtime switchover of RTCNode.
However, Yesterday while trying to join the participants on two different Nodes. I observed that both the participants are able to join the room, individually they have the same room, same room Id, but can't get each other's feed. I checked the join message , there was no mention of other_participant either in both the join respectively. But while doing the list_particioant API on both of the Nodes separately I was getting the correct info(2 Participants). 
Now joining third participant, on each node separately, I was able to see a single participants on both the  Node. Didn't debug this further. Not sure if this was some issue at our end, but we had get this working earlier very well. 
I will further debug this. Can you help me understand if this could also be an issue in the 1.4.1 version

what do you mean by `Yesterday while trying to join the participants on two different Nodes` ?

there has not been any changes to non-signalrelay multi-node path

Hey <@U01KZ61GV5Z>  I meant, when I disabled psrpc, because of the issue I shared in the original message. So, I thought of working using the redis pubsub mode. For me, this was the first time on 1.4.1. Earlier was comfortably using 1.3.5. 
By 'yesterday' I was referring to the moment I switched off psrpc to work with the redis pubsub on 1.4.1. 
But this still could be an issue on my end as I mentioned that I didn't debug this to much depth.
Just wanted to check if something like this was reported.

I don't believe so.. but do keep us posted if you find any issues there. happy to help

HTTPS and public CA is only required if you intend it to be used with browsers. There is no requirement from LiveKit's side. If you are installing your own CA certs onto the devices, then it's perfectly fine to use your own certs.

On Kubernetes.. All LiveKit requires is that your clients are on the same network as your server. (in most use cases, this means having a public IP address). So as long as your instances' IP addresses are accessible to your own clients, this approach should work fine.

Routing traffic to different instances is built-in to the design of LiveKit. A client could connect to any instance's signal connection and LiveKit will route them to the correct server by relaying signal messages to the right instance. See <https://docs.livekit.io/oss/deployment/distributed/|here> for more information

Thanks for your reply. Let me try to clarify and see if I understand it correctly:
1. K8s deployment should be trivial to do. Simply put redis and multiple Livekit server, and they will automatically cooperate through redis.
2. Similarly, without k8s(if I dont need dynamic scaling or something fancy) I could just point multiple standalone VM to the same redis instance, and they will load balance automatically?

that's correct. for 2. you would need to place these instances behind a HTTP load balancer, to ensure the signal traffic could be load balanced as well.

<@U01KZ61GV5Z> another question about scaling. There are some documents showing that Livekit can run SFU across multiple node, I assume that means traffic for a single room can be distributed across multiple nodes. But I also see multiple places in the document mentioned a room has to fix within a single node. So which part did I miss?

Is the “multi-home” architecture a Livekit-cloud only feature?

Hey <@U058NGCKRC2>, yes, the Open Source SFU is single homed. It'll automatically proxy signal traffic to the right server, but media traffic has to go to the same node.

Multi-home is a feature of LiveKit Cloud.

Hello <@U058FKA6A3F>, yes, you can host LiveKit Server yourself (<https://docs.livekit.io/oss/deployment/>).
The client libraries do not distinguish between LiveKit Cloud and a self-hosted LiveKit server. So they work with both LiveKit Cloud and your self-hosted LiveKit Server.

Hey Jonas,

the usual disclaimers that I'm not a lawyer and this is not legal advise.

apache 2.0 typically does not impose requirements for users to disclose/attribute to it. <https://snyk.io/learn/apache-license/|here's> a good explainer on it.

since you are using Cloud, only the SDKs are applicable.

Hi <@U058LSHR5UP>, could you provide some additional information? Which backend (self-hosted or LiveKit Cloud) and which client SDK are you using? Often we can support you even better and faster if you provide us some code samples from your implementation.

Hi <@U03RM7G8Q6A>,
1. i have attached the audio, video tracks(in DOM) both remote and local, along with data streaming simultaneously, its working perfectly in local machine. but when we test in remote server, either audio or video track gets missed.
2. when i try to connect to livekit room with same id, livekit get disconnected and gets queued for reconnection.
3. backend is self-hosted
4. Livekit client version is "1.6.4"
Thanks

what do you mean by this?
&gt; when i try to connect to livekit room with same id, livekit get disconnected and gets queued for reconnection.
are you using LK Components? it sounds like you are, but it's unclear.

Can you reproduce the issue with <http://meet.livekit.io|meet.livekit.io> ?

<@U01KZ61GV5Z> Hi David, i am trying to build a whiteboard with conferencing application , using self hosted  SDK.1.6.4  I have 2 different issues.1. At any point of time, i am able to use only any two of ( Audio, Video or  whiteboard). Live kit blocks the missing other.2.unstable connection/disconnection when joining a room (local &amp; server). Please  help

hi all, i am looking for a way to integrate livekit with unreal engine 5.1.  has anyone done this or have some ideas?

Reposting here in case it was missed on the help desk or is a more relevant place to post. Concerns the different behaviours of Firefox across different platforms
<https://livekit-users.slack.com/archives/C01KVTJH6BX/p1684196427072719>

which version of the server are you running?

for the user cannot connect, what device are they connecting from? and are there any firewalls in their environment.

if they do have firewalls (school, company or government typically do), you may need to set up TURN as well.

There are a couple of ways of doing this.

The simplest one is to have two separate Rooms, a public and private. a user could join both rooms. You can simultaneously publish to both rooms, and just mute one or the other depending on who should get the audio.

<@U05466JB9RD> sounds like you’re trying to build something kind of like Party Chat + Team Chat which you might find in a game like Valorant?

I can join multiple rooms at once?

I thought it was not possible if that's possible that solves the issue.

<@U01KZ0SCDKL> It's more like discord with private rooms but  you have one people that can broadcast to everyone

Yes, a participant can be in multiple rooms simultaneously

Thanks!

do you know if I can get all published tracks on connect ?

You can disable autoSubscribe when your participant connects to a room, here’s some old code (read: please only use as an example) where auto subscriptions are turned off:

<https://github.com/livekit/hq/blob/b0852f2165188134b4b610fff3ac6f14490b5a4c/client/src/web/LiveKit.ts#L60|https://github.com/livekit/hq/blob/b0852f2165188134b4b610fff3ac6f14490b5a4c/client/src/web/LiveKit.ts#L60>

yeah you can get all published tracks on the Room object `room.participants` includes all participants in the room, and each `Participant` obj has a `tracks` field.

We’ll double check that and update the docs!

updated! thank you for bringing this to our attention.

Hey Sina, it's quite complicated to set something like that up and it's not something we support.

The main issue is the server will need to know what it's public IP/port combination is so it could advertise it to clients that are trying to connect.

Hey david, i hope you are doing ok. I am aware that this is complicated but i'm curious about the ways to overcome this issue and willing to implement it if necessary. So let's say i want to host 10 livekit nodes i need 10 public IPs? I mean i can get away with less if i configure different port ranges on different servers having the same public ip but that seems a bit odd. I think there has to be a better solution but i dont even know where to look.

LiveKit does not support this configuration currently. So I don't recommend going down this path.

<@U0558RZ2L20> also said (pls thread related messages):
&gt; Using LiveKit Cloud
cc <@U01KZ61GV5Z> 

Hey Ashutosh, we have an issue <https://github.com/livekit/livekit/issues/1460|here> tracking this feature request.

I think it makes sense, but it'd take a bit of time before we can add this feature

thanks for the reply, will track that feature request for the updates.

Hey Lyle, both paths seem perfectly viable to me. It comes down to a matter of preference.

If your customizations can be done with css alone, you could also have a third option: using built-in components/style but overriding any customizations.

there are also components that do not come with default styling, such as `VideoTrack` .

perhaps, you could try to start with the existing components and see how far that takes you? it'd be easy to swap them out for something more customized as you need.

Great, thanks a lot! are there any examples or docs that show the breakdown of components? Trying to see how i can lay out and wrap the smaller components to get the desired layout. Looks like i can build a storybook using the `monorepo`?

we are still working on improving the docs here. you can def check out the included storybook in the repo. or browsing the source in this <https://github.com/livekit/components-js/tree/main/packages/react/src|directory>.

following up here <@U01KZ61GV5Z> -- still struggling with how to actually override styles on the pre-built components. I'm importing `'@livekit/components-styles'` and then my custom stylesheet afterwords. When I add the same classnames as whats contained in `&lt;LiveKitRoom /&gt;` , the livekit room doesn't update, and the LiveKitRoom component is so heavily abstracted that I'm not sure how to add new styles to it. Also looked into option two of re-building the components with my own structure, but honestly neither of these workflows seem very well documented for a more novice dev like me!

Making some progress! Looks like part of the issue was my novice understanding of NextJs. Created a global stylesheet rather than a module and am successfully modifying individual classes on the livekit components there

Hi there, currently there are no plans to share the same peerconnection.

it's really complex to attempt to coordinate timing of things (especially that with autosubscribe, server may initiate an offer whenever another user publishes to the room).

here we are settling for the simpler solution.

Thanks for the reply.  Hopefully it's not too rude to ask another question, I'm just trying to understand how this all works.  Whenever another user publishes to the room would I not already have a peerconnection up (either to send my media or to receive from existing publishers)?  So, is this for the case where no one is publishing anything (not me nor anyone else) and thus I am "joined" in some vague way to the room but with zero peerconnections up?

when livekit considers you to be joined, there will always be a successful peerconnection set up. take a look at <https://docs.livekit.io/internals/client-protocol/|protocol docs> for details

Right. I guess that's why I'm confused where the possibility for Glares comes from if there is always a peerconnection offered by the SFU right away to anyone joining.

when a new track is added to the session, it needs to be renegotiated. glare happens when it's unclear which side should be the offerer

single peer connection is where we started.. but switched to one way connections as we encountered all sorts of timing issues.

which platform are you interested in specifically?

there’s no ETA yet for Swift and Android.
Flutter has experimental e2e support already and JS will follow next!

Hey <@U05AUMN8G1E>, the Rust SDK can already be used as a server SDK, it is implemented inside `livekit-api` crate (e,g: <https://github.com/livekit/client-sdk-rust/blob/main/livekit-api/src/services/room.rs|room service>).
We don’t support webhooks at the moment (It is because our protocol relies on protojson which is not compatible with <https://github.com/tokio-rs/prost|Prost>)
The WebRTC capability is optional (it is included in the `livekit` crate)

I see. Have you looked into <https://docs.rs/pbjson-build/0.5.1/pbjson_build/|pbjson_build> at all? Case from reading some issues on GitHub it doesn't seem that prost itself is getting protojson any time soon.
Good to hear that the rest of the API is already implemented.

Yes, I think it’s worth a try, Prost has a PR for it but it seems to be inactive?

<@U05AUMN8G1E>, FYI, I have a draft <https://github.com/livekit/client-sdk-rust/pull/99|PR> using pbjson_build (untested)

you can set a preferred codec (given both browser and server support it) via the TrackPublishDefaults on the RoomOptions:

``` const roomOpts: RoomOptions = {
      ...
      publishDefaults: {
        simulcast,
        videoSimulcastLayers: [VideoPresets.h90, VideoPresets.h216],
        videoCodec: preferredCodec || 'vp8',
      },
      
    };```

While working on the Kitt project, I prepared a mini example combining LiveKit with DeepGram. <https://github.com/osmanemek/livekit-deepgram>

Hi <@U05A1L8MLPR>, I can address questions you have around HIPAA. I'll send you a DM and we can discuss further.

Both are true. :blush:  

<https://github.com/livekit/livekit|LiveKit OSS> is A. <https://livekit.io/cloud|LiveKit Cloud> is B.

LiveKit server doesn't transcode. If you're referring to the egress service, there is no GPU support yet, and adding it would require some changes to the gstreamer dockerfiles and probably egress code changes as well

Doesn't simulcast require transcoding to different resolutions?

Simulcast is done by the client/publisher

For those that are using the React Native SDK, we just released <https://github.com/livekit/client-sdk-react-native/releases/tag/v1.1.0|1.1.0> that contains a bunch of improvements related to simulcast encoding. Take a look and let us know how it's working out for you!

Screen sharing and ingesting video (files and an increasing number of streaming formats) are both supported ootb.

We from <https://thing.online|https://thing.online> are in the same situation. We want to support large meetings with many participants meetings like zoom/teams etc but unfortunately HD screenshare would make us poor…
So I can’t give you a good tip expect reducing resolution. But maybe other have far better ideas as my obvious one ?!?

<@U05AZE5GF5H> could you please put this in a Google Doc and share a link with us — we’re not going to download and open a Word Doc for security reasons  :pray:

```response: {
    "egress_id": "EG_5FKerGLx6Mfd",
    "room_id": "RM_6QgLyzeYo633",
    "room_name": "test-room",
    "status": "EGRESS_STARTING",
    "started_at": "0",
    "ended_at": "0",
    "error": "",
    "room_composite": {
        "room_name": "test-room",
        "layout": "speaker",
        "audio_only": false,
        "video_only": false,
        "custom_base_url": "",
        "stream": {
            "protocol": "DEFAULT_PROTOCOL",
            "urls": [
                "<rtmp://a.rtmp.youtube.com/live2/************************>"
            ]
        },
        "file_outputs": [],
        "stream_outputs": [],
        "segment_outputs": []
    },
    "stream": {
        "info": [
            {
                "url": "<rtmp://a.rtmp.youtube.com/live2/************************>",
                "started_at": "0",
                "ended_at": "0",
                "duration": "0",
                "status": "ACTIVE",
                "error": ""
            }
        ]
    },
    "stream_results": [
        {
            "url": "<rtmp://a.rtmp.youtube.com/live2/************************>",
            "started_at": "0",
            "ended_at": "0",
            "duration": "0",
            "status": "ACTIVE",
            "error": ""
        }
    ],
    "file_results": [],
    "segment_results": []
}```

image.png

issue is I couldn’t able to see the media is getting streamed

and I don’t find any thing running on 7980

the logs state your recording ended by the template. this is likely because no one was in the room `test-room`.. did anyone actually join?

which versions of livekit and egress are you using?

can you try with livekit 1.4.3 and see if the behavior is the same?

<@U01KZ61GV5Z> please let me know if you need any information to debug the issue.

<@U05AZE5GF5H> pls don’t keep mentioning folks, we check our threads :slightly_smiling_face: — as a reminder, we provide support/help for free here and try our best to respond to everyone in our spare moments (early mornings, nights, weekends).

So I'm looking at your CURL request again.. and I think the issue is `protocol` isn't being set correctly here. Try setting it to just `RTMP` ?

what does it respond with?

Hi Andres,
seems the problem I faced is relevant with the issue mentioned above, I couldn’t convert the OPUS packet stream from KITT repo to PCM, not matter what libs used

<@U03SP6DMWUS> maybe you’re interested in it

yeah, we didn't fix it properly with that PR. It's still on our radar to fix the root cause.

I think it's likely a different issue from what you are seeing though <@U056Q5W6WGJ>. The bug Andres is referring to is in opusreader, where it's not parsing the samples from Opus correctly in some cases. If you are receiving the audio and converting to PCM, you are probably not using oggreader?

Thanks David.  I tried to use  `utils.NewOggReader`  to do converting, I got no error, but the packet was small ( 70 - 80 bytes), so I thought it’s relevant.
Since I’m new to GO lang, so maybe I’m WRONG.:sweat_smile:
```func OpusDecode(data []byte) ([]byte) {
        ogg, _, err := utils.NewOggReader(bytes.NewReader(data))
        if err != nil {
                panic(err)
        }
        out := make([]byte, 1024)
        out, err = ogg.ReadPacket()
        if err != nil {
                if err == io.EOF {
                        return nil
                }
        }
        return out
}```


Ogg is the container format.. which isn't used for data you are receiving from the room itself.. so I don't think this is right.