LiveKit Community

have worked on intel owt recently, good product but scaling is a challenge

Hey all! Excited to see all these new faces! :hugging_face: 

the server was running on a c2-standard-16 on gke, with the following resource request,
```resources:
  limits:
    cpu: 15000m
    memory: 10240Mi
  requests:
    cpu: 14000m
    memory: 5120Mi```
and `livekit-load-tester` was run on a c5.4xlarge on aws

the full config looked like
```image:
  repository: livekit/livekit-server
  pullPolicy: Always

replicaCount: 1

livekit:
  log_level: debug
  rtc:
    port_range_start: 12000
    port_range_end: 65535
    use_external_ip: true
  redis:
    address: &lt;redis_address&gt;
  keys:
    &lt;key&gt;: &lt;value&gt;
  turn:
    enabled: false

loadBalancer:
  type: gke
  tls:
    - hosts:
        - &lt;domain&gt;
      secretName: &lt;secret&gt;

autoscaling:
  enabled: false
  minReplicas: 1
  maxReplicas: 5
  targetCPUUtilizationPercentage: 60

resources:
  limits:
    cpu: 15000m
    memory: 10240Mi
  requests:
    cpu: 14000m
    memory: 5120Mi

nodeSelector: {}```

1. you should be able to get above 100 rooms with that. In our benchmarks, we had about ~3500 subscribed tracks (num published tracks * num subscribers) before losses started happening. An approximation is roughly 3500 / (4*4*2)
2. we've talked about p2p, but it's not high on our list right now. it will involve a bunch of complexities and likely limited in functionality since LiveKit SFU adds a lot of value on top of WebRTC. FWIW, 1-1 and small groups consume very little resources on the server.

<@U027LJ573GD> with (2), is your desire for a p2p mode for cost savings or a different reason?

At scale, for 1-1 meetings, p2p can be much more efficient.

Also suggestion, Ruby Server SDK would be good,  most of MVP product ideas, tested on Rails.

that's a good suggestion <@U027LJ573GD>! We'll put it on the roadmap. let me know if you are interested in contributing.

qHD has provided a good balance of quality vs bandwidth requirements for most use cases.  By default, Zoom is sending a 360p stream and it seems that most are ok with it.

Sending a higher quality stream will reduce throughput, since the limiting factor for the SFU is total forwarded bits.

Understood. Makes sense. Thanks David. :thumbsup:

Hey all! We released <https://github.com/livekit/livekit-server/releases/tag/v0.10.6|v0.10.6> today, which exposes a `/debug/rooms` endpoint for checking out room/participant state and down track stats if you’re running with the `--dev` flag.
If you’re using it in production, we recommend turning off dev mode before upgrading

Congratulations on the :rocket: 
Did you pick Pion because you found it superior to Mediasoup or other reason? How's Pion congestion control today?

:wave: Hi everyone! Congrats on the Launch :tada:

image.png

Thanks for all of the help everyone! Wanted to share the first version of my audio chat room UI / UX, really loved using livekit, and it was really easy to customize these components

Hey all, we've just released v0.11 to GH &amp; dockerhub. You can read more about the release here: <https://github.com/livekit/livekit-server/releases/tag/v0.11.0>

Hi everyone,
First of all, I'm so happy to meet you after the pion!

The LiveKit team has built a total solution, internally, across all platforms. We’re planning to release the full source code in the next month or two.

When it's ready, I'll absolutely send you a note :slightly_smiling_face:

What about <http://livespot.co|livespot.co> is it a closed source ?? 

<http://livespot.co|livespot.co> uses LiveKit, but is a separate company+product built by <@U01KZ0UETNW>, <@U01KZ6WG1V1>, <@U01LWB8C51C> and friends

Am waiting for full client so I could assign my team to work from there 

will share with you soon :slightly_smiling_face:

:wave: Hi everyone! Congratulations on the :rocket:

Thanks Ian! I've used Wowza back in the day!

Hello, First, I would like to appreciate the effort of the team.

I am trying to use this in production on gke.
I created a cluster and used the helm with gke.yaml to install the repo on gke.

For redis, I used memorystore of google cloud.
For turn, as of now I have made "enabled: false"

Then, I have made the firewall rule to allow all incoming connections with port 7880.

However, when I try to use the web-app. I dont get any response. It always timeout.
Is there something, I'm missing out here..

Yes, I have done that. I dont know why, this is not working. I dont see any log error too..

<@U01KZ61GV5Z> I got it up and running. Thank You for helping me though.

Two things as a request that I look forward to are:
1. Support for Flutter. I think Pion Ion has flutter support and sdk for that. Would love it if Livekit has it too. 
2. Support for room events in server-sdk. I think there could be event notifications when participant leave a room or join it. 
3. Recording would also be great :stuck_out_tongue:

<@U028QT3NY3W> awesome! do you mind sharing what you had to change to get it working? this is helpful to know if there are improvements we could be making to docs.

on your requests:
1. we are workin on flutter! should be ready soon.
2. we are thinking about supporting this via webhooks. would that work for your use cases?
3. yup, we are looking at this too :slightly_smiling_face: 

Thank You <@U01KZ61GV5Z>...
For GKE, I think the cluster ip is not exposed to public. You have to make it public via command.

I have been testing my deployment now.
I have some questions:
1. Is this setup good when we have majorly 2 people in the room.
2. Can you explain more as to how kubernetes will allocate room if there are 3 nodes in it. 
3. how many rooms can be hosted, if I use the default gke.yaml as for the time being for development purposes. 
We have a doctor consultation app and we are using Agora for real time communication.
Do you think this setup will be a good replacement for it.

Also, when will be flutter launched, can I have a look at the sdk or contribute in it ?

Also, websockets will be great for room events in server-sdk

Yeah, this is great for a 2 people setup. It's very low for resource consumption and you could go a long way with a single node (probably &gt;150 concurrent rooms on a 8 core node)

when clients connects to a room for the first time, it'll select a node and store that mapping in Redis. other instances that connect, will automatically establish webrtc connection on the same node

and yeah, it should be a great alternative to your agora set up. I imagine it creates more predictability as to where your data is routed to for a health-focused app.

and yeah, good point on the websocket for server events, would seem easier to use compared to webhooks

150+ is great.
Agora has been relatively quite costly for us to be honest.
LiveKit looks a saviour to me honestly.

When you release the flutter sdk, we will try to do unit testing with this setup as we recently switched to flutter.

Also, the project looks quite good to me as an alternative to Agora in future.
Many people, love pay-per-usage kind of setup, especially in the initial launch.

If you price this service well. You can definitely give a good competition to Agora or AWS Chime.

One more thing that, you can also provide real time messaging service as agora has just started.
socketio is currently being used by a lot of people. A complete package of livekit would be a great idea in my opinion.

definitely! thanks for the suggestions. for real time messaging, may I ask what your needs are?

LiveKit does have the ability to broadcast data messages across participants: <https://docs.livekit.io/guides/room/data#data-message-from-participants>

will keep the channel posted on flutter, probably about two weeks out

Cool, I just checked this. I might have missed it.
So, basically a room can act as a place for chats.
Isnt this more like zoom wherein you have video conferencing and chat functionality at the same time.

Can this thing be given independently like a text chat room. Wherein you can transmit data over tcp.
However, when you want complete solution with Audio/Video, the default livekit is the way to go.

I am saying more in relation to app like Whatsapp. Wherein there is more text transmission

gotcha, so you don't need to use LiveKit's video/audio in order to use its data channels.. they are highly efficient and can guarantee delivery. To just use the data channels, create a room and don't publish any tracks to it. The data channels are automatically open

Great. Have you benchmarked this thing.
Suppose an app like telegram that can have upto 100,000 people in a group.
I think, then it's just that a bigger redis database would be required to store those messages until not received by the client.

it's not designed for 100k people rooms, but we do use redis/the network as the buffer.

Cool. I will try data channels for the chat functionality.
Our chats are also 1 to 1 majorly as its doctor-patient thing. I will update you on this part.
But, as I said, this messaging thing could be easily handled by livekit. Just for the storage part, any database like mongo could be used to save them.
You can ship that as a separate product in my opinion.

yah! give it a try and let us know if you have feedback!

let me know if you need anything to get started. I imagine it would be quite similar to the Node SDK. a few of us devs are hanging out in <#C025KM0S1CK|> if you'd like to discuss details in there

for sure! it'd be great to have a server SDK for it

For anyone interested, <@U01UB0LHTTQ> wrote up some interesting findings + optimizations for scaling LiveKit SFU on multicore boxes: <https://blog.livekit.io/going-beyond-a-single-core-4a464d20d17a>

has anyone tried deploying on to <http://fly.io|fly.io>?

Hey Danny, which platforms is your product delivered on? The client APIs should be pretty simple to transition to, from Twilio.

I ported over from Twilio with very little hassle

I wrote a little summary/feedback item here <https://livekit-users.slack.com/archives/C01KZ61HZRR/p1617814694026300?thread_ts=1617814694.026300&amp;cid=C01KZ61HZRR|https://livekit-users.slack.com/archives/C01KZ61HZRR/p1617814694026300?thread_ts=1617814694.026300&amp;cid=C01KZ61HZRR>

If you're in the position to rewrite your components livekit recently published a great react lib, no need to bespoke it like I had to.

<@U01TC9Z6ELD> you just gave me the idea to add some migration guides to our documentation :pray:

Note: a lot has probably changed since April, but chances are things have been made easier, not the other way around :see_no_evil:

would you be up for taking a look at drafts of those?

Definitely! But let's move this out of <@U028CTVNQ4X>'s thread lol

Thanks for the input! Sounds like it won’t be too hard

Hey Ashwin!

1. Published benchmarks are up to 1000 listeners in an audio-only room. We’re working on even greater scale, but that’s not available yet.
2. If you’re planning on having a hybrid co-working setup where some folks are together, in person, and others are virtual, you'll need echo cancellation. We don't support that (yet) natively in LiveKit. I'm not 100% sure on how to build it, but I'd imagine on the client-side, pre-encode, you'd want to run it through a process that filters out feedback.
3. We don't yet offer a managed service, so the pricing plan is whatever your cloud infrastructure costs are for hosting LiveKit yourself. 
4. If/when we offer a managed service, I'd imagine it would come with a support plan for these types of scenarios. For now, if you run a 1000-person event and there's an issue, feel free to ping us and we'll take a look. :slightly_smiling_face:
5. Unfortunately, we can't offer this. LiveKit is an open source project, but if your engineering team wants to discuss some part of the code, in particular, that can be done via GitHub.

Thank you for your reply <@U01KZ0SCDKL>! When you say greater scale, you mean benchmarks or the product itself?

You're welcome! Greater scale meaning, benchmarks. Right now, rooms are limited to 10 publishers, 1000 subs for audio and 4 pubs and 400 subs for video.

<@U028QNSBP61> how many publishers are you thinking about? if it's 1 publishers, you can do 10k on a single 16 core machine

it should scale linearly so if 4 pubs, 10k subs, then something like a 64 core machine *should* handle it.. we provide a load tester so you could validate the exact use case

If my demand is between 5k-10k viewers ,
Would it be wise to go with livekit rather then rtmp streaming ? Any idea plz

<@U01KZ61GV5Z> as you mentioned with approximately 64 core machine 4 pubs and 10k subs, its for single room, right? If yes then for multiple room how we should scale?

<@U027GNCS0CW>, you'd want to spin up multiple nodes.. livekit currently picks a node at random for each room created.. it's def. possible to improve that room selection algo there

if you are looking to get 5-10k viewers.. RTMP -&gt; HLS is your best shot today. it's not possible to handle that on a single livekit instance

Thank you <@U027CD9ET6V>! :pray::skin-tone-4: 