Hi we do have another question Does FluentBit tai...
# fluent-bit
h
Hi we do have another question Does FluentBit tail input support a high throughput logs generation? Indeed we do have a pod that generates ~1M logs in 2 min (1 log/client request) and we do observe missing logs collected by the FluentBit DeamonSet for container logs. I tried to run a test with https://github.com/kubernetes/kubernetes/tree/v1.6.13/test/images/logs-generator and got the issue at similar pace • 100K in 3 min with the generator --> ok • 500K in 3 min --> ko • 1M in 3 min --> ko Can FluentBit miss the inotify events?
u
I have seen log losses when the file rotates at a high log volume and fluentbit will not be able to find the file to tail anymore. You can tweak your configuration to support higher load. We easily support 25k TPS with our configuration
h
hey thanks @Ujjwal Sharma for the feedback
when you say "tweak your config" .. you talking about the input tail settings?
do you have an example of settings for such high load?
p
It'll depend on your specific set up a bit but generally the answer is ensuring Fluent Bit has enough resources to keep up with your data rates. You'll need to make sure the outputs can cope too, Fluent Bit will send to them and if they back up then it'll have to. Ultimately it's a general resource problem for a process, it needs enough to keep up with the events it's receiving.
👍 1
h
The output rate is much much lower than the Redpanda could take so I am not very worried about it.
the fluentbit pod resources consumption is much lower than the limits too
image.png