Hi, I was trying to load test my fluent bit so I w...
# fluent-bit
l
Hi, I was trying to load test my fluent bit so I was genrating 5kb of pay load plus metadata in a span of 15 mins with total of 1 million records exactly. I observed that around 995k records reached my loki. I queried /metrics endpoint of my fluent bit and i saw only 995k records were sent with 0 dropped failed or retires. So fluent had no errors since I am sending high volume of data and there was frequent file rotation and by default in kubelet file size is set to 20 mb. When I increased the file size to 100mb I stopped seeing logs getting dropped what could be happing?
p
Was backpressure triggered? What do the logs show?
l
No, fluent bit showed no errors logs showed no errors and i had kept good 200M buffer size
i verified all fluent bit logs i see this getting spammed "message":"[2025/05/30 091322] [error] [parser] cannot parse '2025-05-30T091322Z'" other than that there are just 2 failed to flush error over the period of 24 h which is reasonable.
got the issue refresh interval was set to 60sec by default which was causing this issue
p
The tail input you mean? Yeah that is how often it looks for new files