Hey folks 👋
Just wondering how I can get some visibility into an old stale Git Issue that has been closed before responded to?
https://github.com/fluent/fluent-bit/issues/2670
In particular, this bug has stopped our adoption of Fluent Bit in its tracks, with us re-considering our decision and will potentially to be moving back to fluentd.
p
Pat
07/29/2025, 6:23 AM
Lol it is old, legitimate ask though but I guess just not too popular so no one has contributed it, or no enterprise has pushed their vendor of choice hard enough! 🙂
I wonder if the kubelet option would help here.
Not sure how much impact the watch behaviour would have on performance.
Pat
07/29/2025, 6:26 AM
There's a general issue with short lived pods/processes too, similar things happen for example with tail and fargate sometimes - log files appear and disappear so quickly you miss them.
Pat
07/29/2025, 6:31 AM
The optional watch behaviour should be doable though (probably not on by default for large nodes with lots of pods where you're only tailing a subset) but "just" needs implementing.
An alternative I was thinking about was using a LUA filter to add the metadata that could do the K8S watching. It is possible to maintain some state but it can be complex.
What about routing data missing the info to a fluentd aggregator that handles the watching too for now? Then it'll only be triggered when required and you get the normal fluent bit benefits the rest of the time but it's not ideal.
a
Adrian Mace
07/30/2025, 12:07 AM
I think if I find myself having to maintain a fluentd agg. then it's probably better to just drop fluent-bit and maintain one thing rather than two 😕
p
Pat
07/30/2025, 7:18 AM
Yeah I can understand that
Pat
07/30/2025, 7:19 AM
Depends though if the resource cost is too high for fluentd which is why you were switching is my only thought