Hi Team! Can somebody suggest if they have running...
# fluent-bit
v
Hi Team! Can somebody suggest if they have running fluentbit operator(fluent-operator) + fluentbit 4.0.3? I can't get logs in stdout for the fluentbit itself despite the fact that to fluentbit container clearly there's a volume with explicit config set
Copy code
[Service]
    Daemon    false
    Flush    15
    Health_Check    true
    Http_Listen    0.0.0.0
    Http_Port    2020
    Http_Server    true
    Log_Level    trace
    Parsers_File    /fluent-bit/etc/parsers.conf
Any ideas why I can't see any logs of fluentbit itself?
Copy code
[2025/06/20 14:33:26] [ info] [fluent bit] version=4.0.3, commit=f5f5f3c17d, pid=1
[2025/06/20 14:33:26] [ info] [storage] ver=1.5.3, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2025/06/20 14:33:26] [ info] [simd    ] disabled
[2025/06/20 14:33:26] [ info] [cmetrics] version=1.0.3
[2025/06/20 14:33:26] [ info] [ctraces ] version=0.6.6
[2025/06/20 14:33:26] [ info] [input:cpu:cpu.0] initializing
[2025/06/20 14:33:26] [ info] [input:cpu:cpu.0] storage_strategy='memory' (memory only)
[2025/06/20 14:33:26] [ info] [sp] stream processor started
[2025/06/20 14:33:26] [ info] [engine] Shutdown Grace Period=5, Shutdown Input Grace Period=2
[0] cpu.local: [[1750430235.243041749, {}], {"cpu_p"=>27.500000, "user_p"=>4.250000, "system_p"=>23.250000, "cpu0.p_cpu"=>7.000000, "cpu0.p_user"=>3.000000, "cpu0.p_system"=>4.000000, "cpu1.p_cpu"=>1.000000, "cpu1.p_user"=>0.000000, "cpu1.p_system"=>1.000000, "cpu2.p_cpu"=>100.000000, "cpu2.p_user"=>13.000000, "cpu2.p_system"=>87.000000, "cpu3.p_cpu"=>2.000000, "cpu3.p_user"=>1.000000, "cpu3.p_system"=>1.000000}]
p
What do you mean? You have some logs there as all the ones marked info are internal ones. You then have a CPU input to stdout output it seems. You cannot route internal logs to an output at the moment like you can metrics but I've proposed doing that and not implemented it yet
✅ 1
If you're changing config then make sure to trigger a hot reload or restart your pods to pick it up afterwards too
v
Thanks a lot! I got the point of internal not being able to get put into STDOUT. I just run into trouble with elastic not actually having logs inside of it, and was wondering how can I get a proper debug info considering fact that image doesn't have shell.
Not first time debugging, but thanks to https://github.com/fluent/fluent-bit/pkgs/container finding -debug images of the same version I'm running helped. I am not sure if this is somewhere documented explicitly, but will check and try to contribute
p
We reference the debug image in the docs and also some reasoning around the distroless image. K8S now let's you spin up debug images as sidecars for existing pods too. https://docs.fluentbit.io/manual/installation/docker
The logs from fluent bit containers will get written to kubelet log files like any other container so you can tail them like any other one if you want. I'm not sure if the helm chart defaults to not ingesting the Fluent Bit logs as it can cause a snowball effect on failures: you log you cannot write to an output fail which then gets read in and you try to write to the output again...
Anyway the helm chart is that officially supported way to read container logs and the docs cover how to do things
v
@Pat I just realised that https://github.com/digitalocean/fluentbit-operator/tree/docode isn't as much supported as I initially thought it is. Is it considered to be deprecated by any chance, or not supported anymore? I can definitely see that in my case fluentbit entrypoint config doesn't align with my operator one:
Copy code
root           1  0.0  0.0  85760 14792 ?        Ssl  08:56   0:00 /fluent-bit/bin/fluent-bit -c /fluent-bit/etc/fluent-bit.conf
image: ghcr.io/fluent/fluent-bit:4.0.3-debug Operator assumes that config from
Copy code
fluentBitConfigName: {{ include "logging-solution.fullname" . }}-fluent-bit-config
And put it into
Copy code
/fluent-bit/config
I am not sure if I should be keeping usage of operator from my side in this case
p
I have no idea what the digital ocean one is, you'll have to ask them
There is an official fluent operator in the fluent github
There is also the official helm chart, other than that OSS cannot really support anything else
✅ 1
v
Oh, I am so sorry I am actually using
Copy code
repository       = "https://fluent.github.io/helm-charts"
chart            = "fluent-operator"
version = 3.4.0
Missed while was looking for a repository. Valid answer. Still need to check startup args, as it feels like if it uses different config from once I've put there
p
The chart has options to change the config or use your own config map: https://github.com/fluent/helm-charts/tree/main/charts/fluent-bit
❤️ 1
I also tried to walk through a few things here: https://www.fluent.do/blog/kubernetes-logging
v
Extremely valid. I would like to try to get best out of fluent-bit operator, will need to double-check on my issue. If will see it still problematic will make you aware. Thanks a lot for your input!
p
#fluent-operator is for specific operator questions
❤️ 1
v
If somebody follows this thread, I am moving it to https://fluent-all.slack.com/archives/C02642TE9HR/p1750759973630339