<#C0CTQGHKJ|fluent-bit> I am using the Fluent Bit...
# fluent-bit
s
#fluent-bit I am using the Fluent Bit Helm chart version 3.1.8 on Kubernetes. I have configured Fluent Bit to tail logs from a shared persistent volume, but it is unable to read logs from the mounted volume. Below are my configuration files and the error messages I am encountering. Could anyone help me understand why this issue is occurring? Configuration: custom_parsers.conf
Copy code
[PARSER]
    Name        airflow
    Format      regex
    Regex       ^(?<time>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}) (?<log>.*)
    Time_Key    time
    Time_Format %Y-%m-%dT%H:%M:%S
    Time_Keep   On

[PARSER]
    Name        docker_no_time
    Format      json
    Time_Keep   Off
    Time_Key    time
    Time_Format %Y-%m-%dT%H:%M:%S.%L
fluent-bit.conf
Copy code
[SERVICE]
    Flush        1
    Daemon       On
    Log_Level    info
    Config_Watch On
    HTTP_Server  On
    HTTP_Listen  0.0.0.0
    HTTP_Port    2020
    Parsers_File /opt/bitnami/fluent-bit/conf/custom_parsers.conf

[INPUT]
    Name             tail
    Tag              airflow.task
    Path             /opt/bitnami/airflow/logs/dag_id=*/run_id=*/task_id=*/**/*.log
    Parser           airflow
    Read_from_Head   On
    Refresh_Interval 5
    Skip_Long_Lines  On
    Mem_Buf_Limit    100MB
    DB               /buffers/airflow_task.db

[OUTPUT]
    # output config
Fluent Bit Logs:
Copy code
[2025/07/14 04:51:11] [ info] [fluent bit] version=4.0.4, commit=cdb6c4a6e1, pid=1
[2025/07/14 04:51:11] [ info] [storage] ver=1.5.3, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2025/07/14 04:51:11] [ info] [simd    ] disabled
[2025/07/14 04:51:11] [ info] [cmetrics] version=1.0.4
[2025/07/14 04:51:11] [ info] [ctraces ] version=0.6.6
[2025/07/14 04:51:11] [ info] [input:cpu:cpu.0] initializing
[2025/07/14 04:51:11] [ info] [input:cpu:cpu.0] storage_strategy='memory' (memory only)
[2025/07/14 04:51:11] [ info] [sp] stream processor started
[2025/07/14 04:51:11] [ info] [engine] Shutdown Grace Period=5, Shutdown Input Grace Period=2
[2025/07/14 04:51:11] [ info] [output:stdout:stdout.0] worker #0 started
[0] cpu.local: [[1752468671.539507144, {}], {"cpu_p"=>2.571429, ... }]
p
If it can't read logs it's probably a permissions problem. You can check what user you are running as, what the PV is mounted as, etc.
It also looks like you are using the default config at runtime so my guess is your configuration is not actually being used.
By default, Fluent Bit will do a CPU -> stdout pipeline which is what your debug shows.
s
Yes it was picking the default configuration
I updated the path to pick the custom config
and it worked