<#C0CTQGHKJ|> &gt; Hello everyone, &gt; I’m runnin...
# fluent-bit
s
#C0CTQGHKJ
Hello everyone,
I’m running Fluent Bit as a Kubernetes Deployment against an EFS-backed PVC in the
airflow
namespace. My input config uses a deeply nested wildcard path to collect Airflow logs:
[INPUT]
Name tail
Tag airflow.logs
Path /opt/bitnami/airflow/logs/dag_id=*/run_id=*/task_id=*/**/*.log
Parser airflow
DB /buffers/flb_airflow_dead_20250819.db
DB.Sync normal
DB.Compare_Filename On
Storage.Type filesystem
Refresh_Interval 10
Rotate_Wait 5
Read_from_Head On
Ignore_Older 2h
Inotify_Watcher On
Problem:
All new log files under nested folders are immediately “dismissed” in the debug logs (
scan_glob add(): dismissed
) instead of being monitored.
Only absolute, single-level wildcard paths work correctly.
I’ve reset the DB file, toggled
DB.Compare_Filename
, and re-enabled inotify, but still cannot tail recursive
**
patterns on EFS.
Questions:
Has anyone successfully used recursive wildcards (
**
) with the tail plugin on NFS/EFS mounts?
Are there any Fluent Bit configs, patches, or community plugins to support deep directory globbing?
What workarounds have you found (e.g., external scripts, alternative plugins) to reliably discover and tail logs in nested EFS folders?
Would switching to a different storage backend or input plugin help in this scenario?
Any insights, config snippets, or pointers to existing discussions/issues would be greatly appreciated.
Thanks in advance!