Sahil
08/19/2025, 12:03 PMHello everyone,
I’m running Fluent Bit as a Kubernetes Deployment against an EFS-backed PVC in thenamespace. My input config uses a deeply nested wildcard path to collect Airflow logs:airflow
[INPUT]
Name tail
Tag airflow.logs
Path /opt/bitnami/airflow/logs/dag_id=*/run_id=*/task_id=*/**/*.log
Parser airflow
DB /buffers/flb_airflow_dead_20250819.db
DB.Sync normal
DB.Compare_Filename On
Storage.Type filesystem
Refresh_Interval 10
Rotate_Wait 5
Read_from_Head On
Ignore_Older 2h
Inotify_Watcher On
Problem:
All new log files under nested folders are immediately “dismissed” in the debug logs () instead of being monitored.scan_glob add(): dismissed
Only absolute, single-level wildcard paths work correctly.
I’ve reset the DB file, toggled, and re-enabled inotify, but still cannot tail recursiveDB.Compare_Filenamepatterns on EFS.**
Questions:
Has anyone successfully used recursive wildcards () with the tail plugin on NFS/EFS mounts?**
Are there any Fluent Bit configs, patches, or community plugins to support deep directory globbing?
What workarounds have you found (e.g., external scripts, alternative plugins) to reliably discover and tail logs in nested EFS folders?
Would switching to a different storage backend or input plugin help in this scenario?
Any insights, config snippets, or pointers to existing discussions/issues would be greatly appreciated.
Thanks in advance!