Aditya
08/19/2025, 5:06 PMIgnore_Older threshold.
Would appreciate any clarity on whether this is expected in 4.0.3?lecaros
08/21/2025, 2:13 PMAditya
08/22/2025, 3:01 AMpipeline:
inputs:
- name: tail
refresh_interval: 5
path: ${ENV_LOG_PATH}
path_key: path
tag: tail
ignore_older: 3h
rotate_wait: 120
read_from_head: true
storage.type: filesystem
storage.pause_on_chunks_overlimit: true
db: /mnt/logs/fluentbit/fluentbitpos.db
buffer_max_size: 20M
mem_buf_limit: 200MB
threaded: on
I’m monitoring the file using lsof -p <PID of fluentbit> and it shows:
[root@test-primary-0 svcusr]# ls -lart /mnt/data/usr/xyz/abc/db/jobs/logs/25655089.log
-rw------- 1 svcusr svcusr 2048 Aug 21 22:11 /mnt/data/usr/xyz/abc/db/jobs/logs/25655089.log
Current time on the system:
[root@test-primary-0 svcusr]# date
Fri Aug 22 02:53:56 UTC 2025
Deployed on AWS, and tailing from EBS as well as EFS..
and EFS tends to have limits with number of files that can stay open at a time.
<https://docs.aws.amazon.com/efs/latest/ug/troubleshooting-efs-fileop-errors.html#diskquotaerror>
Let me know if you need more detail.