Mohd Bilal
09/25/2025, 10:51 AM[2025/09/18 07:33:47] [ info] [input:tail:tail.0] initializing
[2025/09/18 07:33:47] [ info] [input:tail:tail.0] storage_strategy='memory' (memory only)
[2025/09/18 07:33:47] [error] [/src/fluent-bit/plugins/in_tail/tail_fs_inotify.c:360 errno=24] Too many open files
[2025/09/18 07:33:47] [error] failed initialize input tail.0
[2025/09/18 07:33:47] [error] [engine] input initialization failed
This gets fixed by increasing ulimit , but i dont want to do that. Is there any other way to fix this , like limiting the no of files opened by fluent-bit. I am also attaching the snippet from configmap of fluent-bit.
input.conf: |
[INPUT]
Name tail
Path /var/log/container/*.log
Parser docker
Tag container
Refresh_Interval 5
Read_from_Head True
Skip_Long_Lines On
filter.conf: |
[FILTER]
Name parser
Match container
Key_Name log
Parser docker
Reserve_Data True
[FILTER]
Name lua
Match container
script /fluent-bit/etc/parser.lua
call parse_logPat
09/25/2025, 11:14 AMPat
09/25/2025, 11:14 AMpath value to reduce itMohd Bilal
09/25/2025, 12:08 PMExclude_Path and Ignore_Older in order to restrict the number of files?
and will this log collection in some way?Pat
09/25/2025, 12:10 PMPat
09/25/2025, 12:10 PMPat
09/25/2025, 12:11 PMPat
09/25/2025, 12:12 PMread_from_head set so it will always read the entire log file every time it starts which may not be what you want, e.g. a pod is restartedNarendra Nath Sharma
09/26/2025, 9:32 AMNarendra Nath Sharma
09/26/2025, 9:34 AMfluent-bit.conf: |
[SERVICE]
Flush 1
Daemon Off
Log_Level info
Parsers_File parsers.conf
HTTP_Server On
HTTP_Listen 0.0.0.0
HTTP_Port 2020
@INCLUDE input.conf
@INCLUDE filter.conf
@INCLUDE output.conf
input.conf: |
[INPUT]
Name tail
Path /var/log/object-storage/*.log
Parser docker
Tag object-storage
Refresh_Interval 5
Read_from_Head True
Skip_Long_Lines OnNarendra Nath Sharma
09/26/2025, 10:23 AMNarendra Nath Sharma
09/26/2025, 10:24 AMAnurag Gupta
09/29/2025, 6:27 PMAnurag Gupta
09/29/2025, 6:27 PM