Girishankar
07/28/2025, 2:18 PM[OUTPUT] block to prevent it from exceeding the defined threshold. Although I'm using both memory and filesystem storage, it doesn't seem to be working as expected.
/host/var/log/fluent-bit-storage # du -sh *
13.8M emitter.10
4.0K emitter.11
4.0K emitter.12
12.0K emitter.13
2.2G emitter.14
247.9M emitter.15
508.5M emitter.2
25.1G emitter.3
23.9M emitter.4
3.0G emitter.5
5.7M emitter.6
212.0K emitter.7
174.9M emitter.8
11.6M emitter.9
32.0G tail.0
fluent-bit.conf:
----
[SERVICE]
Daemon Off
Flush 1
Log_Level info
Parsers_File /fluent-bit/etc/parsers.conf
Parsers_File /fluent-bit/etc/conf/custom_parsers.conf
HTTP_Server On
HTTP_Listen 0.0.0.0
HTTP_Port 2020
Health_Check On
storage.path /var/log/fluent-bit-storage
storage.metrics on
storage.max_chunks_up 512
storage.sync normal
storage.backlog.mem_limit 3G
storage.total_limit_size 10G
@INCLUDE custom-*.conf
[INPUT]
Name tail
Path /var/log/containers/*
Exclude_Path /var/log/containers/fluent-bit*.log
multiline.parser docker, cri
Tag kube.*
Mem_Buf_Limit 1GB
storage.type filesystem
DB /var/log/fluent-bit.db
[OUTPUT]
Name es
Match log.eod.hz.billing.sync.*
Host elasticsearch-elasticsearch
Port 9200
Suppress_Type_Name On
Replace_Dots On
Index fluent-bit-elastic-eod-hz-billing-sync
Retry_Limit 20
Buffer_Size false
Generate_ID on
Trace_Error on
storage.total_limit_size 10GGirishankar
07/29/2025, 3:06 PMGirishankar
07/30/2025, 2:19 PMPat
07/30/2025, 2:20 PMGirishankar
07/30/2025, 2:22 PMGirishankar
07/30/2025, 2:23 PMPat
07/30/2025, 2:24 PMPat
07/30/2025, 2:24 PMGirishankar
07/30/2025, 2:24 PMPat
07/30/2025, 2:24 PMPat
07/30/2025, 2:24 PMGirishankar
07/30/2025, 2:24 PMPat
07/30/2025, 2:24 PMPat
07/30/2025, 2:24 PMGirishankar
07/30/2025, 2:25 PMFluent Bit v2.1.9
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io
[2025/07/30 14:25:03] [error] [parser] parser named 'docker' already exists, skip.
[2025/07/30 14:25:03] [ info] [fluent bit] version=2.1.9, commit=c625ad7ea4, pid=1
[2025/07/30 14:25:04] [ info] [storage] ver=1.4.0, type=memory+filesystem, sync=normal, checksum=off, max_chunks_up=512