Hi All. Is there way we can know the output s3 sto...
# fluent-bitp
Padma
10/13/2025, 12:38 PMHi All. Is there way we can know the output s3 storage path storage.total_limit_size reaching its limit
Padma
10/13/2025, 12:40 PMUntitled
Padma
10/13/2025, 12:46 PMWe have fluent bit_chunks_limitset as 512 and that has reached max limit and all chunks been seen to filesystem in fluentbit_storage_fs_chunks_down
Padma
10/13/2025, 12:46 PMBut we don't see some logs getting delivered to s3 when there is high peak of logs reached 512 limit
p
Pat
10/13/2025, 12:47 PMWhat do the metrics say?
Pat
10/13/2025, 12:47 PMp
Padma
10/13/2025, 12:57 PMThat means the chunks are dropped even though the storage.type is filesystem
p
Pat
10/13/2025, 1:03 PMyou can configure whether it pauses (i.e. stops ingesting) when it hits the limit: https://docs.fluentbit.io/manual/administration/buffering-and-storage#filesystem-buffering
p
Padma
10/13/2025, 1:06 PMBut that will lead to log loss as per documentation For some inputs, such as TCP and tail, pausing the input will almost certainly lead to log loss.
p
Pat
10/13/2025, 1:07 PMsure but ultimately you need to choose either enough storage to buffer or lose data - how would you solve the problem when you run out of storage?
Pat
10/13/2025, 1:08 PMif you have an output down for longer than you can buffer for you need to choose whether to pause or drop
Pat
10/13/2025, 1:08 PMI'm not sure that's the question you're asking though, are you saying chunks are free but you're missing data?
Pat
10/13/2025, 1:09 PMI just saw a question asking how to monitor it so pinged that info over but I think your original question then has another message in this thread
p
Padma
10/13/2025, 1:10 PMfluentbit_storage_fs_chunks_up reached the limit of 512 and sent to down (fluentbit_storage_fs_chunks_down) But not sure of the state whether the storage.total_limit_size 20G is fully reached..
Padma
10/13/2025, 1:10 PMWill that increase really help
Padma
10/13/2025, 3:44 PM@eduardo can you please help here