Hi, I've hit a bug in s3 output that leads to log ...
# fluent-bit
a
Hi, I've hit a bug in s3 output that leads to log loss, from the code i haven't been able to figure out why this happens. What's happening? here is my config:
Copy code
[INPUT]
   Name http
   port 8080
   buffer_max_size 150M
   buffer_chunk_size 50M
   mem_buf_limit 150M
   storage.type filesystem

[OUTPUT]
   Name s3
   Match *
   bucket ${var.s3_logs_bucket}
   region ${var.region}
   json_date_key timestamp
   json_date_format iso8601
   total_file_size 150M
   upload_timeout 300s
   upload_chunk_size 50M
   s3_key_format /$TAG/%Y/%m/%d/%H/%M:%S-$UUID.log
   retry_limit 5
   store_dir /var/fluent-bit/s3
As visible, my upload_chunk_size is 50MB, and total file size is 150MB, Somehow my fluentbit deployment created 6 parts, (while it should only atmax create 3/4 based on the total size and chunk size).. And after that CompleteMultipartUpload started failing with error:
Copy code
CompleteMultipartUpload API responded with error='EntityTooSmall', message='Your proposed upload is smaller than the minimum allowed size'
and eventually gave up. Here's a paste of the complete logs. Can someone help take a look here? there seems to be some bug in the multipart upload logic, and since multipart upload is the default for s3 output, this would be leading to log loss across users.