Hi, I am using fluent-bit as a medium to transfer/...
# fluent-bit
g
Hi, I am using fluent-bit as a medium to transfer/transform to OpenTelemetry Collector. I have a service that continuously appends NDJSONs to a file, few thousands per minute. I am reading that data from fluent bit and sending to OpenTelemetry hosted in a server. I am getting the bellow issue: 'Connection error / chunk flush failure with OpenTelemetry grpc output' Can someone give a resolution?
p
Have you confirmed the connection details are correct? It seems like a general networking failure without any more detail.
What config are you using? Are you on the latest version, if not then try that? What else does the log say when you use
log_level debug
?
g
I was trying to send huge amount of data to fluent bit to transform into OTEL format and transmit to Open Telemetry. After adding below config, it was working. In service: storage.sync normal storage.checksum off storage.backlog.mem_limit 50MB In input: Buffer_Chunk_Size 64KB Buffer_Max_Size 128KB Skip_Long_Lines On storage.type filesystem In output: batch_size 200 storage.total_limit_size 100MB net.keepalive off
p
So just a buffering issue