Hi, I'm using the latest fluentbit version and not...
# fluent-bit
c
Hi, I'm using the latest fluentbit version and not able ingest logs to the opensearch. I'm using the following config. outputs: - name: opensearch match: '*' host: localhost port: 9200 tls: true tls.verify: false index: ${INSTANCE_UUID} http_user: ${OPENSEARCH_USERNAME} http_passwd: ${OPENSEARCH_PASSWORD} suppress_type_name: true logstash_format: true Appreciate if someone could guide me here. I'm a beginner.
p
I think you'll have to provide a bit more detail on what's going wrong. Are errors reported? What debugging have you done? Is the configuration right? If you use stdout output do you see the data expect? If not then it's probably input
c
with stdout it's working fine. i see the below output for opensearch [2025/05/30 112830] [debug] [net] socket #82 could not connect to 127.0.0.1:9200 [2025/05/30 112830] [debug] [net] socket #82 could not connect to :19200 [2025/05/30 112830] [debug] [net] could not connect to localhost:9200 [2025/05/30 112830] [debug] [upstream] connection #-1 failed to localhost:9200 [2025/05/30 112830] [debug] [out flush] cb_destroy coro_id=47 [2025/05/30 112830] [debug] [task] task_id=15 reached retry-attempts limit 1/1 �[2025/05/30 112830] [error] [engine] chunk '1-1748604509.417202916.flb' cannot be retried: task_id=15, input=tail.1 > output=opensearch.0
I could confirm that i could cnnect to opensearch curl -X GET -k "https://localhost:9200/" { "name" : "00b0269a2e3a", "cluster_name" : "opensearch-cluster", "cluster_uuid" : "j1LatPxwST2ZTdDGSd8Wow", "version" : { "distribution" : "opensearch", "number" : "3.0.0", "build_type" : "tar", "build_hash" : "dc4efa821904cc2d7ea7ef61c0f577d3fc0d8be9", "build_date" : "2025-05-03T062350.311109522Z", "build_snapshot" : false, "lucene_version" : "10.1.0", "minimum_wire_compatibility_version" : "2.19.0", "minimum_index_compatibility_version" : "2.0.0" }, "tagline" : "The OpenSearch Project: https://opensearch.org/" }
with stdout, i could see the logs getting processed. �[7512] fmc.fmc300.messages: [[1742720528.000000000, {}], {"hostname"=>"vFMC-BART-SEcondary", "process"=>"sftunneld", "pid"=>"5401", "message"=>"[5670] sftunneld:stream_file [INFO] Creating task on SRC for incoming task: curr_read=0, curr_write=0, total_bytes=0, stream_id_src=0, stream_id_dest=1268, seq_id_src=0, seq_id_dest=0, state =Started, started:2025 03 23 090208 UTC, expires:2025 03 23 091208 UTC"}] [7513] fmc.fmc300.messages: [[1742720528.000000000, {}], {"hostname"=>"vFMC-BART-SEcondary", "process"=>"sftunneld", "pid"=>"5401", "message"=>"[5670] sftunneld:stream_file [INFO] Creating task on SRC for incoming task:: File copy 0 % completed , 0 bytes of file copied out of 0 ,avg copy speed: 0.0 bps"}] �[7514] fmc.fmc300.messages: [[1742720528.000000000, {}], {"hostname"=>"vFMC-BART-SEcondary", "process"=>"sftunneld", "pid"=>"5401", "message"=>"[5670] sftunneld:stream_file [INFO] Adding SRC Task on Request, key: 0:1268"}] �[7515] fmc.fmc300.messages: [[1742720528.000000000, {}], {"hostname"=>"vFMC-BART-SEcondary", "process"=>"sftunneld", "pid"=>"5401", "message"=>"[5670] sftunneld:stream_file [INFO] ADDED INIT confirmation to be SRC: curr_read=0, curr_write=0, total_bytes=0, stream_id_src=0, stream_id_dest=1268, seq_id_src=0, seq_id_dest=0, state =Started, started:2025 03 23 090208 UTC, expires:2025 03 23 091208 UTC"}] [7516] fmc.fmc300.messages: [[1742720528.000000000, {}], {"hostname"=>"vFMC-BART-SEcondary", "process"=>"sftunneld", "pid"=>"5401", "message"=>"[5670] sftunneld:stream_file [INFO] ADDED INIT confirmation to be SRC:: File copy 0 % completed , 0 bytes of file copied out of 0 ,avg copy speed: 0.0 bps"}]