Hello team, we are using Flink (1.19.1) with AWS K...
# troubleshooting
a
Hello team, we are using Flink (1.19.1) with AWS KDA and using Kinesis Firehose as a sink. Following is the setup:
Copy code
public class FirehoseSinkFactory {
    public static KinesisFirehoseSink<OutputRecord> createFirehoseSink(final Properties firehoseSinkConfigProperties) {
        final String DELIVERY_STREAM_NAME = "deliveryStreamName";

        return KinesisFirehoseSink.<OutputRecord>builder()
                .setFirehoseClientProperties(firehoseSinkConfigProperties)
                .setSerializationSchema(new FirehoseSerializationSchema())
                .setDeliveryStreamName(firehoseSinkConfigProperties.getProperty(DELIVERY_STREAM_NAME))
                .build();
    }
}
Could someone guide/help on how to make sure connection is not timeout while using it?
Copy code
property_group_id = "FirehoseSinkConfigProperties"
      property_map = {
        "aws.region"         = "aws-region-name"
        "deliveryStreamName" = "stream-name"
      }
j
have you got IAM permissions configured properly? please paste the error
a
Yes, application was working fine for almost 6 months. Pasting the error:
Copy code
java.util.concurrent.CompletionException: software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: connection timed out: <http://firehose.eu-west-1.amazonaws.com/|firehose.eu-west-1.amazonaws.com/>
j
have you got any app backpressure, data skew, otherwise?
that's when i've most often seen this type of error
a
Backpressure increased, but it was mostly due to connection-timeout of kinesis-firehose. Other than that, all metrics were normal. I checked internals of KinesisFirehoseSink and seems like there are no parameters to set for connection-timeout and retries