Hello guys, my pulsar deployment in a 3 nodes k8 cluster is able to receive up to 1000 request per seconds, but my http sink connector is unable to send out more than 8 messages per seconds, please how/where can I optimise this performance, as this is the last thing to fix before moving to production

I am not aware of any HTTP sink connectors, is this one that you wrote yourself?

I have tried both and I am getting same error

here is what my configuration look like

{
  "tenant": "public",
  "namespace": "default",
  "name": "msg-test",
  "className": "co.site.pulsar.HttpSink",
  "sourceSubscriptionPosition": "Latest",
  "inputs": [
    "fake-sms-test"
  ],
  "inputSpecs": {
    "fake-sms-test": {
      "schemaProperties": {},
      "consumerProperties": {},
      "poolMessages": false,
      "regexPattern": false
    }
  },
  "configs": {
    "url": "http://<domain>/webhook"
  },
  "parallelism": 1,
  "processingGuarantees": "ATLEAST_ONCE",
  "retainOrdering": false,
  "retainKeyOrdering": false,
  "resources": {
    "cpu": 1,
    "ram": 1073741824,
    "disk": 10737418240
  },
  "autoAck": true,
  "cleanupSubscription": true
}

Without looking at the code for

co.site.pulsar.HttpSink

, it will be hard to diagnose the issue.

I have tried with this as well it's same https://pulsar.apache.org/docs/3.1.x/io-http-sink/ I just started noticing this issue during our load

Here is a link to the source code, if you want to run it inside a Java profiler tool to find where the majority of the time is spent.

Hello @David K this is the entire scenario, your answer to this will help me to make conclusion please I have an apache pulsar deployed to a k8 machine of 3 nodes, I am using the http sink connector to move message from the pulsar topic to an external server via an ip address, when I perform this process at the rate of 300 msg per seconds for 10000 iterations to an ip from the same provider example k8 ip: 7.161.161.2:5900 and external ip: 7.161.161.06:4444 it takes less than 2 minutes to complete but when I do the same to a different external ip address example 6.87.64.76:8000 it takes up 30 minutes to complete in your best guess where could this issue be comming from, my pulsar configuration or k8 network configuration ?

I would suspect that it is a networking issue. What does the output of

traceroute 6.87.64.76

look like compared to the output for the other IPs?

ok let me check

This will display a list of all the routers or hops between your K8s cluster and the specified destination, showing the round-trip time (RTT) for each hop.