Hi, I am seeing an exception similar to the following
external_reason : exception=org.apache.pinot.client.PinotClientException: java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: org.asynchttpclient.exception.RemotelyClosedException: Remotely closed thread_name:default-eero.application.default-dispatcher-1932846 stack_hash:04d180d0 level:ERROR eero.request-id:9a344af9-18b1-4d62-84db-832a25f46f0b agent.ephemeral_ide0e6c7d7 c174 46eb b31d a5bf228107f3agent.id409f77fc-135f-41f5-aa66-6be46fc48ef7 agent.name:olap.olap-75755c4597-d2vfl agent.type:filebeat agent.version:7.12.1 agent.hostname:olap.olap-75755c4597-d2vfl eero.environment:prod eero.serviceName:olap eero.serviceVersion:2023.08.22 eero.serviceSubsetprod.cloud eks prod.k8s.prod.cloud.internal.eerolevel value40,000 @id:F3v1JZqo6Q7XApUDeEEAAB fields.type:play @application.version:olap-2023.08.22 @application.commit8c86f0b94c90a2013ae2a033511703fe552ed6a4logger nameeero.olap.database.PinotDB stack_trace:#dcd3f50a org.asynchttpclient.exception.RemotelyClosedException: Remotely closed at org.asynchttpclient.exception.RemotelyClosedException.INSTANCE(Unknown Source) Wrapped by: #9523539c java.util.concurrent.ExecutionException: org.asynchttpclient.exception.RemotelyClosedException: Remotely closed at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2096) at org.asynchttpclient.netty.NettyResponseFuture.get(NettyResponseFuture.java:206) at org.apache.pinot.client.JsonAsyncHttpPinotClientTransport$BrokerResponseFuture.get(JsonAsyncHttpPinotClientTransport.java:221) ... 12 common frames omitted Wrapped by: #ef5cae8d java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: org.asynchttpclient.exception.RemotelyClosedException: Remotely closed at org.apache.pinot.client.JsonAsyncHttpPinotClientTransport$BrokerResponseFuture.get(JsonAsyncHttpPinotClientTransport.java:233) at org.apache.pinot.client.JsonAsyncHttpPinotClientTransport$BrokerResponseFuture.get(JsonAsyncHttpPinotClientTransport.java:181) at org.apache.pinot.client.JsonAsyncHttpPinotClientTransport.executeQuery(JsonAsyncHttpPinotClientTransport.java:123) ... 10 common frames omitted Wrapped by: #864567e3 org.apache.pinot.client.PinotClientException: java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: org.asynchttpclient.exception.RemotelyClosedException: Remotely closed at org.apache.pinot.client.JsonAsyncHttpPinotClientTransport.executeQuery(JsonAsyncHttpPinotClientTransport.java:125) at org.apache.pinot.client.Connection.execute(Connection.java:122) at org.apache.pinot.client.Connection.execute(Connection.java:93) at eero.olap.database.PinotDB.$anonfun$executeQuery$2(PinotDB.scala:70) at scala.concurrent.InstrumentedPromiseCompleter.liftedTree1$1(Future.scala:45) at scala.concurrent.InstrumentedPromiseCompleter.run(Future.scala:45) at akka.dispatch.TaskInvocation.run(AbstractDispa... k8s.clustercloud eks prod 1k8s.namespacecloud-prod-olap k8s.helmRelease:olap duration_ms:10 exceptionorg.apache.pinot.client.PinotClientException java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: org.asynchttpclient.exception.RemotelyClosedException: Remotely closed @version:1 ecs.version:1.8.0 eero.headers-to-propagate:eC1yZXF1ZXN0LWlkOiA5YTM0NGFmOS0xOGIxLTRkNjItODRkYi04MzJhMjVmNDZmMGI= '''