Getting this error when doing a simple select * on...
# troubleshooting
j
Getting this error when doing a simple select * on table in multistage engine but the same works on v1 engine
Copy code
pinot-broker-1  | 2025/03/17 11:09:43.231 ERROR [MultiStageBrokerRequestHandler] [jersey-server-managed-async-executor-1] Caught exception executing request 1557543877000000000: select * from HelloCustomer limit 10, Received error query execution result block: {250=ExecutionTimeoutError
pinot-broker-1  | ProcessingException(errorCode:250, message:ExecutionTimeoutError)
pinot-broker-1  |       at org.apache.pinot.common.exception.QueryException.<clinit>(QueryException.java:113)
pinot-broker-1  |       at org.apache.pinot.common.response.broker.BrokerResponseNative.<clinit>(BrokerResponseNative.java:55)
pinot-broker-1  |       at org.apache.pinot.core.query.reduce.BrokerReduceService.reduceOnDataTable(BrokerReduceService.java:74)
pinot-broker-1  |       at org.apache.pinot.broker.requesthandler.SingleConnectionBrokerRequestHandler.processBrokerRequest(SingleConnectionBrokerRequestHandler.java:150)}
pinot-broker-1  | org.apache.pinot.query.service.dispatch.QueryDispatcher.getResultTable(QueryDispatcher.java:335)
pinot-broker-1  | org.apache.pinot.query.service.dispatch.QueryDispatcher.runReducer(QueryDispatcher.java:275)
pinot-broker-1  | org.apache.pinot.query.service.dispatch.QueryDispatcher.submitAndReduce(QueryDispatcher.java:99)
pinot-broker-1  | org.apache.pinot.broker.requesthandler.MultiStageBrokerRequestHandler.handleRequest(MultiStageBrokerRequestHandler.java:191)
pinot-broker-1  |
Copy code
pinot-server-1  | Mar 17, 2025 11:09:39 AM io.grpc.internal.ServerCallImpl handleInternalError
pinot-server-1  | WARNING: Cancelling the stream because of internal error
pinot-server-1  | io.grpc.StatusRuntimeException: INTERNAL: Completed without a response
pinot-server-1  |       at io.grpc.Status.asRuntimeException(Status.java:529)
pinot-server-1  |       at io.grpc.internal.ServerCallImpl.closeInternal(ServerCallImpl.java:218)
pinot-server-1  |       at io.grpc.internal.ServerCallImpl.close(ServerCallImpl.java:208)
pinot-server-1  |       at io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.onCompleted(ServerCalls.java:395)
pinot-server-1  |       at org.apache.pinot.query.service.server.QueryServer.cancel(QueryServer.java:185)
pinot-server-1  |       at org.apache.pinot.common.proto.PinotQueryWorkerGrpc$MethodHandlers.invoke(PinotQueryWorkerGrpc.java:288)
pinot-server-1  |       at io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:182)
pinot-server-1  |       at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:351)
pinot-server-1  |       at io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:860)
pinot-server-1  |       at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
pinot-server-1  |       at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
pinot-server-1  |       at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
pinot-server-1  |       at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
pinot-server-1  |       at java.base/java.lang.Thread.run(Thread.java:829)
pinot-server-1  |
server err logs
Issue: Pinot V2 queries failed with a
gRPC UNAVAILABLE: io exception
error, while Pinot V1 worked fine. Fix: Changing the container network mode to
"host"
on each VM resolved the issue. Possible Cause: Pinot V2 uses gRPC with dynamic ports, which weren’t mapped in the previous setup. This caused connection failures between containers. Using
"host"
mode ensured all ports remained accessible.
fixed the issue please ignore