Hi Pinot team. Support needed, tried to set up mvp...
# troubleshooting
r
Hi Pinot team. Support needed, tried to set up mvp for kafka - pinot cluster. I'm running kafka and apache pinot in podman container, all components are running in the same network. Created PRODUCER java app which runs locally and produces messages. Created CONSUMER java app which runs locally and reads messages - so far so good :) So writing to/reading from kafka container works well, Also reading messages using kafka-console-consumer script is possible. Problem occurs within container - when trying to connect pinot to kafka to read messages, I got ex: "error": "org.apache.pinot.spi.stream.TransientConsumerException: org.apache.pinot.shaded.org.apache.kafka.common.errors.TimeoutException: Failed to get offsets by times in 5001ms" command used: podman run --rm -it --network pinot-demo --name kafka -p 9092:9092 -e KAFKA_BROKER_ID=0 -e KAFKA_CFG_LISTENERS=PLAINTEXT//9092 -e KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092 -e KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181 -e ALLOW_PLAINTEXT_LISTENER=yes bitnami/kafka:3.6 podman run --rm -it --network pinot-demo --name zookeeper -p 2181:2181 -d zookeeper:3.9.2 podman run --rm -it --network pinot-demo --name pinot-controller -p 9000:9000 -e JAVA_OPTS="-Dplugins.dir=/opt/pinot/plugins -Xms1G -Xmx2G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xloggcgc-pinot-controller.log" -d apachepinot/pinot:1.2.0 StartController -zkAddress zookeeper:2181 podman run --rm -it --network pinot-demo --name pinot-broker -p 8000:8000 -e JAVA_OPTS="-Dplugins.dir=/opt/pinot/plugins -Xms2G -Xmx4G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xloggcgc-pinot-broker.log" -d apachepinot/pinot:1.2.0 StartBroker -zkAddress zookeeper:2181 podman run --rm -it --network pinot-demo --name pinot-server -p 7000:7000 -e JAVA_OPTS="-Dplugins.dir=/opt/pinot/plugins -Xms4G -Xmx8G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xloggcgc-pinot-server.log" -d apachepinot/pinot:1.2.0 StartServer -zkAddress zookeeper:2181 when running kafka container I got this - but not sure if this may cause problems: [2025-03-18 101420,276] INFO Stat of the created znode at /brokers/ids/0 is: 550,550,1742292860254,1742292860254,1,0,0,72060825902120989,202,0,550 (kafka.zk.KafkaZkClient) [2025-03-18 101420,277] INFO Registered broker 0 at path /brokers/ids/0 with addresses: PLAINTEXT://127.0.0.1:9092, czxid (broker epoch): 550 (kafka.zk.KafkaZkClient) ... [2025-03-18 101420,662] WARN [RequestSendThread controllerId=0] Controller 0's connection to broker 127.0.0.1:9092 (id: 0 rack: null) was unsuccessful (kafka.controller.RequestSendThread) java.io.IOException: Connection to 127.0.0.1:9092 (id: 0 rack: null) failed. at org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70) at kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:298) at kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:251) at org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:130) pinot-controller logs: Caused by: org.apache.pinot.shaded.org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata 2025/03/18 104733.851 ERROR [PinotTableIdealStateBuilder] [grizzly-http-server-7] Could not get PartitionGroupMetadata for topic: topic of table: topic_REALTIME org.apache.pinot.spi.stream.TransientConsumerException: org.apache.pinot.shaded.org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata