Hi Team, I'm on pinot 1.3 and trying out the multi...
# troubleshooting
y
Hi Team, I'm on pinot 1.3 and trying out the multi topic ingestion in a single pinot table. I've configured my table as shown below
Copy code
"streamIngestionConfig": {
        "streamConfigMaps": [
          {
            "streamType": "kafka",
            "stream.kafka.topic.name": "flattened_spans2",
            "stream.kafka.broker.list": "kafka:9092",
            "stream.kafka.consumer.type": "lowlevel",
            "stream.kafka.consumer.prop.auto.offset.reset": "smallest",
            "stream.kafka.consumer.factory.class.name": "org.apache.pinot.plugin.stream.kafka20.KafkaConsumerFactory",
            "stream.kafka.decoder.class.name": "org.apache.pinot.plugin.stream.kafka.KafkaJSONMessageDecoder",
            "realtime.segment.flush.threshold.rows": "0",
            "realtime.segment.flush.threshold.time": "30m",
            "realtime.segment.flush.threshold.segment.size": "300M"
          },
          {
            "streamType": "kafka",
            "stream.kafka.topic.name": "flattened_spans3",
            "stream.kafka.broker.list": "kafka.pinot-0-nfr-setup.svc.cluster.local:9092",
            "stream.kafka.consumer.type": "lowlevel",
            "stream.kafka.consumer.prop.auto.offset.reset": "smallest",
            "stream.kafka.consumer.factory.class.name": "org.apache.pinot.plugin.stream.kafka20.KafkaConsumerFactory",
            "stream.kafka.decoder.class.name": "org.apache.pinot.plugin.stream.kafka.KafkaJSONMessageDecoder",
            "realtime.segment.flush.threshold.rows": "0",
            "realtime.segment.flush.threshold.time": "30m",
            "realtime.segment.flush.threshold.segment.size": "300M"
          }
        ]
      }
But i am running into this issue.
Copy code
First kafka
2025/06/20 13:21:17.528 INFO [KafkaConsumer] [otel_spans__1__0__20250620T1321Z] [Consumer clientId=otel_spans_REALTIME-flattened_spans2-1, groupId=null] Seeking to offset 0 for partition flattened_spans2-1

Second kafka
025/06/20 13:22:08.659 INFO [KafkaConsumer] [otel_spans__10001__0__20250620T1321Z] [Consumer clientId=otel_spans_REALTIME-flattened_spans3-1, groupId=null] Seeking to offset 0 for partition flattened_spans3-10001
2025/06/20 13:22:08.659 INFO [KafkaConsumer] [otel_spans__10000__0__20250620T1321Z] [Consumer clientId=otel_spans_REALTIME-flattened_spans3-0, groupId=null] Seeking to offset 0 for partition flattened_spans3-10000
the flattened_spans3 has only partitions 1-3 but the pinot server is seeking out partition number 10000 for some reason. Can someone please guide me on where i'm going wrong with my config ?
Bump Can someone pls take a look if i'm making some basic config issue ? When i reverse the order of the kafka streams, only the first one is consuming.