Badhusha Muhammed
04/15/2026, 9:14 AMstreamConfigMaps Pinot table is only consuming from the first config in the list
"streamConfigMaps" : [
{ "stream.kafka.topic.name": "topic_1", ... },
{ "stream.kafka.topic.name": "topic_2", ... }
]
This result in consuming only from topic_1
When looking at log , i can see that, the topic_partition that is trying to consume is going with Segment Partition ID (In this case the second topic segment is created with largest partition ID - 10000, https://github.com/apache/pinot/issues/17260)
Log for correctly consuming topic
2026/04/15 02:19:58.163 INFO [KafkaConsumer] [multi_topic_test_1__0__0__20260414T2017Z] [Consumer clientId=multi_topic_test_1_REALTIME-mulit_topic_test_2-0, groupId=null] Seeking to offset 6 for partition mulit_topic_test_2-0
Seeking to offset 6 for partition mulit_topic_test_2-0 Correct
Log for not consuming topic
2026/04/15 02:19:56.000 INFO [KafkaConsumer] [multi_topic_test_1__10000__0__20260414T2017Z] [Consumer clientId=multi_topic_test_1_REALTIME-mulit_topic_test_1-0, groupId=null] Seeking to offset 0 for partition mulit_topic_test_1-10000
Seeking to offset 0 for partition mulit_topic_test_1-10000 ????????
Is this is a bug ..? Partition part of topic_partition is picking from Segment Partition ID ...?Vipin Rohilla
04/16/2026, 5:10 AMBadhusha Muhammed
04/16/2026, 7:40 AMWhen multiple stream support was added in #13790, it assumes the partition id for a given stream can never go overThey added a safe partition ID for the second stream ( Some high value , In this case 10000 ), So that it won't cause overlapping , But if in case the first stream contains 10000 partitions , Then it cause overlapping, That is what they addressed here i guess , @Xiang Fu, @Mayank. This assumption cannot be held for certain custom stream plugins.10000
Badhusha Muhammed
04/21/2026, 2:44 PMBadhusha Muhammed
04/21/2026, 2:45 PMMayank