In Pinot 1.4.0 , It does support multi topic inges...
# troubleshooting
b
In Pinot 1.4.0 , It does support multi topic ingestion from same kafka cluster right ...? When created table config , with
streamConfigMaps
Pinot table is only consuming from the first config in the list
Copy code
"streamConfigMaps" : [
    { "stream.kafka.topic.name": "topic_1", ... },
    { "stream.kafka.topic.name": "topic_2", ... }
]
This result in consuming only from
topic_1
When looking at log , i can see that, the
topic_partition
that is trying to consume is going with
Segment Partition ID
(In this case the second topic segment is created with largest partition ID - 10000, https://github.com/apache/pinot/issues/17260) Log for correctly consuming topic
Copy code
2026/04/15 02:19:58.163 INFO [KafkaConsumer] [multi_topic_test_1__0__0__20260414T2017Z] [Consumer clientId=multi_topic_test_1_REALTIME-mulit_topic_test_2-0, groupId=null] Seeking to offset 6 for partition mulit_topic_test_2-0
Seeking to offset 6 for partition mulit_topic_test_2-0
Correct Log for not consuming topic
Copy code
2026/04/15 02:19:56.000 INFO [KafkaConsumer] [multi_topic_test_1__10000__0__20260414T2017Z] [Consumer clientId=multi_topic_test_1_REALTIME-mulit_topic_test_1-0, groupId=null] Seeking to offset 0 for partition mulit_topic_test_1-10000
Seeking to offset 0 for partition mulit_topic_test_1-10000
???????? Is this is a bug ..? Partition part of
topic_partition
is picking from
Segment Partition ID
...?
v
b
But this looks something different
When multiple stream support was added in #13790, it assumes the partition id for a given stream can never go over
10000
. This assumption cannot be held for certain custom stream plugins.
They added a safe partition ID for the second stream ( Some high value , In this case 10000 ), So that it won't cause overlapping , But if in case the first stream contains 10000 partitions , Then it cause overlapping, That is what they addressed here i guess , @Xiang Fu, @Mayank
Found this Bug -> https://github.com/apache/pinot/pull/17953 @Mayank, @Xiang Fu Can we mention this in 1.4.0 Documentation
@Jackie ^^
m
Thanks. Sure you can file a PR for doc as well
ack 1