mg
10/16/2025, 9:00 AMConsumerConfig is flagging Pinot-specific properties as unknown, likely because they are wrappers around the core Kafka properties.
Are these warnings benign and expected, or does this indicate a potential issue with our configuration style?
I'm seeking recommendations on whether we can suppress these warnings or if there's an updated configuration pattern we should use to avoid passing these metadata properties to the Kafka client.
1. Controller WARN Logs (Example)
2025/10/16 08:20:15.667 WARN [ConsumerConfig] [pool-14-thread-9] The configuration 'stream.kafka.decoder.class.name' was supplied but isn't a known config.
2025/10/16 08:20:15.667 WARN [ConsumerConfig] [pool-14-thread-9] The configuration 'streamType' was supplied but isn't a known config.
2025/10/16 08:20:15.667 WARN [ConsumerConfig] [pool-14-thread-9] The configuration 'stream.kafka.consumer.type' was supplied but isn't a known config.
2025/10/16 08:20:15.667 WARN [ConsumerConfig] [pool-14-thread-9] The configuration 'stream.kafka.broker.list' was supplied but isn't a known config.
2025/10/16 08:20:15.667 WARN [ConsumerConfig] [pool-14-thread-9] The configuration 'stream.kafka.consumer.factory.class.name' was supplied but isn't a known config.
2025/10/16 08:20:15.667 WARN [ConsumerConfig] [pool-14-thread-9] The configuration 'stream.kafka.topic.name' was supplied but isn't a known config.
2. Relevant Table Config (streamConfigs)
{
"REALTIME": {
"tableName": "XYZ",
"tableType": "REALTIME",
"segmentsConfig": {...},
"tenants": {...},
"tableIndexConfig": {
"streamConfigs": {
"streamType": "kafka",
"stream.kafka.consumer.type": "LowLevel",
"stream.kafka.topic.name": "test.airlineStats",
"stream.kafka.broker.list": "kafka-bootstrap.kafka.svc:9093",
"stream.kafka.decoder.class.name": "org.apache.pinot.plugin.inputformat.json.JSONMessageDecoder",
"stream.kafka.consumer.factory.class.name": "org.apache.pinot.plugin.stream.kafka30.KafkaConsumerFactory",
"security.protocol": "SSL",
// SSL config continues...
},
"other-configs": ...
},
"metadata": {},
"other-configs": ...
}
}
Any guidance on best practices for stream config in recent Pinot versions, or a way to silence these specific ConsumerConfig warnings, would be highly appreciated!
Thanks!Mayank