Hello everyone, is it normal for pinot-server to flood this log? i have noticed this after upgrading to 0.10.0

[Consumer clientId=consumer-null-808, groupId=null] Seeking to offset 59190962 for partition mytopic-0

Seems like this is coming from the kafka consumer

Do you think it is printed because something went wrong or it's completely normal to see this log flooding?

seeing this happen a lot in our cluster but i’m not sure if it’s a bad thing or not, anyone has any ideas?

@Neha Pawar ^^

yup coming from kafka consumer. it comes from seek(topic, offset) call we make to the KafkaConsumer, so we don’t really have control on it. I don’t know if there’s a way for us to edit our log4j.xml file to skip this log. As long as this log is not being followed by this message, it is harmless: https://github.com/apache/pinot/issues/8219

thank you @Neha Pawar 🙏

@Neha Pawar I’m seeing this behavior currently. My config is

"streamConfigs": {
        .
        "stream.kafka.consumer.prop.auto.offset.reset": "largest"
      },

so why would it be seeking? Shouldn’t it start consuming from the latest offset?

if all you’re seeing is

[Consumer clientId=consumer-null-808, groupId=null] Seeking to offset 59190962 for partition mytopic-0

, it’s fine. this simply gets printed everytime the kafka consumer created in pinot servers calls consumer.seek()