I'm running starrocks routine loads for kafka and ...
# questions-and-troubleshooting
t
I'm running starrocks routine loads for kafka and I can't seem to get the routine load to actually start from the beginning of the topic. I've tried stopping and recreating the jobs, removing the consumer group manually, etc. Nothing I do seems to get the routine load to start from the beginning.
m
Have you tried setting the Kafka property to: “property.kafka_default_offsets” = “OFFSET_BEGINNING”
t
That doesn't seem to matter for some reason. It always starts at latest. However I just paused the routine load, altered it to manually set the offsets for each partition to OFFSET_BEGINNING and that worked. I'm not sure if the kafka_fault_offsets property is just not working, or if in this case for some reason starrocks is caching the progress of a previously stopped routine load. I'm not really sure.
I'd guess that the property just isn't working correctly, because as far as I know we never read through to the end of this topic, because the destination table was empty up until I manually reset the offsets just now
oh boy, this actually might be my mistake I'm just now realizing that I have
property.kafka_default_offset
instead of
property.kafka_default_offsets
😅 1
I'm surprised that wasn't a syntax error, though I'm assuming that these properties might just be an arbitrary map under the hood
👍 1