Hi Team, does pushing key-value pair data to realtime table affect to segment creation? because whenever i push data without key-value segments are completing with 500 data points only but when i tried with key-value segments are not completing. even i've pushed 20k data but no segments completed yet.

so do you want to know why the segment hasn’t flushed?

Thanks @Mark Needham for your response, i want to know how segments are creating if we push data with the key value?

maybe I didn’t understand what you mean. Can you explain the difference between what you’re doing when it’s working how you expect and what you’re doing when it doesn’t work?

i was pushing data without key to realtime table and after the minion job its moving to offline table. with the 500 data points. 2nd scenario is pushing data with the key-value pair to realtime table and its not pushing to offline table and logs showing that segments are not completed even after pushing 20k data with the same configuration. so i want to know what is the diffrence between segment creation in both the scenario?

I don’t understand the key-value pair bit - what does that mean? Can you share a small example of the code ?

key-value means we are pushing json data with key

and you’re saying that with a key in kafka it doesn’t work

yeah if i push data without key-value pair its pushing data offline table from realtime but when i push with key-value its not pushing data to offline table because tables are not completing.

that’s weird - I dunno why that’d make any difference. Can you create a small example that reproduces the problem and stick it in a github repository?

okay sure @Mark Needham