This message was deleted.

Might be better to do it upstream in a stream processor like Flink.

there isn't a builtin

row_index

var, but you can do

%

on some other appropriate var you have in the data (which, if it doesn't exist, yeah, you may need to add upstream via flink or similar)

or, if you can pre-computer your topn somehow and using batch ingestion, you can use lookup to replace values not in the lookup cache/map.

If you have some estimate about "rows per minute", you can allow a specific minute? so, instead of

row_index

you can use

CURRENT_TIMESTAMP

or something I'm not sure it's possible, but maybe we can use the JS and return some random number based on the minute, like

Math.random(new Date().getTime()/60000)

and if that number is greater than 0.1 (10% per minute) the value would be stored or something like that...