Hi All, We need to keep the latest record in a tim...
# general
a
Hi All, We need to keep the latest record in a time chunk based on the value of one of the columns ("id" value for the row). How can we achieve this in both realtime & batch ingestion? In batch ingestion, this may be possible by using the
LATEST
function for every column in the MSQ query (and group by the
floor(__time)
and "id" column) during ingestion but this is tedious to apply for each column and needs an additional buffer for String columns. Also wouldn't it be inefficient to apply this function for each column and slow down the ingestion task? In realtime ingestion, we don't have an idea of how to achieve this. Has anyone tried something similar before? Any ideas on how this can be achieved? Thanks, AR.