Is 'purge task' possible for realtime tables in 0....
# general
e
Is 'purge task' possible for realtime tables in 0.11.0 release? Hi all, In the 0.11.0 release notes, I saw that support segment upload for all tables, including real-time tables that are not upsert mode. The same release includes a purgeTaskGenerator implementation (https://github.com/apache/pinot/pull/8589) that facilitates the purge process. However, only works for offline tables. https://github.com/apache/pinot/blob/release-0.11.0/pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/purge/PurgeTaskGenerator.java#L62
Copy code
if (tableConfig.getTableType() == TableType.REALTIME) {
        LOGGER.warn("Skip generating task: {} for real-time table: {}", taskType, tableName);
        continue;
 }
I think, if it is possible to upload a segment to a real-time table, it is possible to download an existing segment and purge the record to create a new segment and upload it. Is the action to purge for realtime table not possible yet? Or is it just that this built-in task is implemented to only target offline tables?
f
Yes it's possible but not merged yet ! I still have a few things to fix.
e
Oh! really? Do you mean fix PurgeTaskGenerator? looking forward 👀 thank you so much
f
But I’m running out of time now to work on it 😞
e
Okay I got it. thanks for your contribution 👍👍I hope to see you soon🥺🥺🥺