This message was deleted.
# general
s
This message was deleted.
a
you should first narrow it down to how much time, an individual sub-task is taking. if that's high and volume processed is low, then I will suggest optimizing that task runtime
a
@Abhishek Agarwal how can I do that?
a
you can look at the sub tasks for your main task by filtering on the task group id
a
I have only one task per compaction interval
actually trying to edit the value for
maxNumConcurrentSubTasks
causes the task to fail without any meaningful error in the logs
g
if you're having issues with classic native batch ingest or compaction, I'd suggest upgrading and trying out the SQL-based approach. It has been totally rewritten to be more reliable
this example is basically equivalent to a compaction: it does a
REPLACE
from a datasource into itself for a specific time range
a
@Gian Merlino Thank you but upgrade is not an option right now