Hi everyone, does anyone know the phases of autocompaction in druid?. I am running autocompaction through a script that triggers the script and polls the api to check if the bytescompacted is 0, once its 0 the script disables the autocompaction. now the issue is that when I am trying to compact the table it is running for days, the parameters from the UI are fluctuation, at one point they are 99 and then the drop to 95 and so on, also I am able to see some idle time where no compaction tasks are created. so I am wondering what is actually happening when it is triggered.

my understanding -- • Auto-compaction driven compaction tasks operate over only one time interval per job • Once the compaction job starts, and a minute or so the Coordinator will detect the time intervals being compacted and will update the percentages on the Datasources UI based on the job that is running. • If the job completes, the percentages are stay. if the job fails, the percentages will revert back to before the job started. • Separately new data can be coming in and will "dirty" the time intervals ... again Coordinately will see this and adjust those percentages accordingly. Do you have a long running compaction job that eventually fails?

None of the jobs are failing, I've set the autocompaction to run for a table and the status shows running while only single digit mbs would be remaining. do you know what happening in this case ? it remains like this for a while and then the percentage goes down, which I figured is when new data come in. The problem is that the autocompaction doesnt seem to be getting completed.

Assuming those percentages are calculated correctly, that could indicate that more than 99.99% of the data size is compacted, but there are a lot of tiny segments uncompacted somewhere. What is the overall size and segment count of this datasource. Can you post your Segment Timeline chart showing both GB size and Segment Count version of the chart? === In order to confirm that a compaction job successfully generated new segments, check out the version timestamp of the segments in the time interval just compacted. it should be a new timestamp roughly matching the Created Time of the compaction job.