Grzegorz Liter
05/23/2024, 4:15 PMFileCompactStrategy.Builder.newBuilder()
.enableCompactionOnCheckpoint(numCheckpointsBeforeCompaction)
.build(),
new RecordWiseFileCompactor<>(new RecordWiseFileCompactorSpecificAvroReaderFactory<>(Record.class)
But what I noticed that betwen Writer and a Commiter it adds CompactorCoordinator with parallelism of only 1. Does it have to be the case that this won't scale?
.setNumCompactThreads(24) thinking that maybe it somehow affects parallelism but no luck.