Michael Parkin
06/14/2023, 11:18 AMcompacted- prefix?
https://github.com/apache/flink/blob/c270a741526def82699a9accbda2e99f42b5a121/flin[…]link/connector/file/sink/compactor/operator/CompactService.java
https://github.com/apache/flink/blob/c270a741526def82699a9accbda2e99f42b5a121/flin[…]link/connector/file/sink/compactor/operator/CompactService.javasap1ens
06/14/2023, 4:09 PMMichael Parkin
06/16/2023, 3:36 PM.part files be written to a different bucket than the compacted- files?
This would help us when the destination bucket is in a different region to where the flink job runs.
Instead of there being a write parts -> read parts -> write compacted sequence across regions, which is expensive, we'd just like to write and read the parts in one bucket and write the compacted file to another bucket.