Jiahao Wang Zhuo
05/15/2023, 4:12 PMFileSink using the parquet BulkFormat (on 1.16). I wanted to create a custom rolling policy based on the file size and processing time.
I see on FLINK-13027 that this was supported for StreamingFileSink since Flink-1.10. However, when I try to extend the CheckpointRollingPolicy , it still rolls only on checkpoint times or org.apache.flink.util.SerializedThrowable: java.lang.UnsupportedOperationException: Bulk Part Writers do not support "pause and resume" operations. (if I set shouldRollOnCheckpoint to false). Would I be correct to say that custom rolling policies are not supported for FileSink but only for StreamingFileSink instead (which is marked as deprecated from the docs)? If so, do you have any plan on supporting that? Is there any blocker/challenge on that being possible?