Hi, I am trying to create a `FileSink` using the p...
# troubleshooting
j
Hi, I am trying to create a
FileSink
using the parquet BulkFormat (on 1.16). I wanted to create a custom rolling policy based on the file size and processing time. I see on FLINK-13027 that this was supported for
StreamingFileSink
since Flink-1.10. However, when I try to extend the
CheckpointRollingPolicy
, it still rolls only on checkpoint times or
org.apache.flink.util.SerializedThrowable: java.lang.UnsupportedOperationException: Bulk Part Writers do not support "pause and resume" operations.
(if I set
shouldRollOnCheckpoint
to false). Would I be correct to say that custom rolling policies are not supported for
FileSink
but only for
StreamingFileSink
instead (which is marked as deprecated from the docs)? If so, do you have any plan on supporting that? Is there any blocker/challenge on that being possible?