This message was deleted.
# random
s
This message was deleted.
b
FileSource.forBulkFileFormat doesn't support writing to a single file afaik, so even if you are able to pre-pend headers to your data stream. How will you figure out which file the header row is written to? https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/connectors/datastream/filesystem/#rolling-policy
a
@Bhupendra Yadav I am able to write the data to a single file. As per my requirements, only a single file is present and the header row can be written to it.
b
I am able to write the data into a single file I am curious to know how?
a
@Bhupendra Yadav I have already shared the code snippet in my question. The same code writes the data to a single file.
b
Oh got it, you are setting the parallelism as 1. Also, I believe in local this might be working but even with 1 parallelism, the output file will be committed after each checkpoint interval, so make sure to test this on a bit large file where at least 1 successful checkpoint happens(in that case there will be multiple files created). Btw, Pls ignore if you are not using checkpoints.
a
noted, will try it, thanks