hi, all. Is it possible to process compressed(bz2)...
# troubleshooting
u
hi, all. Is it possible to process compressed(bz2) csv file with flink sql filesystem connector ? I can select some records but only a few of the dataset) even with raw format. Is the bz2inputstream read the data as 'line' or just block size of the bz2 file ?
ok, i guess i found the reason. flink csv connector will got data stream from bz2inputstream which requires \r\n as the newline seprator. but the plain csv file can use \n newline terminator also. just rewrite the csv file with \r\n and it works
d
Ok, glad you found a solution