Sergii Mikhtoniuk
01/17/2023, 1:23 AMTable or a DataStream<Row> into DataStream<RowData> ?
I'm trying this:
DataStream<Row> resultStream = tEnv.toChangelogStream(resultTable);
RowRowConverter converter = RowRowConverter.create(resultTable.getResolvedSchema().toSinkRowDataType());
DataStream<RowData> resultStreamRowData = resultStream.map(
converter::toInternal,
InternalTypeInfo.of(resultTable.getResolvedSchema().toSinkRowDataType().getLogicalType())
);
but I think I'm assigning TypeInformation incorrectly as it crashes with type casting errors.
Context: I want to periodically save my table's changelog stream into parquet format. The only way I found so far is to go from Row to RowData only then to Avro::GenericRecord and then to use AvroParquetWriters yuck :(