Hello :wave: , Is there a typing system such as on...
# random
d
Hello 👋 , Is there a typing system such as one Spark supports (i.e.
org.apache.spark.sql.types.{ArrayType, DataTypes, MapType, StructField, StructType}
) to specify input schema for datastream API job? (I’ve seen it being used only in table API contexts) If not, to which data type should I set timestamps when writing POJO-schema classes? I’m using Scala if that matters.
m
The DataStream API allows you to define your own data types, only the Table API has one predefined. You could use Instant for a timestamp. See https://github.com/immerok/recipes/blob/main/kafka-json-to-pojo/src/main/java/com/immerok/cookbook/events/Event.java as an example in Java
Or you could consider using the Table API to read and then convert to DataStream
d
thanks for the pointers gratitude thank you