Hello everyone , i am new in apache flink and i ha...
# random
m
Hello everyone , i am new in apache flink and i have some questions that needed an accurate answers i am using flink sql to create source and sink table that consume messages from Kafka topic and insert it to iceberg table my question here is where flink stored table's data for both source and sink table Is it in memory or where ?
d
Metadata about table definitions is stored in catalogs. There is a default in-memory catalog, and various persistent catalogs. See https://nightlies.apache.org/flink/flink-docs-stable/docs/dev/table/catalogs/ for details.
m
thanks for replay yes i know that but, i am asking about table data itself where it stores?
d
In the example you gave, the source Table is backed by Kafka, and the sink Table is backed by Iceberg. Flink will process the data being read from and written to these Tables without mirroring it elsewhere. However, Flink may need to create a materialized view in order to execute your query logic (e.g., if you're doing a GROUP BY aggregation). If so, that data/state will be stored in the state backend configured for your Flink cluster, e.g., RocksDB. That state is not exposed as a Table -- you can't access it directly.