Hi guys Excuse me, I don't understand this post👇. Could you help me? For example, how was it possible for the "drop columns" operation to occur without us noticing? @Leonard Xu @kunni

Thanks for asking. To clarify, no columns were deleted and no

DROP COLUMN

operation occurred. The issue happens when the CDC pipeline starts with an existing downstream table whose schema has fewer columns than the upstream table. In this example, MySQL already had six columns, but the existing Paimon table had only four. During initialization, Flink CDC sent a

CreateTableEvent

containing the six-column upstream schema. Because the Paimon table already existed, the previous implementation continued using its existing four-column schema instead of adding the two missing columns. As a result, records could still be written successfully, but only the four columns available in the Paimon table were written. The values of

source_system

and

event_time_ms

were not written to the downstream table. Therefore, “silently dropped” refers to the values of the missing columns being omitted during the write. It does not mean that the columns were deleted from MySQL or Paimon. Flink CDC 3.7 addresses this initial schema mismatch through

existing-table.schema-expansion.mode

. It can detect the mismatch or add compatible missing columns before data is written.

Thank you very much mr @kunni for the comprehensive explanations you provided.🙏