Hi everyone, I see that the convertCDCTypeToIcebergType doesn't handle

ARRAY

data types. Because of this 1. MySQL to Iceberg pipeline is not able to handle

SET

datatypes in MySQL. a. It mentions here that

SET

MySQL datatype is converted to

ARRAY<STRING>

Flink SQL data type. This should be mapped to a Iceberg datatype which is not currently handled. 2. Postgres to Iceberg pipeline is not able to handle

text[]

datatypes in Postgres. i. Similarly

text[]

Postgres datatype is converted to

ARRAY<STRING>

Flink SQL data type. This should be mapped to a Iceberg datatype which is not currently handled.

Caused by: org.apache.flink.cdc.common.exceptions.SchemaEvolveException: java.lang.IllegalArgumentException: Unsupported cdc type in iceberg: ARRAY<STRING>
	at org.apache.flink.cdc.connectors.iceberg.sink.IcebergMetadataApplier.applyCreateTable(IcebergMetadataApplier.java:238) ~[flink-cdc-pipeline-connector-iceberg-3.6.0.jar:3.6.0]
	at org.apache.flink.cdc.connectors.iceberg.sink.IcebergMetadataApplier.lambda$applySchemaChange$2(IcebergMetadataApplier.java:130) ~[flink-cdc-pipeline-connector-iceberg-3.6.0.jar:3.6.0]
	at org.apache.flink.cdc.common.event.visitor.SchemaChangeEventVisitor.visit(SchemaChangeEventVisitor.java:59) ~[flink-cdc-dist-3.6.0.jar:3.6.0]
	at org.apache.flink.cdc.connectors.iceberg.sink.IcebergMetadataApplier.applySchemaChange(IcebergMetadataApplier.java:119) ~[flink-cdc-pipeline-connector-iceberg-3.6.0.jar:3.6.0]
	at org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator.applyAndUpdateEvolvedSchemaChange(SchemaCoordinator.java:440) ~[flink-cdc-dist-3.6.0.jar:3.6.0]
	at org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator.applySchemaChange(SchemaCoordinator.java:409) ~[flink-cdc-dist-3.6.0.jar:3.6.0]
	at org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator.lambda$startSchemaChangesEvolve$1(SchemaCoordinator.java:250) ~[flink-cdc-dist-3.6.0.jar:3.6.0]
	... 5 more
Caused by: java.lang.IllegalArgumentException: Unsupported cdc type in iceberg: ARRAY<STRING>
	at org.apache.flink.cdc.connectors.iceberg.sink.utils.IcebergTypeUtils.convertCDCTypeToIcebergType(IcebergTypeUtils.java:164) ~[flink-cdc-pipeline-connector-iceberg-3.6.0.jar:3.6.0]
	at org.apache.flink.cdc.connectors.iceberg.sink.utils.IcebergTypeUtils.convertCdcColumnToIcebergField(IcebergTypeUtils.java:59) ~[flink-cdc-pipeline-connector-iceberg-3.6.0.jar:3.6.0]
	at org.apache.flink.cdc.connectors.iceberg.sink.IcebergMetadataApplier.applyCreateTable(IcebergMetadataApplier.java:171) ~[flink-cdc-pipeline-connector-iceberg-3.6.0.jar:3.6.0]
	at org.apache.flink.cdc.connectors.iceberg.sink.IcebergMetadataApplier.lambda$applySchemaChange$2(IcebergMetadataApplier.java:130) ~[flink-cdc-pipeline-connector-iceberg-3.6.0.jar:3.6.0]
	at org.apache.flink.cdc.common.event.visitor.SchemaChangeEventVisitor.visit(SchemaChangeEventVisitor.java:59) ~[flink-cdc-dist-3.6.0.jar:3.6.0]
	at org.apache.flink.cdc.connectors.iceberg.sink.IcebergMetadataApplier.applySchemaChange(IcebergMetadataApplier.java:119) ~[flink-cdc-pipeline-connector-iceberg-3.6.0.jar:3.6.0]
	at org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator.applyAndUpdateEvolvedSchemaChange(SchemaCoordinator.java:440) ~[flink-cdc-dist-3.6.0.jar:3.6.0]
	at org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator.applySchemaChange(SchemaCoordinator.java:409) ~[flink-cdc-dist-3.6.0.jar:3.6.0]
	at org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator.lambda$startSchemaChangesEvolve$1(SchemaCoordinator.java:250) ~[flink-cdc-dist-3.6.0.jar:3.6.0]
	... 5 more

Anyone observed this before? is this a known issue?

@vdontu Looks like reasonable requirements to support complex data type, Could you file two jira tickets ?