Anyone else has been using the Iceberg extension t...
# dev
m
Anyone else has been using the Iceberg extension to ingest from iceberg table? Do you know if the iceberg filtering support non-partition column of the Iceberg table? https://github.com/apache/druid/pull/14329#issuecomment-2143340566
a
Yeah, last when I checked, dynamic filters were not supported on non-partitioned columns. The filter predicates usually only apply to partitioned keys. This is true for reading from a Delta Lake as well. I don’t see an explicit mention of this behavior in the Iceberg docs though.
m
I dont see this in the Iceberg docs too. I think we should say this in the Iceberg docs and either log a warning or fail the ingestion if filtering is given on non paritioned columns
In the future we can probably support this by pushing the iceberg filtering into Druid filter spec.
Then we can filter when Druid ingest/read from the files
👍 1