helpful-optician-78938
03/15/2022, 11:40 PMtype=self._converter._get_column_type(
actual_schema.type,
(
getattr(actual_schema, "logical_type", None)
or actual_schema.props.get("logicalType")
),
),
modern-monitor-81461
03/16/2022, 11:04 AMfield_logical_type_mapping: Dict[str, Any] = {
"date": DateTypeClass,
"timestamp-millis": TimeTypeClass,
"decimal": NumberTypeClass,
}
But the Avro spec has a few more:
• time-millis
• time-micros
• timestamp-micros
• local-timestamp-millis
• local-timestamp-micros
• duration
• uuid
In Iceberg, I need to map the following types:
• UUIDType -> mapped as an Avro String right now, would be nice to use uuid
logical type
• TimeType -> right mapping would be time-micros
• TimestampType (with local and UTC options) -> right mapping would be timestamp-micros
or local-timestamp-micros
modern-monitor-81461
03/16/2022, 9:20 PMnullable
field. I can't figure out how to use it with Avro. I know the magic is happening here, but depending on the Avro mapping used, I get different results. I just can't figure out the pattern to use... I've updated my Iceberg PR with my latest try to make nullable work. It almost works, but it fails for DecimalType. I can go into details, but first I'd like to know if what I'm doing is the right way or not. I thought I could use _nullable
in Avro like this. Am I right?