Hi, I'm struggling importing into Redshift Serverless. I used Airbyte to Export as Parquet to S3 then did a Redshift COPY to load the data (some of the data is coming cross account and it was easier to set up cross account S3 bucket than DB access). But I can't seem to find a valid data type to load `_airbyte_additional_properties`; I've tried SUPER, VARBYTE, VARCHAR(Max). The VARCHAR(MAX) gives a sensible error message saying incompatible parquet schema for column. The other two just fail with seemingly random .cpp errors. Any other suggestions for column data type? I'm starting to think Redshift Serverless is more like Redshift Spectrum than actual Redshift (so data types which say 'this doesn't work with Spectrum' are out?) Any ideas, suggestions or workarounds much appreciated. Parquet for

_airbyte_additional_properties

comes out as

_airbyte_additional_properties: map<string, string ('_airbyte_additional_properties')>

child 0, _airbyte_additional_properties: struct<key: string not null, value: string not null> not null

child 0, key: string not null

child 1, value: string not null

(This also seems to be affecting a timestamp field I'm using, which comes through as:

struct<member0: timestamp[us, tz=UTC], member1: string>

child 0, member0: timestamp[us, tz=UTC]

child 1, member1: string

)

Hello Dave Tomkinson, it's been a while without an update from us. Are you still having problems or did you find a solution?

I think this may have been due to the 1Mb size limit on SUPER columns in Redshift, I switched how I was loading. So now the data loads, but with just the records that are too big for Redshift missing, which is acceptable for us for now (then hopefully when 16Mb SUPER columns goes GA next year it'll all work fine 👍)

@Mehak Kapoor @Vibhor Agrawal @Shiraj Gagneja

Thanks @mangole just faced this issue.