Hi Everyone.. can I confirm if the s3 ingestion mechanism in datahub supports kms encrypted buckets?

it should I think

hmm ok Ive given datahub access to GetObjects from the s3 bucket I'm trying to ingest from and access to the kms key being used to encrypt the bucket... and getting errors around unable to GetObject... having a look at some of the code I think the ingestion mechanism is using pyspark.. which looks like it needs extra parameters set defining the kms key being used

the source only uses spark for profiling but using boto3 under the hood to get the s3 files