Hello! I would like to ingest data from dbt into DataHub. I want to ingest manifest and catalog that are put on s3 bucket. Is there a way to use the instance level metadata for the role? (IAM roles) https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html#id1 Right now I am getting the following error when I do not specify a role:

INFO - ERROR:root:Error: Unable to locate credentials

From the code perspective, I guess that for this to work this line needs to be modified: https://github.com/datahub-project/datahub/blob/master/metadata-ingestion/src/datahub/ingestion/source/aws/aws_common.py#L114 Am I right? Using version

acryl-datahub[dbt] == 0.8.36

Hi Bartłomiej! Looking into this to see if I can get you an answer asap

It looks like you're correct - I don't see anywhere that we read from instance-level metadata - I'm presuming you're running ingest from an EC2?

@bumpy-daybreak-85714… any update? Running ingestion with airflow I’d like to pass an airflow connection id rather than send the individual aws keys. Are you doing that?