Hey all, just looking into Airbyte and ran into a similar issue as mentioned here. Looking to set up s3 as a destination but at our organisation we have temporary keys. The main issue is testing in local becomes impossible as we aren't allowed to have any users with static IAM credentials. Does anyone know if there has been any resolution to this?

@[DEPRECATED] Marcos Marx turned this message into Zendesk ticket 2718 to ensure timely resolution!

Did you try to use the Instance Profile method?

@Marcos Marx (Airbyte) Yep was able to get it working with the Instance Profile method 🙂 thank you!!

Amazing!

hello @[DEPRECATED] Marcos Marx i am trying to setup an S3 connection with the EC2 instance profile instead of AWS keys but getting this error:

2023-05-04 17:56:51 ERROR i.a.w.i.VersionedAirbyteStreamFactory(internalLog):313 - Traceback (most recent call last):
  File "/airbyte/integration_code/source_s3/source_files_abstract/source.py", line 63, in check_connection
    for file_info in stream.filepath_iterator():
  File "/airbyte/integration_code/source_s3/stream.py", line 56, in filepath_iterator
    response = client.list_objects_v2(**kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 530, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 960, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied

the same config works fine with AWS keys. verified the instance profile is setup correctly and the IAM/ bucket permissions are in order. any ideas on how i can fix this?