Hi! I apologize in advance if that's not the right place for that. I am using the airbyte-python-sdk to create, update, and delete sources (currently focused on file sources only, served from a private S3 bucket). Source creation and deletion work flawlessly, and the source test on the Airbyte UI has no issues. When I try to patch a source, even using the same source configuration, it updates the source successfully, though testing the source on the Airbyte UI crashes:

Internal message: __init__() got an unexpected keyword argument 'sourceType'
Failure origin: source
Failure type: system_error

Traceback (most recent call last):
  File "/airbyte/integration_code/main.py", line 12, in <module>
    launch(SourceFileSecure(), sys.argv[1:])
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py", line 156, in launch
    for message in source_entrypoint.run(parsed_args):
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py", line 91, in run
    yield from map(AirbyteEntrypoint.airbyte_message_to_string, self.check(source_spec, config))
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py", line 112, in check
    check_result = self.source.check(self.logger, config)
  File "/usr/local/lib/python3.9/site-packages/source_file/source.py", line 114, in check
    client = self._get_client(config)
  File "/usr/local/lib/python3.9/site-packages/source_file/source.py", line 78, in _get_client
    client = self.client_class(**config)
TypeError: __init__() got an unexpected keyword argument 'sourceType'

2023-07-12 21:56:37 INFO i.a.w.t.TemporalAttemptExecution(get):138 - Cloud storage job log path: /workspace/93215c4c-9414-46cd-9886-be76c4ed7229/0/logs.log

I am using

airbyte-api==0.15.1

which is currently the most up-to-date version according to pip. Some code snippets of this action (were cut from the original context, but I believe it serves the point):

configuration = shared.SourceFileSecure(
    dataset_name=dataset_name,
    format=data_source_object.get_type(),
    url=data_source_object.value.url,
    source_type=shared.SourceFileSecureFileSecure.FILE_SECURE,
    provider=shared.SourceFileSecureProviderS3AmazonWebServices(
        aws_access_key_id=storage.access_key,
        aws_secret_access_key=storage.secret_key,
        storage=shared.SourceFileSecureProviderS3AmazonWebServicesStorage.S3,
    )
)

req = operations.PatchSourceRequest(
    source_id=source_id,
    source_patch_request=shared.SourcePatchRequest(
        configuration=configuration,
        name=name,
    )
)

res = get_client().sources.patch_source(req)

Thanks for your help!

Ron - Thanks for bringing this to our attention! Can you try updating without adding source_type to the configuration and then re-testing the source in the UI? That field is not required when updating as it would be when creating. I do see that our API docs incorrectly show us receiving sourceType in the configuration for different sources on updates so will take that as an action item to resolve that assuming it’s just the fact that it’s being passed in and not something else that’s going wrong.

Yes, I should have mentioned that I already tried not adding the source type to the configuration, which led to a validation exception saying that this field is required, so maybe there's a mismatch between the expected payload and the configuration validation

Ah yes I see that. I’ll work on a fix for this which will allow you to pass that source type in and we’ll revisit this as a team to see how we can improve the ergonomics around updates. Thanks for sending this in and will let you know when it’s fixed!

Thanks! I appreciate your responsiveness. Let me know when there's a fix so I can update and retest this issue

A fix should hopefully be in for this @Ron Braha - give it a bit as our deploy pipeline usually gets pretty full during this time of day. I’d say either this evening or tomorrow morning should be a safe bet to retest and see if you still run into this issue.

Thanks! I will retest it today and let you know if that worked