Hi! I am using the

airbyte-python-sdk

and trying to use the

streams/get_stream_properties

endpoint, but it crashes before returning the response. From checking the get_stream_properties method with a debugger, I can see that the response is returned from the API request successfully, but crashes as soon as it tries to run the

utils.unmarshall_json

method to normalize the response. Here's the stack trace of your end:

File "/venv/lib/python3.11/site-packages/airbyte/streams.py", line 34, in get_stream_properties
    out = utils.unmarshal_json(http_res.text, Optional[shared.StreamPropertiesResponse])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/airbyte/utils/utils.py", line 681, in unmarshal_json
    out = unmarhsal.from_dict({"res": json_dict})
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/dataclasses_json/api.py", line 72, in from_dict
    return _decode_dataclass(cls, kvs, infer_missing)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/dataclasses_json/core.py", line 206, in _decode_dataclass
    init_kwargs[field.name] = _decode_generic(field_type,
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/dataclasses_json/core.py", line 282, in _decode_generic
    res = _decode_dataclass(type_arg, value, infer_missing)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/dataclasses_json/core.py", line 141, in _decode_dataclass
    kvs = {decode_names.get(k, k): v for k, v in kvs.items()}
                                                 ^^^^^^^^^

Exception Type: AttributeError at /v1/data-streams/
Exception Value: 'list' object has no attribute 'items'

I will add that the

source_id

I used is of a file source (CSV), and the

destination_id

I used is of a Snowflake destination

Hey Ron, thanks for the detailed error report! We’ll take a look to see what’s going on, likely something to do with how our SDK is being generated. If we end up needing to publish a new version of the SDK we will let you know.

Thanks! following this thread for new updates

@Ron Braha do you mind sharing what version of the python SDK you’re using? I assume latest, but wanted to confirm.

Hi, When posting this, I was using version 0.15.1 I noticed today that there was a newer version (0.24.2), which I tested today and experienced the same issue

Hey Ron - just a quick update, our team is looking into this but the fix isn’t straightforward. Will keep you posted as we get closer to a solution.

Thanks for the update! I'll keep following up here

Hi Ron - as of yesterday we actually do think we’ve found the issue and are going to try to correct it and re-release the SDK in the next day or so. Thanks for your patience!

That's great news! I will hold and wait for updates

Hey Ron - that wasn't the update that needed to go out, I'll try to have that out today and it will hopefully be available tomorrow!

Thanks for clarifying, I assumed that, but figured it worth posting the exception here in case it can be helpful

We do appreciate that Ron - thanks for your patience. We had some unexpected work come up today and didn’t get a chance to put up a new version of the SDK. We’re planning on working on that tomorrow and will have a new version of the SDK for you by Wednesday!

Hey, any updates on this issue?

Ron, thanks for your patience with this one. We republished the sdk yesterday, can you update and give this another try?

Hi, Thanks for releasing this update! unfortunately, it seems like I'm getting a similar response:

unable to unmarshal [{"streamName":"Phone numbers api test","syncModes":["full_refresh_append","full_refresh_overwrite"],"sourceDefinedCursorField":false,"propertyFields":[["name"],["phone_number"]]}] as typing.Optional[airbyte.models.shared.streampropertiesresponse.StreamPropertiesResponse]

Stack Trace:

Traceback (most recent call last):
  File "/venv/lib/python3.11/site-packages/airbyte/utils/utils.py", line 682, in unmarshal_json
    out = unmarhsal.from_dict({"res": json_dict})
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/dataclasses_json/api.py", line 70, in from_dict
    return _decode_dataclass(cls, kvs, infer_missing)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/dataclasses_json/core.py", line 219, in _decode_dataclass
    init_kwargs[field.name] = _decode_generic(field_type,
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/dataclasses_json/core.py", line 306, in _decode_generic
    res = _decode_dataclass(type_arg, value, infer_missing)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/dataclasses_json/core.py", line 149, in _decode_dataclass
    kvs = {decode_names.get(k, k): v for k, v in kvs.items()}
                                                 ^^^^^^^^^

The above exception ('list' object has no attribute 'items') was the direct cause of the following exception:
  File "/venv/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/django/views/decorators/csrf.py", line 56, in wrapper_view
    return view_func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/rest_framework/viewsets.py", line 125, in view
    return self.dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
    ^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The tested source and destination are File -> Snowflake

Hmm I’m wondering if our publish process isn’t picking up the changes we’re making. We’ll dig back into that now.

I'll add that this issue is no longer in high severity on my end, but I'd be happy to test it further after you have any updates

We’re digging in - glad to hear it’s no longer blocking you, but will keep you posted here on the progress we make. Thanks again for your patience and help in testing!