https://linen.dev logo
#ask-community-for-troubleshooting
Title
# ask-community-for-troubleshooting
i

Isaline L.

07/15/2021, 12:34 PM
Hello, anybody ever made a source connector to deal with SOAP APIs? I'm trying to adapt the HTTP connector to do that, but was just curious to know if somebody already went through this and has some tips/wants to share the experience.
āœ… 1
s

s

07/15/2021, 2:42 PM
is there an SDK available for this already? If so it might be easiest to inherit the
Stream
class (instead of
HttpStream
directly) and use that. In general, it might be helpful to consider if the
Stream
class is a better fit than
HttpStream
, Iā€™d be curious to know how it ends up!
i

Isaline L.

07/15/2021, 2:50 PM
For now I only needed to override
_create_prepared_request
from HttpStream to add the "json" (which is actually xml then) into args["data"] instead of args["json"] and it works.
I think HttpStream could very easily be adapted to handle different type of data, but it depends up to which level it has to be resilient. I'm new to Airbyte so I don't know exactly what you promise there. šŸ˜…
I actually adapted HttpStream first but somehow couldn't find a way to have my own connector uses my changed version so I decided to take that shortcut for now.
But yeah basically, as far as I understand, the only missing link for now is that if you have SOAP, you need to send a XML body, so you shouldn't use the
json
parameter of request, but the
data
parameter. So what I would do is offering another method next to
request_body_json
, like
request_body_data
. People are free to implement whichever they need, and you can call both of them next to each other in
read_records
.
In my case, the json one will return None because I don't have anything to implement there, and that just work perfectly fine.
I just don't know what happens when both
data
and
json
are provided to requests, but that should be easy to find out. ^^
s

s

07/15/2021, 3:48 PM
makes a lot of sense! Created a ticket for this: https://github.com/airbytehq/airbyte/issues/4764
i

Isaline L.

07/15/2021, 3:50 PM
Oh super nice, thanks!
I'll add more details to the ticket as soon as I have a bit of time. šŸ™‚
s

s

07/15/2021, 3:54 PM
that would be v helpful!
10 Views