Hi All. I have a scenario where I need to connect to same endpoint using different IDs as request param. Response for all of these IDs is same and need to be persisted in same table at our end. To solve this problem we have come up with two approaches: 1. Create a simple connector that would fetch the response for each ID passed as parameter and write to target table in Append mode. Loop through the IDs at orchestration layer. This approach is working. 2. Pass the list of IDs as parameter to the airbyte and based on number of IDs the looping should happen inside the Airbyte connector. For this I have defined one class and JSON schema is passed as via config file. This only works for first element and the job goes to success. For my testing I passed two IDs and also defined two class one for each ID. I am able to get the response but on UI this would mean writing to two separate output.(following the example of Customer and Employee stream) Is there a way I can have only one stream defined in the connector code and can loop through the same stream(using different params) inside connector code?

Hi Dipti, I need to look into this further, let me get back to you!

thanks @Nataly Merezhuk (Airbyte)

Dipti, everything that I'm seeing so far points to looping inside a connector like this not being possible. So that I can better understand, could you clarify - do you basically need to connect to the same API but with different IDs, and then have the resulting information persisted to one single destination?

Yes you are correct. It is the same endpoint. ID is passed as report param. This is why I wanted to define only one stream. Within the code i was fetching ID as list from the config and then trying to add to stream list :

streams = []

for category in category_list:

payload_prep = '{' + f'"category":"{category_new}",{report_params}' + '}'

payload = json.loads(payload_prep)

streams.append(SensortowerStream(config=config, header=header, url=report_url, payload=payload,response_json=response_json, authenticator=auth))

return streams

Hi Dipti! Thank you for clarifying. Unfortunately this type of connector isn't possible at this time, as a work around you'd. need to create multiple connections! Sorry for the inconvenience!

Thanks for clarifying Nataly.

No problem!