Hi! We have a supplier with a bulk http API that have the following lifecycle: 1. Authentication to an

/auth

endpoint 2. POST request with a csv file as payload with list of ids that should be exported 3. Request a status endpoint and wait for the result file to be ready 4. Download the file through another endpoint. We wanna fetch data from this api at least nightly. Is this workflow worth implementing a custom connector for this, or should I just hack something together in Airflow? :)

a number of airbyte connectors already do this e.g: amazon ads, facebook marketing, salesforce BULK APi, etc.. if you’re already using airbyte it seems like doing a custom connector would be the better way to go since it keeps all your pipelines in one place and you get all the benefits of the platform.

Interesting! I'm pretty new to Airbyte, and especially to the CDK. Is this where you would start?: https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-amazon-ads https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-facebook-marketing etc

yeah those dirs contain the code for the connectors mentioned