Hi team! I have a question about polling for job s...
# connector-development
e
Hi team! I have a question about polling for job status for a custom destination connector. My team is developing a custom destination connector (using Python), sending data using POST requests. The API we are calling with this connector performs data import asynchronously - the POST request returns a job id, and you can then run GET requests to check the status of that job. Ideally we would like the Airbyte sync to only display “Succeeded” if the status of the job indicates it was successful. Sometimes this might take a while (up to an hour) for very long jobs. Is there a recommended approach to implementing status polling when posting data to an API with this behavior? Of course the easy thing is to use some “while” logic with a timeout to keep checking the status of the job id, but we are wondering if there is some better practice we should be aiming to use instead.