Hey team I am working on differentiating between <https gith Airbyte #contributing-to-airbyte

Hey team, I am working on differentiating between ...

Davin Chia (Airbyte)

03/12/2021, 2:36 PM

Hey team, I am working on differentiating between user and airbyte exception in our error tracking and would love some quick input. Right now error tracking mostly happens in the core Airbyte components (Server and Scheduler). This means more precise exception handling will require some changes to the Airbyte Protocol fields. I’m wondering where do we think these exceptions will come from and what of these are worth catching? In my (limited) experience, it seems modifying the

AirbyteConnectionStatus

will get us most of the way there in terms of user errors (don’t expect authentication/networking information to change often, is that a wrong assumption?) The alternative is amending the other protocol methods to allow them to also pass more detailed errors; obviously a bigger change.

Davin Chia (Airbyte)

03/15/2021, 11:07 AM

Following up on this, after taking some time to understand the code (focusing on

CheckConnection

): • We actually do not report operation failures if the

CheckConnection

operation fails. This is because we report failures based on the job outcome. So even though the `StandardCheckConnectionOutput`’s status field is set to

FAILED

here, we still report this as a success here. This is a bug that should be fixed. • We don’t actually track jobs the first time a source/destination is set up (e.g. the onboarding flow) as scope is null. We are potentially missing data here. I see this was implemented explicitly in this PR. Should we change this? • I’m not sure if the way we have it set up propagates temporal exceptions to the client. Something like this doesn’t propagate and temporal keeps on trying. I’m still looking into this (@Jared Rhizor (Airbyte) do you know?)

6 Views

Open in Slack

Previous Next