Airbyte is an open-source data integration engine that helps you consolidate your data in your data warehouses, lakes and databases.

Airbyte

Hi everyone, any recommendation/doc for monitoring Airbyte in k8s environment?

Hi Roy, what would you like to monitor exactly?

count of job success, failure, pending, job duration etc; planing to create a dashboard in Grafana

and some connector specific metrics, Salesforce API latency

Hi <@U02R3BWTR39> I'm afraid that we do not yet expose these metrics for monitoring. I'd suggest you upvote this <https://github.com/airbytehq/airbyte/issues/392|issue> and share your need. Our <https://airbyte-public-api-docs.s3.us-east-2.amazonaws.com/rapidoc-api-docs.html#overview|API> could help you to build this kind of metric but it will require quite a bit of scripting work from your side.