I'm running a self-hosted Airbyte deployed to Kube...
# replication-troubleshooting
m
I'm running a self-hosted Airbyte deployed to Kubernetes on EKS with helm charts. After upgrading to Airbyte 2.0.1 (from 1.6 so previous versions could cause this) I noticed that some syncs finish with Sync succeeded status on the dashboard but don't load any data (0 bytes | no records loaded). When I looked into Airbyte database I noticed that attempts are marked as succeeded status but the attempt output report sync failures. I see many different reasons why jobs may fail like this (details in the 🧵) and they can be handled and resolved separately but my primary question is why Airbyte reports failed attempts as succeeded. This causes some major issues: • Airbyte doesn't restart failed attempts • Airbyte doesn't send failed connector webhooks • It's hard to notice failed attempts when looking at the connectors list Has anyone noticed a similar behavior? Is this an Airbyte issue that we should report on Github?
The airbyte query that selects succeeded attempts that has failures in the output:
Copy code
SELECT 
    attempts.status,
    attempts.failure_summary,
    attempts.output->'sync'->'failures'->0->'failureType' AS failureType,
    attempts.output->'sync'->'failures'->0->'failureOrigin' AS failureOrigin,
    attempts.output->'sync'->'failures'->0->'externalMessage' AS externalMessage,
    attempts.output->'sync'->'failures'->0->'internalMessage' AS internalMessage
FROM attempts 
WHERE JSONB_ARRAY_LENGTH(attempts.output->'sync'->'failures') > 0 
AND attempts.status != 'failed'