With Github source and Postgres destination each sync misses Airbyte #troubleshooting

With Github source and Postgres destination, each ...

Hamza Liaqat

04/05/2022, 2:10 PM

With Github source and Postgres destination, each sync misses some data. Specifically, for a set of repos, size of

_airbyte_raw_issues

table is 1041 after sync 1, for example. Yet, if I rerun this experiment from scratch, the size of

_airbyte_raw_issues

would be 1030 (for example) such it's not reproducible. I ran many experiments today but airbyte seems to miss at least a few records in raw issues table in each sync.

Hamza Liaqat

04/05/2022, 2:11 PM

• I'm using Airbyte API • In Github Configuration, my list of repos is long string with > 20 repos e.g. "org/repo1 org repo2 ..."

Hamza Liaqat

04/05/2022, 2:12 PM

• This discrepancy doesn't seem to occur in other streams/tables (e.g. comments)

Marcos Marx (Airbyte)

04/05/2022, 5:04 PM

Hi there! Thank you for your time posting in our Slack Channel. This week we’re moving our support discussions over to discuss.airbyte.io; if you would be so kind as to post your questions there, you’ll be able ask for help from fellow users as well as the Airbyte team. It is our intention that as of Friday afternoon (Pacific Time) we will lock this channel to everything except administrator posts. We’ll leave everything in place for the rest of April if you need to get information or troubleshooting from existing threads, with the intention of archiving the channel in early May. We really appreciate everyone helping us to build our community, and we’re looking forward to the next phase.

6 Views

Open in Slack

Previous Next