Hello everyone, I'm facing a problem regarding Airbyte. Can anyone help? Fetching data from shopify and sending it to Big Query. We are fetching orders data and transaction data from shopify account. Now the amount of total

orders

data fetched is

71,613 committed records

. But total

transaction

data fetched is

11,134 committed records

. But the number was supposed to be same. We fetched orders data and transaction data together and also separately, but the result is same. Transaction data’s are not getting fetched totally. Can't seems to find the reason behind this. Can anyone please help? AirByte is hosted in one of our cloud machines and the configs are: • OS Version / Instance: Ubuntu 22.04 x64 • Memory / Disk: 8Gb / 160 Gb • Deployment: Docker • Airbyte Version: 0.40.8 • Source name/version: Shopify • Destination name/version: Big Query i have attached the log files.

@[DEPRECATED] Marcos Marx turned this message into Zendesk ticket 2520 to ensure timely resolution!

How can I check it's status? @[DEPRECATED] Marcos Marx

For the log you shared are you only syncing only `transaction`stream, where did you saw the 71 order records being commited this data comes from Shopify UI?

Hi @Marcos Marx (Airbyte), I am replying on behalf of my colleague since we are working on the same project. The 71k orders were synced in the orders table. Since there must be at least one transaction per order, we should be seeing at least 71k transactions as well. In the log, we see all the orders are being read for transactions, but when it is committed, there are only 11k transactions being written to the database. We have tried both normalised and pure JSON. The data is being pushed on BigQuery. Also, we have tried every sync mode.

Got it, my suggestion is: join the data and see what the missing orders have in common; maybe is their status or something. Also get one ORDER that was not possible to retrieve the transaction and try to send the

curl

command manually to see what Shopify returns to you. I’m not familiar with Shopify and if there is a filter applied to transaction stream

@Marcos Marx (Airbyte) hey Marcos, i am having this problem as well, I checked for all Transactions and Orders and found that the statuses that differentiated them is whether the Transaction has "Receipt" and "message=Success" only