Hey Airbyte folks! I'm wondering, is there any plans to add a sync timeout in the nearish future? I found today one of our postgres->postgres connections was running for over 6 days, clearly hung on something, but I didn't know because I only have error slacks turned on. It would be nice to set a 2 hour timeout on these big syncs, so they don't get hung indefinitely.

There is an option to configure timeout using env variables: https://github.com/airbytehq/airbyte/blob/90a5dfb5176b05d8c62097c58747b540f0c96f32/.env#L79-L86

Can I do something like 0.1 days?

No https://github.com/airbytehq/airbyte/blob/8934f3e8f60823b42c4f45e1083519be0f650b68[…]e-config/models/src/main/java/io/airbyte/config/EnvConfigs.java

Setting it to one day is probably good enough for now. Thanks!!

@Derek Wene can you share logs? We are debugging an issue that may be related.

Sure, here's the 2 attempts. First took 73h and failed, then the second attempt 81 hours

I also had issues using the incremental + history dedupe. For whatever reason, the second time it ran, all the tables were emptied, even though there was tons of data in the airbyte tables. So now I'm using full refresh overwrite

^^ @charles

thank you!

we will be debugging this more today so i'll report back here today or tomorrow

Yea @charles postgres destination seems broken. I tried resetting my data, but that's hung too.

@charles any update? 🙂

Hey @Derek Wene, is it possible to you share the logs from the reset sync? It's hanging in what part?

@Derek Wene we have pushed a fix for a related issue that lowers our timeouts in failure cases. Could you try again using v0.32.2 and see if you have the same issue? If you do, then this might require a separate fix

Oh cool! I'll give it a shot. I was able to revert back a few versions, and nuke all my

_airbyte_

tables, and it was able to sync again.

I really appreciate that there's more postgres logs now!

@[DEPRECATED] Marcos Marx @Lake Mossman here's my latest postgres logs. It appears the job was successful, although I don't know about that last exception.

Just took a look; that null pointer exception is from an unrelated change that was made recently, but it shouldnt affect your syncs. I will reach out to the one who made the change to fix that null pointer. Thanks!