My stripe 'Payment Intents' stream is in Failure, ...
# replication-troubleshooting
g
My stripe 'Payment Intents' stream is in Failure, and I don't understand why 🤔 Here are the logs (error of first attempt before line 510) The state is never saved because the stripe connector is super long and I never manage to reach the success state, so I always come back from the start, failure after failure. How can I fix my issue / at least do incremental loadings day per day that save the states regularly, without waiting for the whole replication to be in success? Thank you for your help! 🙏
Loading mode : Incremental | Deduped History Lookback Window in days : 8 Data Request time increment in days : 1 (-> I thought this would enable to do 1 day batches and save the state regularly, but it does not! the state is never saved before the whole history is processed)
Thanks in advance for your help 🙏
m
If you use another encoding besides avro it works?
g
The ingestion actually works like a charm on stripe accounts with fewer data, with the same parameters. What makes you think that avro could be the culprit? To me, the main problem is Stripe slow ingestion time https://github.com/airbytehq/airbyte/issues/18638 With my Salesforce connector, it takes 10 minutes to ingest 1 GB With my Stripe connector, it takes 6 hours There's a 35-factor 😕 It would take 2 weeks to ingest my stripe data but it always fail in between, and I have to start from scratch every time. The state is never saved and I have no way to control the dates externally and proceed by batches, so I concretely can't ingest my stripe data with Airbyte 😞
m
I saw you found this issue in Github: https://github.com/airbytehq/airbyte/issues/18638
g
Yes indeed - we abandoned using Airbyte to ingest Stripe and went back to our own python ETLs 😕