Hello! Got two performance improvements PRs. One i...
# connector-contributions
t
Hello! Got two performance improvements PRs. One is super quick and fresh: https://github.com/airbytehq/airbyte/pull/46374 The second one is out there for some time, and I got no feedback. https://github.com/airbytehq/airbyte/pull/44428 Both changes improve performance substantially, when we run them on our infra, with patched docker images.
w
I'm assuming this performance improvement would help all sources with resumable streams? We have seen Stripe dogging and I'm wondering if this might be the same root cause
u
Thanks a lot for the contribution @Tomasz Szuba, performance improvements are always welcome. I asked the Python CDK team to take a look and review/merge both again.
t
@William Kaper the seconf PR addresses all non-declarative resumable streams.
Declarative ones are certainly possible to also improve, but I wanted first to check how the team likes this, before proceeding further
The first PR addresses all declarative streams. Json schema is parsed once, instead of on every time new record is created.
u
https://github.com/airbytehq/airbyte/pull/46374: running tests https://github.com/airbytehq/airbyte/pull/44428: @Brian Lai (Airbyte) @Maxime Carbonneau-Leclerc (Airbyte) are going to take a look.