[Github connector] Hi team. Is there a reason why there is no support for incremental replication for the

pull_request_commits

stream? Is it a technical limitation with GH's api?

I think it can be add a incremental sync mode for this endpoint, probably using the same strategy as Commits and other streams: https://github.com/airbytehq/airbyte/blob/c472340ad34a0c61c440ea22256c6b854abd61f9[…]-integrations/connectors/source-github/source_github/streams.py

Let me know if Marcos' comment helps, Renato!

Hi, thanks for the reply, happy to try it out and contribute a PR if it works, I'll keep you posted on this thread.

Thanks, looking forward to your PR and update!

Hi @Nataly Merezhuk (Airbyte). Looks like it is indeed a technical limitation. Airbyte's incremental system relies on some sort of timestamp and this specific stream does not have any

created_at

or

updated_at

. The only other way I could think about solving this would be to only fetch

pull_request_commits

for pull-requests created or changed after the last sync but not really sure this is even possible within Airbyte's architecture. From my understanding, streams are fetched independently of each other.

Hi Renato, I looked into the schema for that stream and there's a date field. I think this field could be used to setup an incremental sync. I don't have time to test this out today, but will later in the week. If you have a chance to test this though, let me know!

I would love to have the incremental sync as Renato's idea! Just created a discussion on Github.