Hey all, So my organization really likes Airbyte and we are looking to build a CICD pipeline to deploy our fork of Airbyte onto an EC2 instance anytime we contribute to our fork. However, we also have some custom connectors we would love to contribute back to the broader Airbyte repo. Are there recommendations on a git workflow pattern to support this? Because right now, if we were to create a PR back to the main Airbyte repo we would inadvertently be contributing back the custom connector code plus all of our CICD process code of Terraform and GitHub Actions.

That’s quite a puzzle! Maybe have two forks? One is used for your internal stuff, and then you cherry-pick to the other fork to create a PR for inclusion in the airbyte version?

Hmmm....gotcha gotcha. With that in mind, I'm thinking through a git workflow that would prevent the management of two forks for contributions and our own CICD workflow. Here is what I came up with: https://miro.com/app/board/o9J_lq0I4-A=/?invite_link_id=455287240968 I'd be curious what you think of this as a proposal for supporting both systems

Hmm. Your picture makes sense in general. You might find you prefer your “master” to be the yellow line rather than the blue line, since that’s core to your company, although either will probably work fine. It’s basically maintaining two master branches, one of which is set to the opensource project and related contributions and the other set to your internal. That’s roughly equivalent to two forks but might be cleaner from a day to day management perspective. You’ll need to be fairly aware of what’s in each branch so you don’t pollute a branch unexpectedly.

Hey Jenny, Thank you again for so much support and communication with my various inquiries! After working through the git workflow with another one of the developers at our company we are realizing it might be too cumbersome to maintain a single repo with git management and are instead going with you're initial configuration of two repos where we have a private clone of airbyte we will update manually intermittently and maintain private custom connectors in for internal APIs and then a public external repo where we will develop custom connectors we intend to contribute back to the community. With that said, it might be helpful as part of the custom connectors getting started section to outline a strategy like this as I'm sure we aren't the only group that would want to have a deployable Airbyte system with private custom connectors and public custom connectors that we are willing to share out. I'm happy to try writing up an initial draft if you point me in the right direction for that.

The docs/contributing-to-airbyte/ directory seems like the right place for the document. You can add to an existing doc or start a fresh one and we’ll figure out where to link it in. I’m CCing @abhi for visibility on this - I think it’ll be great to have the forking process documented for people who want to maintain an internal version.