Any ideas on how to setup Airbyte in k8s that can scale horizontally? I’m happy to pay for someone to help

Sure, I can guide you on how to set up Airbyte on Kubernetes (k8s) that can scale horizontally based on the information provided in the extracts. Airbyte allows scaling sync workloads horizontally using Kubernetes. The core components (api server, scheduler, etc) run as deployments while the scheduler launches connector-related pods on different nodes [source](https://docs.airbyte.com/deploying-airbyte/on-kubernetes-via-helm). Here are the steps to set up Airbyte on Kubernetes: 1. Install Kubernetes and Helm and add the Airbyte Helm repository:

bash helm repo add airbyte <https://airbyte.github.io/airbyte> helm repo update

2. Create a namespace for Airbyte:

bash kubectl create namespace airbyte

3. Install Airbyte using the Helm chart:

bash helm install airbyte airbyte/airbyte -n airbyte -f ./helm/values.yaml

4. Forward ports:

bash kubectl --namespace airbyte port-forward service/airbyte-airbyte-webapp-svc 8000:80

[source](https://airbytehq.slack.com/archives/C021JANJ6TY/p1677162094191539) In terms of scaling, you need to ensure that your Kubernetes cluster can accommodate the number of pods Airbyte creates. As a rule of thumb, make sure the Kubernetes cluster can schedule up to

2 x <number-of-possible-concurrent-connections>

pods at once [source](https://docs.airbyte.com/operator-guides/scaling-airbyte). If you encounter any issues or need further clarification, you can reach out to the Airbyte community on their [Slack](https://slack.airbyte.io/) or [create an issue on GitHub](https://github.com/airbytehq/airbyte/issues/new?assignees=&labels=type%2Fbug&template=bug-report.md&title=) [source](https://docs.airbyte.com/deploying-airbyte/on-kubernetes-via-helm). If you prefer a managed solution, you can use the free, open-source project [Plural](https://www.plural.sh/) or [Restack](https://www.restack.io) to bring up a Kubernetes cluster and Airbyte for you [source](https://docs.airbyte.com/deploying-airbyte/on-kubernetes-via-helm). Please note that while I can provide guidance based on the information available, I'm an AI and can't personally assist with the setup. If you need hands-on help, you might want to consider hiring a Kubernetes expert or reaching out to the Airbyte team for professional support.