Divyanshu Jaiswal
05/29/2024, 9:44 AMorchestrator-repl-job
jobs. Its not deployed via helm (helm list output is empty). Any idea how can we configure toleration/selector for orchestrator-repl-job job in airbyte?kapa.ai
05/29/2024, 9:45 AMorchestrator
pod not respecting the JOB_KUBE_TOLERATIONS
environment variable in earlier versions of Airbyte. This issue has been addressed in later versions. ### Steps to Configure Tolerations for `orchestrator-repl-job`: 1. Upgrade Airbyte Version: Ensure you are using a version of Airbyte where this issue has been fixed. According to the [Github Issue #28389](https://github.com/airbytehq/airbyte/issues/28389), the problem was resolved in version 0.53.196
. Upgrading to this version or later should ensure that the orchestrator
pod respects the JOB_KUBE_TOLERATIONS
environment variable. 2. Set JOB_KUBE_TOLERATIONS
: Ensure that the JOB_KUBE_TOLERATIONS
environment variable is correctly set in your Airbyte configuration. This variable should define the tolerations for the job pods. Here is an example of how to set it: yaml JOB_KUBE_TOLERATIONS: key=<http://k8s.company.com/node-pool,operator=Equal,value=data|k8s.company.com/node-pool,operator=Equal,value=data>
3. Verify Configuration: After setting the JOB_KUBE_TOLERATIONS
, verify that the environment variable is correctly applied to the orchestrator pods. You can check the pod specifications to ensure that the tolerations are included. ### Example Configuration: Here is an example of how you might configure the JOB_KUBE_TOLERATIONS
in your Airbyte configuration: yaml apiVersion: v1 kind: ConfigMap metadata: name: airbyte-config data: JOB_KUBE_TOLERATIONS: key=<http://k8s.company.com/node-pool,operator=Equal,value=data|k8s.company.com/node-pool,operator=Equal,value=data>
### Additional Resources: - [Airbyte Documentation on Configuring Jobs](https://docs.airbyte.com/operator-guides/configuring-airbyte#jobs-1) - [Github Issue #28389](https://github.com/airbytehq/airbyte/issues/28389) for more context on the issue and its resolution. If you are still facing issues after upgrading and configuring the tolerations, you might want to check the specific lines of code mentioned in the [Github Issue #28389](https://github.com/airbytehq/airbyte/issues/28389) to ensure that the tolerations are being applied correctly. If you need further assistance, please provide more details about your current configuration and any error messages you are encountering.Quang Nguyen
08/16/2024, 11:09 AM