<@U01MMSDJGC9> Hi Marcos thank you for answering m...
# replication-ask-ai
v
@[DEPRECATED] Marcos Marx Hi Marcos thank you for answering my questions! My team was wondering if we can override these resource values
k
Based on the information provided in the extracts, it's not directly mentioned how to override the specific resource values you linked to. However, there are ways to customize resource requirements in Airbyte. For instance, you can customize your deployment by creating a
values.yaml
file in the local folder and populate it with default configuration override values. An example of this file can be found in the [charts/airbyte](https://github.com/airbytehq/airbyte-platform/blob/main/charts/airbyte/values.yaml) folder of the Airbyte repository. After specifying your own configuration, you can run the following command to deploy Airbyte with your custom values:
Copy code
bash helm install --values path/to/values.yaml %release_name% airbyte/airbyte
(Source: [Deploy Airbyte on Kubernetes using Helm](https://docs.airbyte.com/deploying-airbyte/on-kubernetes-via-helm)) Additionally, you can configure connector resource requirements for a Sync in three different ways: instance-wide, connector-specific, and connection-specific. The narrower the scope of the requirement, the higher the precedence. (Source: [Configuring Connector Resources](https://docs.airbyte.com/operator-guides/configuring-connector-resources)) For more specific guidance on overriding the resource values you linked to, I would recommend reaching out to the Airbyte community or support for more detailed assistance.
v
For the writer/reader resource questions, for example I have this writer node
destination-snowflake-write-338-0-gvlqp
and I'm seeing the init containers:
main containers:
for our normalization node, we bumped this up to:
over Helm chart, we defined these under
<http://global.jobs|global.jobs>
and these under
worker
our question is: how can we bump up the reader and writer node resources in the chart
Previously I showed you a wrong link. I just updated in main question to point to the
airbyte-worker
folder,
worker.kube-job-configs
t
@Vicky Kwan were you able to find a solution?
v
no we didn't follow up on our end. looked like there had been no more issues with resources. did you also run into this issue @Thiago Guimarães?
t
i actually had some problems already with this part of airbyte`s deployment. the job resources configurations is kinda disorganized in the code and there are a lot of issues. Im currently running 50.12 with no issues - ive tried upgrading to the latest version (50.28) last week and the source jobs simply didnt respect any of the job resources configs. ive kinda hacked my way into the configs and set the env variables by hand in my values.yml file - ref https://docs.airbyte.com/operator-guides/configuring-airbyte/#jobs and https://docs.airbyte.com/operator-guides/configuring-airbyte/#kubernetes-only
v
this is very helpful thank you! Alan is our boss. I'll take a closer look at your solution this week @Thiago Guimarães
🙂 1
t
@Vicky Kwan did you guys manage to make it work ?
v
@Thiago Guimarães yes until we hit another issue recently with Snowflake destination (we are only using v1.x and we are testing if bumping to v3.x will solve this)
how about your team? any performance issue recently?
t
i recently had a problem with the mongodb source connector latest version - it was totally ignoring the env variables, therefore never being able to be alocated to a pod
i managed to fix that by rolling back to the previous source version, but now my write pods (using mostly big query destination) are totally ignoring my job resource configs, resulting in the same problem