https://linen.dev logo
d

Davin Chia (Airbyte)

07/09/2021, 3:19 AM
@Chris (deprecated profile) @Jared Rhizor (Airbyte) did we double check that our Java connectors are actually using the entire RAM allocated to them in the Kube resources? Java's default behaviour is to use 1/4 of the host's container memory so I'm not sure the Java connector allows itself to use all the allocated memory
u

user

07/09/2021, 4:10 AM
I'm guessing what this means is the connector uses 1/4 of the RAM allocated if it's specified
u

user

07/09/2021, 4:10 AM
and 1/4 of the node's memory if it's not specified
u

user

07/09/2021, 6:06 AM
I thought that wasn’t a problem after Java 10
u

user

07/09/2021, 6:06 AM
d

Davin Chia (Airbyte)

07/09/2021, 6:06 AM
I didn’t test it though
u

user

07/09/2021, 6:13 AM
yes those no longer apply
u

user

07/09/2021, 6:45 AM
Ah
u

user

07/09/2021, 6:46 AM
It’d be tricky to reliably specify custom memory reqs to different entrypoints / executables
u

user

07/09/2021, 6:47 AM
Hmm
u

user

07/09/2021, 6:48 AM
Haha yeah I punted on this by writing this issue: https://github.com/airbytehq/airbyte/issues/3440
u

user

07/09/2021, 8:33 AM
so you’re saying that now that we can specify cpu/memory requirements to both docker and kube containers, these settings should be somehow forwarded to the jvm too?
u

user

07/09/2021, 8:33 AM
yeap
d

Davin Chia (Airbyte)

07/09/2021, 8:34 AM
we need to figure out a way to inject these via the
-Xmx
flag