https://linen.dev logo
Join Slack
Powered by
Hi Everyone, When deploying airbyte on EKS cluster...
# ask-ai
a
Hi Everyone, When deploying airbyte on EKS cluster where I want to use S3 for storage and logging I am getting below error 
ERROR i.m.r.Micronaut(handleStartupException):338 - Error starting Micronaut server: null java.lang.IllegalArgumentException: null at com.google.common.base.Preconditions.checkArgument(Preconditions.java:131) ~[guava-31.1-jre.jar:?] at io.airbyte.config.storage.DefaultS3ClientFactory.validateBase(DefaultS3ClientFactory.java:36) ~[io.airbyte.airbyte-config-config-models-0.44.0.jar:?] at io.airbyte.config.storage.DefaultS3ClientFactory.validate(DefaultS3ClientFactory.java:31) ~[io.airbyte.airbyte-config-config-models-0.44.0.jar:?] at io.airbyte.config.storage.DefaultS3ClientFactory.<init>(DefaultS3ClientFactory.java:24) ~[io.airbyte.airbyte-config-config-models-0.44.0.jar:?] at io.airbyte.config.helpers.CloudLogs.createCloudLogClient(CloudLogs.java:51) ~[io.airbyte.airbyte-config-config-models-0.44.0.jar:?] at io.airbyte.config.helpers.LogClientSingleton.createCloudClientIfNull(LogClientSingleton.java:226) ~[io.airbyte.airbyte-config-config-models-0.44.0.jar:?] at io.airbyte.config.helpers.LogClientSingleton.setWorkspaceMdc(LogClientSingleton.java:213) ~[io.airbyte.airbyte-config-config-models-0.44.0.jar:?] at io.airbyte.server.LoggingEventListener.onApplicationEvent(LoggingEventListener.java:34) ~[io.airbyte-airbyte-server-0.44.0.jar:?] at io.airbyte.server.LoggingEventListener.onApplicationEvent(LoggingEventListener.java:21) ~[io.airbyte-airbyte-server-0.44.0.jar:?] at io.micronaut.context.event.ApplicationEventPublisherFactory.notifyEventListeners(ApplicationEventPublisherFactory.java:262) ~[micronaut-inject-3.8.8.jar:3.8.8] at io.micronaut.context.event.ApplicationEventPublisherFactory.access$200(ApplicationEventPublisherFactory.java:60) ~[micronaut-inject-3.8.8.jar:3.8.8] at io.micronaut.context.event.ApplicationEventPublisherFactory$2.publishEvent(ApplicationEventPublisherFactory.java:229) ~[micronaut-inject-3.8.8.jar:3.8.8] at io.micronaut.http.server.netty.NettyHttpServer.lambda$fireStartupEvents$15(NettyHttpServer.java:587) ~[micronaut-http-server-netty-3.8.8.jar:3.8.8] at java.util.Optional.ifPresent(Optional.java:178) ~[?:?] at io.micronaut.http.server.netty.NettyHttpServer.fireStartupEvents(NettyHttpServer.java:581) ~[micronaut-http-server-netty-3.8.8.jar:3.8.8] at io.micronaut.http.server.netty.NettyHttpServer.start(NettyHttpServer.java:298) ~[micronaut-http-server-netty-3.8.8.jar:3.8.8] at io.micronaut.http.server.netty.NettyHttpServer.start(NettyHttpServer.java:104) ~[micronaut-http-server-netty-3.8.8.jar:3.8.8] at io.micronaut.runtime.Micronaut.lambda$start$2(Micronaut.java:81) ~[micronaut-context-3.8.8.jar:3.8.8] at java.util.Optional.ifPresent(Optional.java:178) ~[?:?] at io.micronaut.runtime.Micronaut.start(Micronaut.java:79) ~[micronaut-context-3.8.8.jar:3.8.8] at io.micronaut.runtime.Micronaut.run(Micronaut.java:323) ~[micronaut-context-3.8.8.jar:3.8.8] at io.micronaut.runtime.Micronaut.run(Micronaut.java:309) ~[micronaut-context-3.8.8.jar:3.8.8] at io.airbyte.server.Application.main(Application.java:15) ~[io.airbyte-airbyte-server-0.44.0.jar:?]
I have created a serviceaccount with attached IAM policy to access the desired S3 bucket and updated the values.yaml file accordingly Attaching the values.yaml file for reference any help will be really appreciated
values.yaml_shared
a
airbyte is getting a blank string for the logs access key. I think you need to define either the key as a password or via existing secret in your values file, not both
a
@Adam Bloom I do not have aws access key and secret key I am trying to access the bucket through instance profile as I have created a service account with attached IAM policy for the same In the values.yaml I have kept password and existing keys as blank (“”) but the issue still persists
@kapa.ai
k
A message from kapa.ai
Sorry @Aman Kesharwani, I seem to be having a bit of trouble. Don't worry though, I've pinged my support folks and they will try to fix me up! 🧑‍💻 Perhaps try again in 5mins?
a
@kapa.ai
k
A message from kapa.ai
Sorry @Aman Kesharwani, I seem to be having a bit of trouble. Don't worry though, I've pinged my support folks and they will try to fix me up! 🧑‍💻 Perhaps try again in 5mins?
a
@Aman Kesharwani I don't think instance profiles are supported for log/state storage. The env vars are explicitly checked in the application code to be non-blank
a
@Adam Bloom is there any way we can setup the airbyte without using access keys ?
a
@Aman Kesharwani I'm just another user - don't have much extra knowledge to offer. You could use minio instead of S3? I'd certainly switch to IAM profiles over access keys if it were an option
a
@Adam Bloom yes I have created service account with attached IAM role which allows me to access S3 bucket from EKS cluster I have updated the values.yaml file as well with service account name and IAM role but the issue still persists
@Adam Bloom if I use external minio instead of S3 I get below error
Copy code
Collecting content into /tmp/toBePublished6209292906792156831.tmp before uploading.
Cannot start publish with com.van.logging.aws.S3PublishHelper@703961dc due to error: Cannot start publishing: Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: null; S3 Extended Request ID: null; Proxy: null)
Publishing to S3 (bucket=sample-bucket; key=job-logging/workspace/66cd15e7-0f0c-4a7c-89be-79863ba48b6b/0/logs.log/20230427075426_int-qa-airbyte-worker-59ff685c8f-s76xt_dcce21db15244bf8a191652ceb5710f5):
java.lang.RuntimeException: Cannot publish to S3: Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: null; S3 Extended Request ID: null; Proxy: null)
a
I've never used minio, I don't know how to help debug that.
j
@Adam Bloom hey any chance you can share the 
.env
vars you are using? i have looked at docs and they are a bit unclear.. im running into the same micronaut issue.
a
@Sunny Hashmi @John Wasserman A lot of us are running into the same issue, any help or workaround will be really appreciated
Open in Slack
PreviousNext
Powered by