https://linen.dev logo
#ask-community-for-troubleshooting
Title
# ask-community-for-troubleshooting
m

Muhammad Imtiaz

06/10/2022, 1:27 PM
Hello all, Needs urgent help; quick response will be appriciated. I've upgraded airbyte version in EKS from
0.35.31-alpha
to
0.39.17-alpha
, by following the guide given here. Essentially, I've pulled the latest code from master branch, provided all the required details in
.env
and
secrets
file and run
kubectl apply -k kube/overlays/stable
. All pods at the start run successfully i.e. there is no restart. SERVER, TEMPORAL and Worker pod's logs (files attached below) are convincing and they don't show any errors. But I'm not able to port-forward
airbyte-webapp
service locally as It is breaking connections (screenshot attached below). When I see the logs of
server-pod
, I see some of the api calls are failing (screenshot attached below). Port-farward command
Copy code
kubectl port-forward svc/airbyte-webapp-svc 8088:80 -n dev
PS: I"m using RDS postgres instance and S3 bucket. Airbyte is able to access postgres and S3 bucket.
m

Marcos Marx (Airbyte)

06/10/2022, 1:35 PM
Please keep post in one channel only and read channel description to find the best place. I deleted your other post in #public-introductions
🙏 1
m

Muhammad Imtiaz

06/10/2022, 1:37 PM
Ok. I'll take care of it in the future.
Can someone please look into this?
a

Adam Bloom

06/10/2022, 3:49 PM
@Muhammad Imtiaz looks like you're using the wrong port. It should be
8000
, not
8088
, so
kubectl port-forward svc/airbyte-webapp-svc 8000:80
. You can see this in your screenshots of airbyte webapp - it shows
Now ready at <http://localhost:8000/>
m

Muhammad Imtiaz

06/10/2022, 3:56 PM
Thanks for your reply @Adam Bloom.
8088
is my host port in the port-farward command. airbyte-web-svc is running at port
80
.
8000
is the port at which
airbyte-server
is running.
a

Adam Bloom

06/10/2022, 3:57 PM
ah, you're right. I'm clearly still waking up!
m

Muhammad Imtiaz

06/10/2022, 4:00 PM
@Adam Bloom Every other components is running without any errors. I only see these error logs in
airbyte-server
pods Don't know what they are about and how to solve them. PS: I see no
scheduler
pod in version
0.39.17-alpha
. Is there any architectural change?
@abhi @Adam Montgomerie Hello, I'm trying to upgrade airbyte version in EKS from
0.35.31-alpha
to
0.39.17-alpha
and using S3 bucket for logging. I've used the following S3 settings in .env file, keeping the rest same.
Copy code
# S3/Minio Log Configuration
S3_LOG_BUCKET=cdpaas-airbyte-qa-logs
S3_LOG_BUCKET_REGION=us-east-2
S3_MINIO_ENDPOINT=
S3_PATH_STYLE_ACCESS=
AWS_ACCESS_KEY_ID=<access-key-id>
AWS_SECRET_ACCESS_KEY=<secret-access-key-id>
When airbyte starts, it uses this bucket and pushes the files into it. But when I run the workflow, it shows the following error, Detailed error's logs are attached below. I"ve searched this error from open issues and Internet but I've found any help from there. Quick response will be appreciated.
@Harshith (Airbyte) found this thread, but haven't found any other help. https://discuss.airbyte.io/t/kubernetes-bigquery-sync-failed/1222/8
h

Harshith (Airbyte)

06/13/2022, 7:33 AM
Hey is it the same issue ?
m

Muhammad Imtiaz

06/13/2022, 7:39 AM
Yes It is the same issue except I'm not using the Bigquery destination connector. The main issue is with S3 bucket. I've just checked that with minio deployment, airbyte runs successfully. But it raises some issues with S3 for logging.
a

Augustin Lafanechere (Airbyte)

06/14/2022, 1:53 PM
Hey @Muhammad Imtiaz it will be easier to track your problems and help you if you opened a post on our forum for each problem you have. 🙏🏻
4 Views