https://linen.dev logo
Join SlackCommunities
Powered by
I'm still having an issue with Snowflake's destina...
# ask-community-for-troubleshooting
l
I'm still having an issue with Snowflake's destination. It gives me a "Non-Json response" error. I can set up Snowflake source without any issues.
s
Hey @Leo G are there any logs you can provide? What version of airbyte are you using, what version connector are you using for snowflake? Anything specific about your setup?
l
I'm using the latest version. I pulled it from git a couple of days ago. There is nothing special, I think. As I said I can create the Snowflake source and test it fine but when I try to make the same instance I get this "non-json" error while the testing process. I can try to pull logs from Setting-> Configuration for server logs. Is that the logs you want me to post?
Here are some java errors in the log
2022-11-03 143541 [33mWARN[m i.a.s.s.AirbyteGithubStore(getLatestSources):69 - Unable to retrieve latest Source list from Github. Using the list bundled with Airbyte. This warning is expected if this Airbyte cluster does not have internet access. javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:851) ~[java.net.http:?] at jdk.internal.net.http.HttpClientFacade.send(HttpClientFacade.java:123) ~[java.net.http:?] at io.airbyte.server.services.AirbyteGithubStore.getFile(AirbyteGithubStore.java:83) ~[io.airbyte-airbyte-server-0.40.17.jar:?] at io.airbyte.server.services.AirbyteGithubStore.getLatestSources(AirbyteGithubStore.java:67) ~[io.airbyte-airbyte-server-0.40.17.jar:?]
I think I found something.
Copy code
2022-11-02 15:43:54 INFO i.a.c.i.LineGobbler(voidCall):114 - airbyte/destination-snowflake:0.4.38 not found locally. Attempting to pull the image...
2022-11-02 15:48:53 ERROR i.t.i.s.WorkflowExecuteRunnable(logWorkflowExecutionException):125 - Workflow execution failure WorkflowId=9ff7dd03-33d9-45a9-9acd-1e221261410b, RunId=1b705d4d-b3c7-4a9b-b553-860ba5f2073c, WorkflowType=CheckConnectionWorkflow
io.temporal.failure.ActivityFailure: scheduledEventId=6, startedEventId=7, activityType='RunWithJobOutput', activityId='bb22476c-3d0b-3286-8106-215581463313', identity='', retryState=RETRY_STATE_NON_RETRYABLE_FAILURE
@Sajarin Dider Any suggestions?
s
Hey @Leo G, update to the latest version of AIrbyte and retry the setup. Secondly, could you tell me more about how you deployed your airbyte instance? Not sure why you're getting SSLHandshake exceptions.
l
@Sajarin Dider I downloaded the docker desktop and followed
git clone https://github.com/airbytehq/airbyte.git cd airbyte docker-compose up
I had to use
Copy code
git -c http.sslverify=false clone <https://github.com/airbytehq/airbyte.git>
s
Are you using wsl2?
l
About WSL2. I think I'm as it was part of the Docker desktop install. My work laptop is very limited admin rights-wise.
The SSL errror does not make sense because source connector uses SSL as well. Am I right?
Is there a stable branch?
s
You can try downgrading to v0.40.14 but I’m not sure how effective that would be. The setup seems to fail when it tries to fetch the latest image for the snowflake destination, which implies a network issue. I’m wondering if when you ran the snowflake source connector, if it also repeated the same “airbyte/source-snowflake not found locally, attempting to pull image” message.
l
If I want to start over then how do I do it?
s
Remove the repo/directory, restart docker and then try the steps for local deployment again
l
So I see snowflake source but not destination.
image.png
If I look inside \\wsl$\docker-desktop where should see if snowflake destination has been downloaded?
I re-did the whole thing and I see some error. ERROR i.m.s.DefaultTaskExceptionHandler(handle):47 - Error invoking scheduled task for bean [io.airbyte.cron.selfhealing.Temporal@7d1c8878] NOT_FOUND: Namespace name "default" not found
@Sajarin Dider
Now logs show this airbyte-worker | 2022-11-03 201036 INFO i.a.c.i.LineGobbler(voidCall):114 - airbyte/destination-snowflake:0.4.38 not found locally. Attempting to pull the image... airbyte-cron | 2022-11-03 201051 INFO i.a.c.s.DefinitionsUpdater(updateDefinitions):54 - Connector definitions update disabled.
Now I can't create snowflake source connection either.
s
Can you post the full logs?
What version of Airbyte are you running?
l
How do I post logs?
the latest 0.40.18
Now it downloaded snowflake source but still no snowflake destination. Can I pull it manually?
Here is the latest log for snowflake destination.
Copy code
airbyte-worker | 2022-11-03 22:12:12 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- START CHECK -----
airbyte-worker | 2022-11-03 22:12:12 INFO i.a.c.i.LineGobbler(voidCall):114 - 
airbyte-worker | 2022-11-03 22:12:12 INFO i.a.c.i.LineGobbler(voidCall):114 - Checking if airbyte/destination-snowflake:0.4.38 exists...
airbyte-worker | 2022-11-03 22:12:12 INFO i.a.c.i.LineGobbler(voidCall):114 - airbyte/destination-snowflake:0.4.38 not found locally. Attempting to pull the image...
airbyte-temporal | {"level":"info","ts":"2022-11-03T22:12:16.038Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/CONNECTION_UPDATER/1","wf-task-queue-type":"Workflow","lifecycle":"Starting","logging-call-at":"taskQueueManager.go:238"}
airbyte-temporal | {"level":"info","ts":"2022-11-03T22:12:16.038Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/CONNECTION_UPDATER/1","wf-task-queue-type":"Workflow","lifecycle":"Started","logging-call-at":"taskQueueManager.go:242"}
airbyte-server | 2022-11-03 22:12:21 INFO i.a.s.RequestLogger(filter):112 - REQ 172.18.0.4 GET 200 /api/v1/health
airbyte-webapp | 172.18.0.9 - airbyte [03/Nov/2022:22:12:21 +0000] "GET /api/v1/health HTTP/1.0" 200 18 "<http://localhost:8000/workspaces/fbbbb533-43bf-4bdd-b0d9-38f8b0665b5e/destination/new-destination>" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36 Edg/107.0.1418.26" "172.18.0.1"
airbyte-proxy | 172.18.0.1 - airbyte [03/Nov/2022:22:12:21 +0000] "GET /api/v1/health HTTP/1.1" 200 18 "<http://localhost:8000/workspaces/fbbbb533-43bf-4bdd-b0d9-38f8b0665b5e/destination/new-destination>" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36 Edg/107.0.1418.26"
airbyte-server | 2022-11-03 22:12:29 INFO i.a.s.RequestLogger(filter):112 - REQ 172.18.0.4 GET 200 /api/v1/health
airbyte-webapp | 172.18.0.9 - airbyte [03/Nov/2022:22:12:29 +0000] "GET /api/v1/health HTTP/1.0" 200 18 "<http://localhost:8000/workspaces/fbbbb533-43bf-4bdd-b0d9-38f8b0665b5e/destination/new-destination>" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36 Edg/107.0.1418.26" "172.18.0.1"
airbyte-proxy | 172.18.0.1 - airbyte [03/Nov/2022:22:12:29 +0000] "GET /api/v1/health HTTP/1.1" 200 18 "<http://localhost:8000/workspaces/fbbbb533-43bf-4bdd-b0d9-38f8b0665b5e/destination/new-destination>" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36 Edg/107.0.1418.26"
airbyte-temporal | {"level":"info","ts":"2022-11-03T22:12:32.274Z","msg":"Get dynamic config","name":"frontend.sendRawWorkflowHistory","value":false,"default-value":false,"logging-call-at":"config.go:79"}
airbyte-cron | 2022-11-03 22:12:33 INFO i.a.c.s.DefinitionsUpdater(updateDefinitions):54 - Connector definitions update disabled.
@Sajarin Dider Any other ideas?
@Sajarin Dider What does "Connector definitions update disabled."?
Copy code
airbyte-worker      | 2022-11-07 17:57:35 INFO i.a.w.t.TemporalAttemptExecution(get):141 - Executing worker wrapper. Airbyte version: 0.40.18
airbyte-worker      | 2022-11-07 17:57:35 INFO i.a.c.i.LineGobbler(voidCall):114 -
airbyte-worker      | 2022-11-07 17:57:35 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- START CHECK -----
airbyte-worker      | 2022-11-07 17:57:35 INFO i.a.c.i.LineGobbler(voidCall):114 -
airbyte-worker      | 2022-11-07 17:57:35 INFO i.a.c.i.LineGobbler(voidCall):114 - Checking if airbyte/destination-snowflake:0.4.38 exists...
airbyte-worker      | 2022-11-07 17:57:35 INFO i.a.c.i.LineGobbler(voidCall):114 - airbyte/destination-snowflake:0.4.38 not found locally. Attempting to pull the image...
airbyte-server      | 2022-11-07 17:57:44 INFO i.a.s.RequestLogger(filter):112 - REQ 172.18.0.4 GET 200 /api/v1/health
airbyte-webapp      | 172.18.0.9 - airbyte [07/Nov/2022:17:57:44 +0000] "GET /api/v1/health HTTP/1.0" 200 18 "<http://localhost:8000/workspaces/ae943b66-e33d-423d-8ff3-ae555cd272a0/destination/new-destination>" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36 Edg/107.0.1418.35" "172.18.0.1"
airbyte-proxy       | 172.18.0.1 - airbyte [07/Nov/2022:17:57:44 +0000] "GET /api/v1/health HTTP/1.1" 200 18 "<http://localhost:8000/workspaces/ae943b66-e33d-423d-8ff3-ae555cd272a0/destination/new-destination>" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36 Edg/107.0.1418.35"
airbyte-cron        | 2022-11-07 17:57:54 INFO i.a.c.s.DefinitionsUpdater(updateDefinitions):54 - Connector definitions update disabled.
airbyte-server      | 2022-11-07 17:58:04 INFO i.a.s.RequestLogger(filter):112 - REQ 172.18.0.4 GET 200 /api/v1/health
airbyte-webapp      | 172.18.0.9 - airbyte [07/Nov/2022:17:58:04 +0000] "GET /api/v1/health HTTP/1.0" 200 18 "<http://localhost:8000/workspaces/ae943b66-e33d-423d-8ff3-ae555cd272a0/destination/new-destination>" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36 Edg/107.0.1418.35" "172.18.0.1"
airbyte-proxy       | 172.18.0.1 - airbyte [07/Nov/2022:17:58:04 +0000] "GET /api/v1/health HTTP/1.1" 200 18 "<http://localhost:8000/workspaces/ae943b66-e33d-423d-8ff3-ae555cd272a0/destination/new-destination>" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36 Edg/107.0.1418.35"
airbyte-cron        | 2022-11-07 17:58:24 INFO i.a.c.s.DefinitionsUpdater(updateDefinitions):54 - Connector definitions update disabled.
s
Hey @Leo G sorry for the delay in response, we’re dealing with an influx of user questions and are trying our best to reply to everyone. Not sure why Connector definitions have been disabled. Could you try downgrading to an earlier version of Airbyte? I’ve been seeing a lot of people get the non-json error. Some people have said it was due to a network issue and some people have said they started seeing the error from the latest version
l
@Saj Dider (Airbyte) Which version would you recommend?
s
Perhaps 0.40.16?
l
@Saj Dider (Airbyte) How do I get it from GitHub? Can you put the actual command? Thanks
a
I am having a similar issue , were you able to get past this issue @Leo G ?
u
Hello Leo G, it's been a while without an update from us. Are you still having problems or did you find a solution?
f
Hi @Leo G I am having the same issue. Did you find a solution?
@Marcos Marx (Airbyte) I have downgraded from 0.40.12 to 0.39.41 and it worked
s
@Leo G sorry for the long delay in response. Were you able to get this working? If not, could you try downgrading to 0.39.41? Hopefully we can get this working on our latest version soon.
✅ 1
s
Seeing the same issue in version 0.40.32 setting up the snowflake destination. Appears to have the problem of getting the image 2023-02-18 230400 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- START CHECK ----- 2023-02-18 230400 INFO i.a.c.i.LineGobbler(voidCall):114 - 2023-02-18 230400 INFO i.a.c.i.LineGobbler(voidCall):114 - Checking if airbyte/destination-snowflake:0.4.47 exists... 2023-02-18 230400 INFO i.a.c.i.LineGobbler(voidCall):114 - airbyte/destination-snowflake:0.4.47 not found locally. Attempting to pull the image... 2023-02-18 230412 ERROR i.a.w.g.DefaultCheckConnectionWorker(run):124 - Unexpected error while checking connection: io.airbyte.workers.exception.WorkerException: Could not find image: airbyte/destination-snowflake:0.4.47
The source connection to snowflake is working. Following other suggestions i downgrade to 0.39.41 and the snowflake destination is now working. Running on AWS Linux 2 git clone https://github.com/airbytehq/airbyte.git docker-compose up . Interesting the destination snowflake source is working on local development on windows 10 Ubuntu 20.04, but need to have this working on EC2. Is there a planned fix on this?
s
@Scott DeMarzio are there any logs you can provide from the EC2 instance?
s
https://github.com/airbytehq/airbyte/blob/c6b8279386cb731b4f81322d9f4515d02792b32b/docs/troubleshooting/new-connection.md I refer edto this ticket/issue about pulling the destination docker image directly. I also rebuilt the environment mkdir airbyte && cd airbyte wget https://raw.githubusercontent.com/airbytehq/airbyte-platform/main/{.env,flags.yml,docker-compose.yaml} docker compose up -d going thru this and it appears to be working now.
Thanks for the followup
268 Views
Open in Slack
PreviousNext
Powered by