https://linen.dev logo
#feedback-and-requests
Title
# feedback-and-requests
k

Kriti (Postman)

09/29/2021, 9:17 AM
Hi, I am trying to configure Redshift destination. The way our Redshift is setup, it requires
ssl=true
. How can I do this in Airbyte?
u

user

09/29/2021, 9:17 AM
These are logs for the connection I tried to setup with parameters available on Airbyte UI.
Copy code
2021-09-29 08:38:49 INFO  Executing worker wrapper. Airbyte version: 0.29.21-alpha
2021-09-29 08:38:49 INFO  on-redshift-worker-b54414b1-0c0a-4e89-ab0e-bc4d82f67c35-0-rznow stdoutLocalPort = 9034
2021-09-29 08:38:49 INFO  on-redshift-worker-b54414b1-0c0a-4e89-ab0e-bc4d82f67c35-0-rznow stderrLocalPort = 9035
2021-09-29 08:38:49 INFO  Creating stdout socket server...
2021-09-29 08:38:49 INFO  Creating pod...
2021-09-29 08:38:49 INFO  Creating stderr socket server...
2021-09-29 08:38:49 INFO  Waiting for init container to be ready before copying files...
2021-09-29 08:38:50 INFO  Init container present..
2021-09-29 08:38:51 INFO  Init container ready..
2021-09-29 08:38:51 INFO  Copying files...
2021-09-29 08:38:51 INFO  Uploading file: source_config.json
2021-09-29 08:38:52 INFO  Uploading file: FINISHED_UPLOADING
2021-09-29 08:38:52 INFO  Waiting until pod is ready...
2021-09-29 08:38:53 INFO  Setting stdout...
2021-09-29 08:38:53 INFO  Setting stderr...
2021-09-29 08:38:53 INFO  Reading pod IP...
2021-09-29 08:38:53 INFO  Using null stdin output stream...
2021-09-29 08:38:54 INFO  2021-09-29 08:38:54 [32mINFO[m i.a.i.d.r.RedshiftDestination(main):97 - {} - starting destination: class io.airbyte.integrations.destination.redshift.RedshiftDestination
2021-09-29 08:38:54 INFO  2021-09-29 08:38:54 [32mINFO[m i.a.i.b.IntegrationRunner(run):96 - {} - Running integration: io.airbyte.integrations.destination.redshift.RedshiftDestination
2021-09-29 08:38:54 INFO  2021-09-29 08:38:54 [32mINFO[m i.a.i.b.IntegrationCliParser(parseOptions):135 - {} - integration args: {check=null, config=source_config.json}
2021-09-29 08:38:54 INFO  2021-09-29 08:38:54 [32mINFO[m i.a.i.b.IntegrationRunner(run):100 - {} - Command: CHECK
2021-09-29 08:38:54 INFO  2021-09-29 08:38:54 [32mINFO[m i.a.i.b.IntegrationRunner(run):101 - {} - Integration config: IntegrationConfig{command=CHECK, configPath='source_config.json', catalogPath='null', statePath='null'}
2021-09-29 08:38:54 INFO  2021-09-29 08:38:54 [33mWARN[m c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2021-09-29 08:38:54 INFO  2021-09-29 08:38:54 [33mWARN[m c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2021-09-29 08:38:54 INFO  2021-09-29 08:38:54 [32mINFO[m i.a.i.d.j.c.SwitchingDestination(check):75 - {} - Using destination type: INSERT
u

user

09/29/2021, 9:23 AM
Hey @Kriti (Postman) Looks like those logs you sent got cut off, could you send a file with the full logs please and I'll take a look 🙂
u

user

09/29/2021, 9:38 AM
Just checked - this is all there is.
u

user

09/29/2021, 9:38 AM
ALso, the file was downloaded from the UI - the download button on top right
u

user

09/29/2021, 10:00 AM
hmm that's rather puzzling, can't see anything immediately obvious pointing to the fail
k

Kriti (Postman)

09/29/2021, 10:00 AM
@[DEPRECATED] Marcos Marx / @Harshith (Airbyte) have you seen this happen before?
u

user

09/29/2021, 10:12 AM
Hey Kriti are you trying to deploy to Kube?. Can you help me with more info something like this https://docs.airbyte.io/troubleshooting#slack-issue-template This template give us the minimal information to help you debugging your problem quicker.
u

user

09/30/2021, 4:46 AM
I guess download button on UI only downloads the visible log content, not complete file.
u

user

09/30/2021, 12:13 PM
u

user

09/30/2021, 12:22 PM
Thanks @Oliver Meyer simple smile
m

Milad saidi

10/04/2021, 7:07 AM
Have been trying a hack for SSL - This is the jdbc connection string for ssl enable connections -
jdbc:<redshift://redshift-test2.abcdefg.us-east-1.redshift.amazonaws.com:5439/company?ssl=true>
In the above
company
is the database name. So if we pass
company?ssl=true
as database name, it should create the ssl enabled jdbc string.
u

user

10/04/2021, 7:11 AM
@Harshith (Airbyte), adding more details to the logs in the initial message. Is this your first time deploying Airbyte: Yes OS Version / Instance: t3.medium Memory / Disk: - Deployment: Kubernetes Airbyte Version: 0.29.21-alpha Source name/version: - Destination name/version: Redshift (How to check version?) Step: Setting new destination
u

user

10/04/2021, 7:12 AM
Earlier, the destination setup tests failed due to connection error. Now the process keeps running for 10 mins (maybe more) and fails without logs.
u

user

10/04/2021, 8:48 PM
Destination name/version: Redshift (How to check version?)
Setting Page.
8 Views