Hi, I am trying to configure Redshift destination. The way our Redshift is setup, it requires

ssl=true

. How can I do this in Airbyte?

These are logs for the connection I tried to setup with parameters available on Airbyte UI.

2021-09-29 08:38:49 INFO  Executing worker wrapper. Airbyte version: 0.29.21-alpha
2021-09-29 08:38:49 INFO  on-redshift-worker-b54414b1-0c0a-4e89-ab0e-bc4d82f67c35-0-rznow stdoutLocalPort = 9034
2021-09-29 08:38:49 INFO  on-redshift-worker-b54414b1-0c0a-4e89-ab0e-bc4d82f67c35-0-rznow stderrLocalPort = 9035
2021-09-29 08:38:49 INFO  Creating stdout socket server...
2021-09-29 08:38:49 INFO  Creating pod...
2021-09-29 08:38:49 INFO  Creating stderr socket server...
2021-09-29 08:38:49 INFO  Waiting for init container to be ready before copying files...
2021-09-29 08:38:50 INFO  Init container present..
2021-09-29 08:38:51 INFO  Init container ready..
2021-09-29 08:38:51 INFO  Copying files...
2021-09-29 08:38:51 INFO  Uploading file: source_config.json
2021-09-29 08:38:52 INFO  Uploading file: FINISHED_UPLOADING
2021-09-29 08:38:52 INFO  Waiting until pod is ready...
2021-09-29 08:38:53 INFO  Setting stdout...
2021-09-29 08:38:53 INFO  Setting stderr...
2021-09-29 08:38:53 INFO  Reading pod IP...
2021-09-29 08:38:53 INFO  Using null stdin output stream...
2021-09-29 08:38:54 INFO  2021-09-29 08:38:54 [32mINFO[m i.a.i.d.r.RedshiftDestination(main):97 - {} - starting destination: class io.airbyte.integrations.destination.redshift.RedshiftDestination
2021-09-29 08:38:54 INFO  2021-09-29 08:38:54 [32mINFO[m i.a.i.b.IntegrationRunner(run):96 - {} - Running integration: io.airbyte.integrations.destination.redshift.RedshiftDestination
2021-09-29 08:38:54 INFO  2021-09-29 08:38:54 [32mINFO[m i.a.i.b.IntegrationCliParser(parseOptions):135 - {} - integration args: {check=null, config=source_config.json}
2021-09-29 08:38:54 INFO  2021-09-29 08:38:54 [32mINFO[m i.a.i.b.IntegrationRunner(run):100 - {} - Command: CHECK
2021-09-29 08:38:54 INFO  2021-09-29 08:38:54 [32mINFO[m i.a.i.b.IntegrationRunner(run):101 - {} - Integration config: IntegrationConfig{command=CHECK, configPath='source_config.json', catalogPath='null', statePath='null'}
2021-09-29 08:38:54 INFO  2021-09-29 08:38:54 [33mWARN[m c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2021-09-29 08:38:54 INFO  2021-09-29 08:38:54 [33mWARN[m c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2021-09-29 08:38:54 INFO  2021-09-29 08:38:54 [32mINFO[m i.a.i.d.j.c.SwitchingDestination(check):75 - {} - Using destination type: INSERT

Hey @Kriti (Postman) Looks like those logs you sent got cut off, could you send a file with the full logs please and I'll take a look 🙂

Just checked - this is all there is.

ALso, the file was downloaded from the UI - the download button on top right

hmm that's rather puzzling, can't see anything immediately obvious pointing to the fail

@[DEPRECATED] Marcos Marx / @Harshith (Airbyte) have you seen this happen before?

Hey Kriti are you trying to deploy to Kube?. Can you help me with more info something like this https://docs.airbyte.io/troubleshooting#slack-issue-template This template give us the minimal information to help you debugging your problem quicker.

I guess download button on UI only downloads the visible log content, not complete file.

AFAIK Redshift does not yet support SSL connections - see these issues: https://github.com/airbytehq/airbyte/issues/6429 https://github.com/airbytehq/airbyte/issues/5628 https://github.com/airbytehq/airbyte/issues/4600

Thanks @Oliver Meyer simple smile

Have been trying a hack for SSL - This is the jdbc connection string for ssl enable connections -

jdbc:<redshift://redshift-test2.abcdefg.us-east-1.redshift.amazonaws.com:5439/company?ssl=true>

In the above

company

is the database name. So if we pass

company?ssl=true

as database name, it should create the ssl enabled jdbc string.

@Harshith (Airbyte), adding more details to the logs in the initial message. Is this your first time deploying Airbyte: Yes OS Version / Instance: t3.medium Memory / Disk: - Deployment: Kubernetes Airbyte Version: 0.29.21-alpha Source name/version: - Destination name/version: Redshift (How to check version?) Step: Setting new destination

Earlier, the destination setup tests failed due to connection error. Now the process keeps running for 10 mins (maybe more) and fails without logs.

Destination name/version: Redshift (How to check version?)

Setting Page.