Hi, I am new to Airbyte and created a toy example of copying data between

google ads

and

databricks lakehouse

(deployed on minikube with helm, v0.40.25) Unfortunately, it causes an error during an execution of an SQL statement in the absence of column types (I edited it a bit for readability):

CREATE TABLE <table> (_airbyte_ab_id string, _airbyte_emitted_at string, `campaign.id` , `metrics.clicks` , `segments.date`) USING csv LOCATION '<location>' options ("header" = "true", "multiLine" = "true")

Is there something wrong with my configuration? Could you please advise what could I look into? UPD: I've made some research. Here is the json schema from my logs:

Json schema for stream usr_igsaf.google_ads_test: {"type":"object","$schema":"<http://json-schema.org/draft-07/schema#>","properties":{"campaign.id":{"type":["integer","null"]},"campaign.name":{"type":["string","null"]},"segments.date":{"type":["string","null"],"format":"date"},"metrics.clicks":{"type":["integer","null"]},"metrics.conversions":{"type":["number","null"]},"metrics.cost_micros":{"type":["integer","null"]},"metrics.impressions":{"type":["integer","null"]},"user_location_view.country_criterion_id":{"type":["integer","null"]}},"additionalProperties":true}

Looks like the code in the databricks integration is not ready to see arrays in the

type

field:

final String type = node.get("type").asText();
      schemaString.append(", `").append(header).append("` ").append(type.equals("number") ? "double" : type);

If I contribute a fix, would it take long until it gets released?

Hi Igor, thanks so much for the details! We are in a code freeze until January 3rd so nothing is getting merged till then. If you make a PR and make sure that integration tests pass, we should be able to merge quicker!

Here is the PR https://github.com/airbytehq/airbyte/pull/21238 Integration tests for azure are currently disabled, but I have run them locally on my cluster and everything went smoothly.

Thanks! I'm looking into it!

image.png