Hello!
Am new to Airbyte, I might immaturely askin...
# ask-community-for-troubleshootingo
Omar Abdullahi Ahmed
10/07/2022, 2:42 PMHello!
Am new to Airbyte, I might immaturely asking question that have been answered before bare with me.
am getting this error.
2022-10-07 13:34:02 - Additional Failure Information: ('42000', "[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]'COLUMNSTORE' is not supported in this service tier of the database. See Books Online for more details on feature support in different service tiers of Windows Azure SQL Database. (40536) (SQLExecDirectW)")✍️ 1
e
Eli Sigal
10/08/2022, 6:40 AMHi.
based on what you have said it seems like a normalization issue since you do have data in the dw.
airbyte creates raw json tables since this is good practice to have it as a data lake and format as needed.
so the fixes for both questions are managing dbt correctly.
you need to create your own custom dbt model and there normalize and change the data type
o
Omar Abdullahi Ahmed
10/08/2022, 6:55 AM@Eli Sigal How can i do that?
Omar Abdullahi Ahmed
10/08/2022, 6:56 AMis there a resource I follow along?
e
Eli Sigal
10/08/2022, 8:14 AM2 things to do:
1. Use dbt docs to reference how to use dbt and write models - https://docs.getdbt.com/docs/building-a-dbt-project/documentation
2. you need to go to where dbt is located in airbyte and change the logic accordingly
o
Omar Abdullahi Ahmed
10/09/2022, 8:03 AMI do have 100s of dbt models already, Just trying to migrate to airbyte than Azure Data Factory
Omar Abdullahi Ahmed
10/09/2022, 8:06 AMSo my question is how can I solve the error above. If it's dbt transform How can I easily do this without changing all the 100 dbt models I was using before
e
Eli Sigal
10/09/2022, 10:24 AMLike I said above- you need to create a normalization model
m
Marcelo Pio de Castro
10/10/2022, 12:32 AMAirbyte works this way:
• First it syncs the data in a raw form, the json that you saw
• If configured in the connection, it will try to normalize your data using dbt. It will generate a dbt model of your data based on the source schema and convert the json to an actual table
Your problem seems like the azure SQL database that you are using as destination does not support a SQL instruction that the airbyte generated DBT is trying to use.
Marcelo Pio de Castro
10/10/2022, 12:33 AMu
user
10/10/2022, 11:02 AM@[DEPRECATED] Marcos Marx turned this thread into Zendesk ticket 2674 to ensure timely resolution!
u
user
10/11/2022, 1:22 PMHi! Thank you for your patience, I was out of the office yesterday.
These are all excellent points, great to see the community at work here :) Omar, could you please tell me what connectors/versions you are using and also what sync mode you're using as well? It seems indeed that this is a normalization issue.
u
user
10/11/2022, 1:22 PMHi! Thank you for your patience, I was out of the office yesterday.
These are all excellent points, great to see the community at work here :) Omar, could you please tell me what connectors/versions you are using and also what sync mode you're using as well? It seems indeed that this is a normalization issue.
o
Omar Abdullahi Ahmed
10/11/2022, 2:32 PM@Marcelo Pio de Castro’s comment were quite helpful but still the same problem
How can I normalize without write dbt models for all the tables. Is there a debug solution for those stucked with normalization issues?
Omar Abdullahi Ahmed
10/11/2022, 3:18 PMtried Again with few tables
13.05 MB33,217 emitted records33,217 committed records8m 4sSyncFailure Origin: normalization, Message: Normalization failed during the dbt run. This may indicate a problem with the data itself.2022-10-11 14:24:26 - Additional Failure Information: ('42000', "[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]'COLUMNSTORE' is not supported in this service tier of the database. See Books Online for more details on feature support in differentOmar Abdullahi Ahmed
10/11/2022, 3:20 PMdelete.png
e
Eli Sigal
10/11/2022, 3:22 PMHi.
Like I thought - try to go to the json data provided in airbyte table and then look at airbyte dbt normalization code and see where it fails -
A workaround is to create your own dbt normalization model
an issue needs to be opened though
n
Nataly Merezhuk (Airbyte)
10/14/2022, 3:42 PM@Omar Abdullahi Ahmed, could you please tell me what Airbyte and connector names and versions you are using as well as the sync mode? Thanks!
r
Rafael Paraíso Rossim
12/19/2023, 4:21 PMI'm facing the same problem, using Google Analytics (Universal Analytics) v0.2.2 with MS SQL Server v0.2.0 as a source.
Do we have any solution other than creating a specific dbt model for this correction?
44 Views