Hi all, I am using the MS SQL Server destination i...
# replication-troubleshooting
m
Hi all, I am using the MS SQL Server destination in a connection to load some JSON data from a custom source connector into an Azure SQL Server. I have set up an Incremental | Append stream in the replication settings and I have enabled the Normalized tabular table under Normalization in the Transformation Settings. After the sync succeeds I can see the raw table in my destination database and it has the JSON data I expected in the column __airbyte_data_, but there is no normalized table containing the JSON properties as columns anywhere in the database. Do I have to change something to see the normalized table? Can you help me solve this? I have attached the logs in a comment below.
✅ 1
airbyte.rtf
n
Hi Maria, if you see the raw data but not the normalized data.. but looking through the logs it looks like normalization ran successfully. Let me look into this a bit further. Has this run successfully before?
m
Thanks for the help Nataly. It has not ran successfully before. It is the first connection I am setting up with MS SQL Server as destination.
u
Could you try running this without normalization so we can establish a baseline?
m
I have executed it without normalization. The data sent to the destination database loads the raw table as well and the data in the _airbyte_data column in the raw table has the same JSON format and properties than the previous load withot normalization, but since it loaded incrementally today and there was more data today there are more rows. Here are the logs:
Hi @Nataly Merezhuk (Airbyte) I have solved my issue. The problem was coming from my own custom connector. It was not creating a JSON schema if one of my source files was empty. For that reason, the destination didn't have a schema or model to normalize the tables. I have added the JSON schema to my source's stream and the normalization works now.
n
So great to hear you've resolved it!